Commit graph

121 commits

Author SHA1 Message Date
720ae4ff76
feat(scraper): add login stubs for codechef, kattis, usaco 2026-03-05 15:00:25 -05:00
2161ad5020
feat(codeforces): implement validated login with browser session
Problem: Codeforces credentials were cached without verification,
identical to the AtCoder issue.

Solution: add `_login_headless_cf()` that loads cookies, checks for
the "Logout" link, navigates to `/enter` with Cloudflare solving if
needed, and validates success post-login. Cookies persisted only on
confirmed login.
2026-03-05 15:00:16 -05:00
077f1124d2
feat(atcoder): implement validated login with browser session
Problem: AtCoder credentials were cached without verification;
authentication failures were only surfaced during submit.

Solution: add `_login_headless()` that opens a StealthySession, checks
cookies, solves Turnstile if needed, fills the login form, and
validates success by checking for the "Sign Out" link. Cookies are
only persisted on confirmed login.
2026-03-05 15:00:06 -05:00
dd134324dc
feat(cses): implement validated login via _check_token/_web_login
Problem: CSES credentials were cached without verifying them, so bad
passwords were only discovered at submit time.

Solution: reuse `_check_token` (fast path) and `_web_login` in the new
`login()` method. Return credentials with API token on success so
subsequent submits skip re-authentication.
2026-03-05 14:59:55 -05:00
dfb648531b
feat(scraper): add LoginResult model and abstract login() interface
Problem: `:CP <platform> login` blindly caches credentials without
validating them against the platform.

Solution: add `LoginResult` to `models.py`, abstract `login()` method
and `_login_error` helper to `BaseScraper`, and wire up the `"login"`
CLI dispatch in `_run_cli_async`.
2026-03-05 14:59:46 -05:00
Barrett Ruth
a202725cc5
fix(submit): use file path over stdin; fix CF CodeMirror textarea (#305)
## Problem

After the initial submit hardening, two issues remained: source code was
read in Lua and piped as stdin to the scraper (unnecessary roundtrip
since
the file exists on disk), and CF's `page.fill()` timed out on the hidden
`textarea[name="source"]` because CodeMirror owns the editor state.

## Solution

Pass the source file path as a CLI arg instead — AtCoder calls
`page.set_input_files(file_path)` directly, CF reads it with
`Path(file_path).read_text()`. Fix CF source injection via
`page.evaluate()`
into the CodeMirror instance. Extract `BROWSER_SUBMIT_NAV_TIMEOUT` as a
per-platform `defaultdict` (CF defaults to 2× nav timeout). Save the
buffer
with `vim.cmd.update()` before submitting.
2026-03-05 14:34:14 -05:00
Barrett Ruth
127089c57f
fix(submit): harden atcoder and codeforces submit flow (#304)
## Problem

AtCoder file upload always wrote a `.cpp` temp file regardless of
language. CF submit used `solve_cloudflare=True` on the submit page,
causing a spurious "No Cloudflare challenge found" error;
`_wait_for_gate_reload` in `login_action` was dead code. Stale cookies
caused silent auth failures with no recovery path. The `uv.spawn` ndjson
path for submit had no overall timeout.

## Solution

Replace AtCoder's temp file with `page.set_input_files` using an
in-memory buffer and correct extension via `_LANGUAGE_ID_EXTENSION`.
Replace CF's temp-file/fallback dance with a direct
`textarea[name="source"]` fill and set `solve_cloudflare=False` on the
submit fetch. Add a login fast-path that skips the homepage check when
cookies exist, with automatic stale-cookie recovery via `_retried` flag
on redirect-to-login detection. Remove `_wait_for_gate_reload`. Fix
`_ensure_browser` to propagate install errors. Add a 120s kill timer to
the ndjson `uv.spawn` submit path in `scraper.lua`.
2026-03-05 11:18:34 -05:00
Barrett Ruth
6fcb5d1bbc
feat(codeforces): implement submit; cache CSES token (#300)
## Problem

Codeforces submit was a stub. CSES submit re-ran the full login flow on
every invocation (~1.5s overhead).

## Solution

**Codeforces**: headless browser submit via StealthySession (same
pattern as AtCoder). Solves Cloudflare Turnstile on login, uploads
source via file input, caches cookies at
`~/.cache/cp-nvim/codeforces-cookies.json` so repeat submits skip login.

**CSES**: persist the API token in credentials via a `credentials`
ndjson event. Subsequent submits validate the cached token with a single
GET before falling back to full login.

Also includes a vimdoc table of contents.
2026-03-05 10:37:39 -05:00
Barrett Ruth
e9f72dfbbc
feat(cses): implement submit via REST API (#299)
## Problem

CSES submit was a stub returning "not yet implemented".

## Solution

Authenticate via web login + API token bridge (POST `/login` form, then
POST `/api/login` and confirm the auth page), submit source to
`/api/courses/problemset/submissions` with base64-encoded content, and
poll for verdict. Uses the same username/password credential model as
AtCoder — no browser dependencies needed. Tested end-to-end with a real
CSES account (verdict: `ACCEPTED`).

Also updates `scraper.lua` to pass the full ndjson event object to
`on_status` and handle `credentials` events for future platform use.
2026-03-05 01:07:57 -05:00
Barrett Ruth
c194f12eee
feat(atcoder): extract submit helpers; add live status notifications (#294)
## Problem

`_submit_sync` was a 170-line nested closure with `_solve_turnstile` and
the browser-install block further nested inside it. Status events went
to
stderr, which `run_scraper()` silently discards, leaving the user with a
10–30s silent hang after credential entry. The NDJSON spawn path also
lacked stdin support, so submit had no streaming path at all.

## Solution

Extract `_TURNSTILE_JS`, `_solve_turnstile`, `_ensure_browser`, and
`_submit_headless` to module level in `atcoder.py`; status events
(`installing_browser`, `checking_login`, `logging_in`, `submitting`) now
print to stdout as NDJSON. Add stdin pipe support to the NDJSON spawn
path in `scraper.lua` and switch `M.submit` to streaming with an
`on_status` callback. Wire `on_status` in `submit.lua` to fire
`vim.notify` for each phase transition.
2026-03-04 19:27:29 -05:00
Barrett Ruth
18a60da2d8
misc (#290)
fix atcoder :CP logins
propagate scraper error codes
2026-03-04 12:47:48 -05:00
7e48ba05cf feat(kattis): rewrite scraper to support real contests
Problem: scrape_contest_list paginated the entire Kattis problem database
(3000+ problems) treating each as a "contest". scrape_contest_metadata
only handled single-problem access. stream_tests_for_category_async could
not fetch tests for multiple problems in a real contest.

Solution: replace the paginated problem loop with a single GET to
/contests that returns ~150 real timed contests. Add contest-aware path
to scrape_contest_metadata that fetches /contests/{id}/problems and
returns all problem slugs; fall back to single-problem path when the ID
is not a contest. Add _stream_single_problem helper and update
stream_tests_for_category_async to fan out concurrently over all contest
problem slugs before falling back to the single-problem path.
2026-03-03 16:02:09 -05:00
de5a20c567 fix: resolve typecheck errors in cache, atcoder, cses, and usaco
Problem: lua typecheck flagged missing start_time field on ContestSummary;
ty flagged BeautifulSoup Tag/NavigableString union on csrf_input.get(),
a 3-tuple unpack where _extract_problem_info now returns 4 values in
cses.py, and an untyped list assignment in usaco.py.

Solution: add start_time? to ContestSummary LuaDoc, guard csrf_input
with hasattr check and type: ignore, unpack precision from
_extract_problem_info in cses.py callers, and use cast() in usaco.py.
2026-03-03 15:09:41 -05:00
bad219e578 ci: format 2026-03-03 15:09:41 -05:00
a75694e9e0 feat(submit): add solution submission UI
Add submit.lua that reads credentials from a local JSON store (prompting
via vim.ui.input/inputsecret on first use), reads the source file, and
delegates to scraper.submit(). Add language_ids.py with platform-to-
language-ID mappings for atcoder, codeforces, and cses.
2026-03-03 15:09:41 -05:00
4e8da84882 feat(platforms): add kattis and usaco scrapers
Add KattisScraper and USACOScraper with contest list, metadata, and
test case fetching. Register kattis and usaco in PLATFORMS,
PLATFORM_DISPLAY_NAMES, and default platform configs.
2026-03-03 15:09:41 -05:00
90bd13580b feat(scraper): add precision extraction, start_time, and submit support
Problem: problem pages contain floating-point precision requirements and
contest start timestamps that were not being extracted or stored. The
submit workflow also needed a foundation in the scraper layer.

Solution: add extract_precision() to base.py and propagate through all
scrapers into cache. Add start_time to ContestSummary and extract it
from AtCoder and Codeforces. Add SubmitResult model, abstract submit()
method, submit CLI case with get_language_id() resolution, stdin/env_extra
support in run_scraper, and a full AtCoder submit implementation; stub
the remaining platforms.
2026-03-03 15:09:41 -05:00
b6f3398bbc fix(ci): formatting and typing 2026-02-18 14:13:37 -05:00
1162e7046b try to fix the setup 2026-02-18 14:13:37 -05:00
89c1a3c683 fix(ci): more fixes 2026-01-27 15:56:34 -05:00
83514c453e fix(ci): remove unused import 2026-01-27 15:48:26 -05:00
d5c6783124 feat(scrapers): refactor 2026-01-27 15:43:40 -05:00
5293515aca feat(scrapers): refactor 2026-01-27 14:44:08 -05:00
dfd8275421 fix: use a diff scraper for now 2025-12-08 19:46:14 -06:00
ac51b2c799 fix(scraper): done 2025-12-08 00:20:48 -06:00
3c3e6172fc fix(ci): rename parameter for type-checking 2025-12-07 16:14:00 -06:00
06f8627331 fix: update pkgs 2025-12-07 15:38:56 -06:00
Barrett Ruth
0e778a128e Merge main into feat/io/view-togggle
Resolved conflicts:
- scrapers/atcoder.py: kept defensive if tests else '' checks
- scrapers/codechef.py: kept defensive if tests else '' checks
- tests/test_scrapers.py: kept comprehensive validation from main
- lua/cp/ui/views.lua: removed misplaced navigation code from loop
2025-11-05 23:01:04 -05:00
Barrett Ruth
e7ba6b4bb4 fix(test): update scrapers 2025-11-05 18:43:01 -05:00
Barrett Ruth
127de3d6a5 fix 2025-11-04 23:39:43 -05:00
Barrett Ruth
cea90dbda5 preliminary updates 2025-11-04 22:10:42 -05:00
Barrett Ruth
1b0d5e4d77 feat: fix typign 2025-11-04 22:08:07 -05:00
Barrett Ruth
aab211902e feat: multi-test case view 2025-11-04 21:32:40 -05:00
Barrett Ruth
fef73887e4 feat(io): multi-test case view 2025-11-04 08:15:08 -05:00
Barrett Ruth
3654748632 fix(scrapers): fix multi-test case codeforces running 2025-11-02 22:42:05 -05:00
7bfa839c84 fix(codeforces): correct problem url 2025-10-31 21:47:15 -04:00
Barrett Ruth
e89c2e1cf5 feat(codechef): finalize codechef impl 2025-10-25 01:41:55 -04:00
Barrett Ruth
2fda5a74ca feat: codechef 2025-10-25 00:26:33 -04:00
Barrett Ruth
a9ac06de83 fix(ci): regex 2025-10-23 22:24:39 -04:00
Barrett Ruth
ce975d0f1e fix(ci): regex 2025-10-23 22:24:20 -04:00
352f98f26f fix: open problem-specific url 2025-10-15 11:00:31 -04:00
c0e175d84b feat(config): open url option 2025-10-12 16:19:02 -04:00
b30c036478 fix(ci): typing 2025-10-05 23:08:54 -04:00
4fac6c8019 feat(tests): fixtures 2025-10-05 23:06:38 -04:00
2426e1cbd4 fix: scrapers 2025-10-05 22:10:26 -04:00
c509102b37 feat(tests): basic tests 2025-10-05 21:58:43 -04:00
d4df57bd05 fix(scrapers): cses interactive problems 2025-10-05 20:55:43 -04:00
25fde26943 feat(scrapers): cses soft too 2025-10-05 13:42:06 -04:00
ee88450b3b feat(scrapers): make scrapers softer 2025-10-05 13:40:56 -04:00
3fbbfa9423 normalize scraper behavior 2025-10-04 16:13:04 -04:00