Commit graph

114 commits

Author SHA1 Message Date
6faf9c7537
refactor(atcoder): extract submit helpers to module level; emit status events
Problem: `_submit_sync` was a deeply nested closure containing
`_solve_turnstile` and the browser-install block as further nesting.
Status events went to stderr, which `run_scraper()` silently discards.

Solution: Extract `_TURNSTILE_JS`, `_solve_turnstile`, `_ensure_browser`,
and `_submit_headless` to module level. Status events (`installing_browser`,
`checking_login`, `logging_in`, `submitting`) now print to stdout as NDJSON.
`submit()` delegates to `asyncio.to_thread(_submit_headless, ...)`.
2026-03-04 19:02:37 -05:00
90f7e9b20b
ci: scripts + format 2026-03-04 13:52:20 -05:00
8c8e49d75c
ci: scripts + format 2026-03-04 13:51:56 -05:00
Barrett Ruth
18a60da2d8
misc (#290)
fix atcoder :CP logins
propagate scraper error codes
2026-03-04 12:47:48 -05:00
7e48ba05cf feat(kattis): rewrite scraper to support real contests
Problem: scrape_contest_list paginated the entire Kattis problem database
(3000+ problems) treating each as a "contest". scrape_contest_metadata
only handled single-problem access. stream_tests_for_category_async could
not fetch tests for multiple problems in a real contest.

Solution: replace the paginated problem loop with a single GET to
/contests that returns ~150 real timed contests. Add contest-aware path
to scrape_contest_metadata that fetches /contests/{id}/problems and
returns all problem slugs; fall back to single-problem path when the ID
is not a contest. Add _stream_single_problem helper and update
stream_tests_for_category_async to fan out concurrently over all contest
problem slugs before falling back to the single-problem path.
2026-03-03 16:02:09 -05:00
de5a20c567 fix: resolve typecheck errors in cache, atcoder, cses, and usaco
Problem: lua typecheck flagged missing start_time field on ContestSummary;
ty flagged BeautifulSoup Tag/NavigableString union on csrf_input.get(),
a 3-tuple unpack where _extract_problem_info now returns 4 values in
cses.py, and an untyped list assignment in usaco.py.

Solution: add start_time? to ContestSummary LuaDoc, guard csrf_input
with hasattr check and type: ignore, unpack precision from
_extract_problem_info in cses.py callers, and use cast() in usaco.py.
2026-03-03 15:09:41 -05:00
bad219e578 ci: format 2026-03-03 15:09:41 -05:00
a75694e9e0 feat(submit): add solution submission UI
Add submit.lua that reads credentials from a local JSON store (prompting
via vim.ui.input/inputsecret on first use), reads the source file, and
delegates to scraper.submit(). Add language_ids.py with platform-to-
language-ID mappings for atcoder, codeforces, and cses.
2026-03-03 15:09:41 -05:00
4e8da84882 feat(platforms): add kattis and usaco scrapers
Add KattisScraper and USACOScraper with contest list, metadata, and
test case fetching. Register kattis and usaco in PLATFORMS,
PLATFORM_DISPLAY_NAMES, and default platform configs.
2026-03-03 15:09:41 -05:00
90bd13580b feat(scraper): add precision extraction, start_time, and submit support
Problem: problem pages contain floating-point precision requirements and
contest start timestamps that were not being extracted or stored. The
submit workflow also needed a foundation in the scraper layer.

Solution: add extract_precision() to base.py and propagate through all
scrapers into cache. Add start_time to ContestSummary and extract it
from AtCoder and Codeforces. Add SubmitResult model, abstract submit()
method, submit CLI case with get_language_id() resolution, stdin/env_extra
support in run_scraper, and a full AtCoder submit implementation; stub
the remaining platforms.
2026-03-03 15:09:41 -05:00
b6f3398bbc fix(ci): formatting and typing 2026-02-18 14:13:37 -05:00
1162e7046b try to fix the setup 2026-02-18 14:13:37 -05:00
89c1a3c683 fix(ci): more fixes 2026-01-27 15:56:34 -05:00
83514c453e fix(ci): remove unused import 2026-01-27 15:48:26 -05:00
d5c6783124 feat(scrapers): refactor 2026-01-27 15:43:40 -05:00
5293515aca feat(scrapers): refactor 2026-01-27 14:44:08 -05:00
dfd8275421 fix: use a diff scraper for now 2025-12-08 19:46:14 -06:00
ac51b2c799 fix(scraper): done 2025-12-08 00:20:48 -06:00
3c3e6172fc fix(ci): rename parameter for type-checking 2025-12-07 16:14:00 -06:00
06f8627331 fix: update pkgs 2025-12-07 15:38:56 -06:00
Barrett Ruth
0e778a128e Merge main into feat/io/view-togggle
Resolved conflicts:
- scrapers/atcoder.py: kept defensive if tests else '' checks
- scrapers/codechef.py: kept defensive if tests else '' checks
- tests/test_scrapers.py: kept comprehensive validation from main
- lua/cp/ui/views.lua: removed misplaced navigation code from loop
2025-11-05 23:01:04 -05:00
Barrett Ruth
e7ba6b4bb4 fix(test): update scrapers 2025-11-05 18:43:01 -05:00
Barrett Ruth
127de3d6a5 fix 2025-11-04 23:39:43 -05:00
Barrett Ruth
cea90dbda5 preliminary updates 2025-11-04 22:10:42 -05:00
Barrett Ruth
1b0d5e4d77 feat: fix typign 2025-11-04 22:08:07 -05:00
Barrett Ruth
aab211902e feat: multi-test case view 2025-11-04 21:32:40 -05:00
Barrett Ruth
fef73887e4 feat(io): multi-test case view 2025-11-04 08:15:08 -05:00
Barrett Ruth
3654748632 fix(scrapers): fix multi-test case codeforces running 2025-11-02 22:42:05 -05:00
7bfa839c84 fix(codeforces): correct problem url 2025-10-31 21:47:15 -04:00
Barrett Ruth
e89c2e1cf5 feat(codechef): finalize codechef impl 2025-10-25 01:41:55 -04:00
Barrett Ruth
2fda5a74ca feat: codechef 2025-10-25 00:26:33 -04:00
Barrett Ruth
a9ac06de83 fix(ci): regex 2025-10-23 22:24:39 -04:00
Barrett Ruth
ce975d0f1e fix(ci): regex 2025-10-23 22:24:20 -04:00
352f98f26f fix: open problem-specific url 2025-10-15 11:00:31 -04:00
c0e175d84b feat(config): open url option 2025-10-12 16:19:02 -04:00
b30c036478 fix(ci): typing 2025-10-05 23:08:54 -04:00
4fac6c8019 feat(tests): fixtures 2025-10-05 23:06:38 -04:00
2426e1cbd4 fix: scrapers 2025-10-05 22:10:26 -04:00
c509102b37 feat(tests): basic tests 2025-10-05 21:58:43 -04:00
d4df57bd05 fix(scrapers): cses interactive problems 2025-10-05 20:55:43 -04:00
25fde26943 feat(scrapers): cses soft too 2025-10-05 13:42:06 -04:00
ee88450b3b feat(scrapers): make scrapers softer 2025-10-05 13:40:56 -04:00
3fbbfa9423 normalize scraper behavior 2025-10-04 16:13:04 -04:00
b9a2c7a4ff fix(scrapers): fix 2025-10-04 15:00:37 -04:00
f929c8e826 feat(scrapers/atcoder): atcoder scraper 2025-10-03 23:26:09 -04:00
179b333505 update pyproject 2025-10-03 22:38:24 -04:00
b8c79401da fix(scrapers/codeforces): suppress scrapling logs 2025-10-03 21:14:28 -04:00
f48acb4672 fix(scrapers/codeforces): scrape time 2025-10-03 21:06:20 -04:00
33cc2ca36b fix(scrapers/cses): rename scraper 2025-10-03 19:29:10 -04:00
4498c4a7fa fix scrapers 2025-10-03 19:19:02 -04:00