Commit graph

63 commits

Author SHA1 Message Date
38cd0482f0
ci: remove unused var 2026-03-05 10:37:13 -05:00
3ecd200da7
refactor(codeforces): use separate fetches for login and submit
Problem: the single `do_login_and_submit` page action navigated between
pages within one `session.fetch` call, which was fragile and couldn't
leverage `solve_cloudflare` for the Turnstile gate on the submit page.
The submit button click also blocked on navigation completion, causing
timeouts when CF was slow to process.

Solution: split into three separate `session.fetch` calls (homepage
login check, `/enter` login, `/contest/{id}/submit`) with
`solve_cloudflare=True` on login and submit. Use `no_wait_after=True`
on the submit click with a doubled nav timeout. Extract `span.error`
text on submit failure instead of a generic timeout message.
2026-03-05 10:35:36 -05:00
1afe41103f
ci: format 2026-03-05 01:39:59 -05:00
2cdde85d36 refactor: centralize timeout constants in scrapers/timeouts.py
Problem: each scraper defined its own timeout constants
(`TIMEOUT_S`, `TIMEOUT_SECONDS`) with inconsistent values (15s vs 30s)
and browser timeouts were scattered as magic numbers (60000, 15000,
5000, 500).

Solution: introduce `scrapers/timeouts.py` with named constants for
HTTP requests, browser session/navigation/element/turnstile/settle
timeouts, and submission polling. All six scrapers now import from
the shared module.
2026-03-05 01:35:40 -05:00
f4055b071b feat(codeforces): implement submit via headless browser
Problem: Codeforces submit was a stub returning "not yet implemented".

Solution: use StealthySession (same pattern as AtCoder) to handle
Cloudflare Turnstile on the login page, fill credentials, navigate to
the contest submit form, upload source via file input, and cache
cookies at `~/.cache/cp-nvim/codeforces-cookies.json` so repeat
submits skip the login entirely. Uses a single browser page action
that checks for the submit form before navigating, avoiding redundant
page loads and Turnstile challenges.
2026-03-05 01:31:30 -05:00
bad219e578 ci: format 2026-03-03 15:09:41 -05:00
90bd13580b feat(scraper): add precision extraction, start_time, and submit support
Problem: problem pages contain floating-point precision requirements and
contest start timestamps that were not being extracted or stored. The
submit workflow also needed a foundation in the scraper layer.

Solution: add extract_precision() to base.py and propagate through all
scrapers into cache. Add start_time to ContestSummary and extract it
from AtCoder and Codeforces. Add SubmitResult model, abstract submit()
method, submit CLI case with get_language_id() resolution, stdin/env_extra
support in run_scraper, and a full AtCoder submit implementation; stub
the remaining platforms.
2026-03-03 15:09:41 -05:00
b6f3398bbc fix(ci): formatting and typing 2026-02-18 14:13:37 -05:00
1162e7046b try to fix the setup 2026-02-18 14:13:37 -05:00
89c1a3c683 fix(ci): more fixes 2026-01-27 15:56:34 -05:00
5293515aca feat(scrapers): refactor 2026-01-27 14:44:08 -05:00
dfd8275421 fix: use a diff scraper for now 2025-12-08 19:46:14 -06:00
Barrett Ruth
aab211902e feat: multi-test case view 2025-11-04 21:32:40 -05:00
Barrett Ruth
fef73887e4 feat(io): multi-test case view 2025-11-04 08:15:08 -05:00
Barrett Ruth
3654748632 fix(scrapers): fix multi-test case codeforces running 2025-11-02 22:42:05 -05:00
7bfa839c84 fix(codeforces): correct problem url 2025-10-31 21:47:15 -04:00
352f98f26f fix: open problem-specific url 2025-10-15 11:00:31 -04:00
c0e175d84b feat(config): open url option 2025-10-12 16:19:02 -04:00
c509102b37 feat(tests): basic tests 2025-10-05 21:58:43 -04:00
ee88450b3b feat(scrapers): make scrapers softer 2025-10-05 13:40:56 -04:00
b9a2c7a4ff fix(scrapers): fix 2025-10-04 15:00:37 -04:00
179b333505 update pyproject 2025-10-03 22:38:24 -04:00
b8c79401da fix(scrapers/codeforces): suppress scrapling logs 2025-10-03 21:14:28 -04:00
f48acb4672 fix(scrapers/codeforces): scrape time 2025-10-03 21:06:20 -04:00
4498c4a7fa fix scrapers 2025-10-03 19:19:02 -04:00
67c23c4d69 better scraper config 2025-09-30 22:33:36 -04:00
49ba922ff7 fix(scraper): use scrapling 2025-09-30 20:16:59 -04:00
ae2f8b94cf feat: interactive problem finer-tuning 2025-09-27 10:05:58 -04:00
53562eb6a8 fix(scrapers): reorg codeforces scraper 2025-09-22 22:48:24 -04:00
db391da52c feat(scrapers): total refactor 2025-09-22 22:00:20 -04:00
eb3f7762de fix(ci): typing 2025-09-22 20:46:27 -04:00
afb15150af fix(ci): format 2025-09-21 15:11:10 -04:00
78fb4f8f4b feat(cache): cache clearing, updating and resetting 2025-09-21 15:08:55 -04:00
9761cded88 fix(scrapers): dont limit results to 100 contests 2025-09-21 14:23:48 -04:00
fe158aa65f fix(qol): remove ai-like comments 2025-09-21 14:00:38 -04:00
0dd145b71e feat(doc): make docs more concise 2025-09-21 12:06:45 -04:00
1f38dba57f fix(scrape): proper vars 2025-09-21 00:31:10 -04:00
9deedec15a fix(scraper): comments 2025-09-21 00:10:10 -04:00
7b8aae7921 fix(ci): move imports 2025-09-20 23:52:32 -04:00
8e13b8c61d feat(cses): update cses with concept of a category 2025-09-20 14:01:18 -04:00
ff9a3d1abb fix(ci): run as modukle 2025-09-19 21:20:31 -04:00
ddff996ee2 fix(ci): test 2025-09-19 21:11:12 -04:00
514761733b fix(scraper): relative python module import path 2025-09-19 20:55:06 -04:00
b12844c3a0 fix(scraper): import path 2025-09-19 20:46:34 -04:00
793063a68e feat(test_panel): integrate scraped data 2025-09-19 20:41:19 -04:00
aedbccffb4 feat(scrapers): update all scrapers to provide time & memory limit 2025-09-19 20:28:20 -04:00
ffaec3b947 fix(ci): type scrapers 2025-09-18 22:14:13 -04:00
8a6b5dc373 fix(ci): import cleanup 2025-09-18 22:03:42 -04:00
ca6f8417c0 feat: scraper cleanup 2025-09-18 21:49:25 -04:00
91e066bbd6 fix(scraper/codeforces): scrape multiple tc 2025-09-18 13:30:53 -04:00