test coverage: submit, login, stress, scraper negative cases #281

New issue

Closed

opened 2026-03-03 20:13:14 +00:00 by barrettruth · 1 comment

barrettruth commented

2026-03-03 20:13:14 +00:00

Current coverage only reaches metadata and test-case scraping for AtCoder/CF/CSES. Missing:

Submit + login

Submit flow (mock scraper responses): credential prompt path, success/failure ndjson events, credential caching behavior
Login validation path: bad credentials do not cache, good credentials do

Stress

Compile step failure
Generator/brute not found
Mismatch detection

Scrapers

Negative cases: bad contest IDs, empty HTML, missing fields, malformed ndjson
Assertion depth: tests currently check shape only, not actual problem IDs or test case content
Kattis, USACO: no fixtures at all
`precision` field emitted in ndjson but absent from `TestsResult` model — either add it or stop emitting it

Current coverage only reaches metadata and test-case scraping for AtCoder/CF/CSES. Missing: **Submit + login** - Submit flow (mock scraper responses): credential prompt path, success/failure ndjson events, credential caching behavior - Login validation path: bad credentials do not cache, good credentials do **Stress** - Compile step failure - Generator/brute not found - Mismatch detection **Scrapers** - Negative cases: bad contest IDs, empty HTML, missing fields, malformed ndjson - Assertion depth: tests currently check shape only, not actual problem IDs or test case content - Kattis, USACO: no fixtures at all - \`precision\` field emitted in ndjson but absent from \`TestsResult\` model — either add it or stop emitting it

barrettruth commented

2026-03-05 06:07:04 +00:00

failed cli login
success
bad path

failed cli login success bad path