fix: replace curl_cffi with scrapling in codeforces metadata
Problem: `codeforces.py` used `curl_cffi` to bypass Cloudflare when fetching contest problem HTML, making it unavailable in the nix python env and requiring an extra dependency. Solution: rewrite `_fetch_problems_html` to use scrapling `StealthySession` with `solve_cloudflare=True`, matching the existing CF submit pattern. Extend `needs_browser` in `scraper.lua` to route CF `metadata` and `tests` through the FHS env on NixOS. Remove `curl-cffi` from `pyproject.toml`, `flake.nix`, and test mocks.
This commit is contained in:
parent
543480a4fe
commit
297c71e7c7
5 changed files with 30 additions and 35 deletions
|
|
@ -45,6 +45,7 @@ local function run_scraper(platform, subcommand, args, opts)
|
|||
end
|
||||
|
||||
local needs_browser = subcommand == 'submit' or subcommand == 'login'
|
||||
or (platform == 'codeforces' and (subcommand == 'metadata' or subcommand == 'tests'))
|
||||
|
||||
if needs_browser then
|
||||
utils.setup_nix_submit_env()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue