fix: replace curl_cffi with scrapling in codeforces metadata (#334)
## Problem `codeforces.py` used `curl_cffi` to bypass Cloudflare when fetching contest problem HTML, making it unavailable in the nix python env and requiring an extra dependency across `pyproject.toml` and `flake.nix`. ## Solution Rewrite `_fetch_problems_html` to use scrapling `StealthySession` with `solve_cloudflare=True`, matching the existing CF submit pattern. Extend `needs_browser` in `scraper.lua` to route CF `metadata` and `tests` through the FHS env on NixOS. Remove `curl-cffi` from `pyproject.toml`, `flake.nix`, and test mocks.
This commit is contained in:
parent
6a3d2fe4f8
commit
0b40e0f33e
5 changed files with 32 additions and 36 deletions
|
|
@ -19,7 +19,6 @@
|
|||
pkgs.python312.withPackages (ps: [
|
||||
ps.backoff
|
||||
ps.beautifulsoup4
|
||||
ps.curl-cffi
|
||||
ps.httpx
|
||||
ps.ndjson
|
||||
ps.pydantic
|
||||
|
|
@ -31,7 +30,6 @@
|
|||
pkgs.python312.withPackages (ps: [
|
||||
ps.backoff
|
||||
ps.beautifulsoup4
|
||||
ps.curl-cffi
|
||||
ps.httpx
|
||||
ps.ndjson
|
||||
ps.pydantic
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue