fix: replace curl_cffi with scrapling in codeforces metadata #334

Merged

barrettruth merged 3 commits from fix/curl-cffi into main

2026-03-06 18:25:44 +00:00

barrettruth commented

2026-03-06 18:18:27 +00:00

Problem

codeforces.py used curl_cffi to bypass Cloudflare when fetching contest problem HTML, making it unavailable in the nix python env and requiring an extra dependency across pyproject.toml and flake.nix.

Solution

Rewrite _fetch_problems_html to use scrapling StealthySession with solve_cloudflare=True, matching the existing CF submit pattern. Extend needs_browser in scraper.lua to route CF metadata and tests through the FHS env on NixOS. Remove curl-cffi from pyproject.toml, flake.nix, and test mocks.

## Problem `codeforces.py` used `curl_cffi` to bypass Cloudflare when fetching contest problem HTML, making it unavailable in the nix python env and requiring an extra dependency across `pyproject.toml` and `flake.nix`. ## Solution Rewrite `_fetch_problems_html` to use scrapling `StealthySession` with `solve_cloudflare=True`, matching the existing CF submit pattern. Extend `needs_browser` in `scraper.lua` to route CF `metadata` and `tests` through the FHS env on NixOS. Remove `curl-cffi` from `pyproject.toml`, `flake.nix`, and test mocks.