feat: stress testing (generator + brute force) #257

New issue

Closed

opened 2026-02-27 00:20:23 +00:00 by barrettruth · 0 comments

barrettruth commented

2026-02-27 00:20:23 +00:00

Problem

Stress testing is one of the most important debugging techniques in competitive programming above Div.2 level. The workflow: a random input generator produces cases, a correct-but-slow brute force solution and the optimized candidate solution are both run on each case, and outputs are compared to find a counterexample. No existing Neovim CP plugin supports this natively.

Proposed solution

Add :CP stress [GENERATOR_CMD] [BRUTE_CMD] (or configurable via hooks). The command:

Runs GENERATOR_CMD to produce stdin
Runs both the candidate solution (current file, via existing build/run infrastructure) and BRUTE_CMD on the generated input concurrently
Compares outputs (using the same normalization as regular test runs)
On mismatch: displays the failing input and output diff in the panel, stops iteration
On match: increments iteration count and loops

This reuses execute.run and the existing diff UI. A config key stress = { max_iterations = 1000 } controls the loop limit.

The most common case — Python brute force against C++ optimized — is trivially supported since BRUTE_CMD is just python brute.py.

Alternatives considered

Using oj generate-input + oj generate-output externally. This works but requires leaving Neovim and loses the inline diff visualization.

## Problem Stress testing is one of the most important debugging techniques in competitive programming above Div.2 level. The workflow: a random input generator produces cases, a correct-but-slow brute force solution and the optimized candidate solution are both run on each case, and outputs are compared to find a counterexample. No existing Neovim CP plugin supports this natively. ## Proposed solution Add `:CP stress [GENERATOR_CMD] [BRUTE_CMD]` (or configurable via hooks). The command: 1. Runs `GENERATOR_CMD` to produce stdin 2. Runs both the candidate solution (current file, via existing build/run infrastructure) and `BRUTE_CMD` on the generated input concurrently 3. Compares outputs (using the same normalization as regular test runs) 4. On mismatch: displays the failing input and output diff in the panel, stops iteration 5. On match: increments iteration count and loops This reuses `execute.run` and the existing diff UI. A config key `stress = { max_iterations = 1000 }` controls the loop limit. The most common case — Python brute force against C++ optimized — is trivially supported since `BRUTE_CMD` is just `python brute.py`. ## Alternatives considered Using `oj generate-input` + `oj generate-output` externally. This works but requires leaving Neovim and loses the inline diff visualization.