feat(presets/mermaid): audit and validate failure summary coverage #97

New issue

Closed

opened 2026-04-26 00:12:34 +00:00 by barrettruth · 0 comments

barrettruth commented

2026-04-26 00:12:34 +00:00

Follow-up to #81 and #70.

Scope

Audit the mermaid preset and implement a trustworthy short failure summary using the merged failure_summary(result, ctx) hook from #86.

Why this needs its own issue

mermaid has its own compiler output shape and failure modes. We should not mark #81 done until this preset has explicit positive and negative coverage with both automated tests and manual validation.

Acceptance criteria

Positive path: a representative valid mermaid document compiles successfully, no failure summary is shown, and stale diagnostics/quickfix clear after success.
Primary negative path: the canonical mermaid failure produces the intended one-line summary, and the exact summary string is asserted in automated tests.
Noisy negative path: boilerplate, wrapped output, repeated errors, or surrounding CLI noise still yield the right summary instead of truncated or misleading text.
Fallback path: when no trustworthy mermaid summary is available, behavior falls back cleanly to the generic failure or :Preview output hint path; never empty, never misleading.
Raw output remains available through :Preview output and preview.result().
Automated coverage lands in repo specs/fixtures for both success and failure behavior.
Manual validation is run in nix develop .#presets against the real toolchain, and the validated output shapes are recorded in the PR.

Representative cases to cover

parse errors such as Expecting ... got ...
source excerpts and surrounding CLI noise must not replace the real parse message
successful SVG generation must not leave stale failure messaging behind

Follow-up to #81 and #70. ## Scope Audit the `mermaid` preset and implement a trustworthy short failure summary using the merged `failure_summary(result, ctx)` hook from #86. ## Why this needs its own issue `mermaid` has its own compiler output shape and failure modes. We should not mark #81 done until this preset has explicit positive and negative coverage with both automated tests and manual validation. ## Acceptance criteria - [ ] Positive path: a representative valid `mermaid` document compiles successfully, no failure summary is shown, and stale diagnostics/quickfix clear after success. - [ ] Primary negative path: the canonical `mermaid` failure produces the intended one-line summary, and the exact summary string is asserted in automated tests. - [ ] Noisy negative path: boilerplate, wrapped output, repeated errors, or surrounding CLI noise still yield the right summary instead of truncated or misleading text. - [ ] Fallback path: when no trustworthy `mermaid` summary is available, behavior falls back cleanly to the generic failure or `:Preview output` hint path; never empty, never misleading. - [ ] Raw output remains available through `:Preview output` and `preview.result()`. - [ ] Automated coverage lands in repo specs/fixtures for both success and failure behavior. - [ ] Manual validation is run in `nix develop .#presets` against the real toolchain, and the validated output shapes are recorded in the PR. ## Representative cases to cover - parse errors such as `Expecting ... got ...` - source excerpts and surrounding CLI noise must not replace the real parse message - successful SVG generation must not leave stale failure messaging behind