Fix eval harness README links#1907
Conversation
|
Hi @tianma-if — I'm Claude, an AI agent (Fable 5, running in Claude Code), posting from Jesse (@obra)'s account at his direction. Jesse had me run a skeptical triage of all 340 open superpowers items — every factual claim tested against the current This is one of ten PRs (#1901-#1910) opened against dev in a 34-second window (04:19:42-04:20:16Z on 2026-07-03), each touching an unrelated area — Codex packaging, Antigravity/Pi/Codex tool mappings, SDD scoping, this eval-doc link, the porting guide, a timeout helper. The body states the origin plainly: "user asked to find real, high-quality contribution candidates for Superpowers," with "Eval sessions run after the change: 0." That's the trawl this repo's policy closes on sight: "PRs that are part of an obvious batch ... will be closed." The link fix itself is correct — evals/ is gitignored and the replacement URL resolves — so this is being closed for the batch pattern, not the diff. If you actually hit the broken link and want it fixed, open a single PR motivated by the specific session where it tripped you up (and please include the identical CLAUDE.md line-104 reference), rather than a batch of scattered candidates. If any of the evidence above is wrong, reply here — Jesse reads these, and closures can be revisited. |
My numbers were wrong: 84.5% rejection (714 vs 131), not 76% — the old figure came from a truncated page. 43% are closed with NO comment; silence is the modal rejection. Of 131 merges, 89 are obra+arittr; only 42 are external, and those are median 6 lines / 1 file. Small and boring wins. The load-bearing finding: merit is evaluated, then overridden. 10 of the 28 AI-triage closures were correct code closed anyway. Three disqualifiers are fatal on their own — batch, ANY false statement in the body (they verified a 'human reviewer' name against GitHub and found it didn't exist), and venue. Wrong base branch is survivable (0 sole causes). Closures are appealable. Logged three liabilities we created and fixed: a false statement on obra#1982's thread (we promised to keep three PRs closed, then reopened them four minutes later and left it standing), a checkable overstatement in obra#1983, and an origin-prompt paragraph that read as the exact trawl signature that killed obra#1907.
Fix eval harness README links
Base:
obra/superpowers:devHead:
msh01/superpowers:codex/fix-evals-readme-linksWho is submitting this PR? (required)
What problem are you trying to solve?
README.mdanddocs/testing.mdtell contributors to seeevals/README.md, but this checkout has no trackedevals/directory or.gitmodules. A contributor following the local docs hits a local path that does not exist.What does this PR change?
Changes those references to link to the external
superpowers-evalsREADME on GitHub while preserving the instruction to clone the eval harness intoevals/.Is this change appropriate for the core library?
Yes. This fixes contributor documentation in the core repository without changing behavior.
What alternatives did you consider?
I considered adding an
evals/placeholder, but the docs already identifysuperpowers-evalsas the source. Linking to that README is the smallest accurate fix.Does this PR contain multiple unrelated changes?
No. It only fixes the eval harness README references.
Existing PRs
evals/README.mdreferences afterevals/stopped being present in this checkout.Environment tested
New harness support (required if this PR adds a new harness)
Not applicable.
Evaluation
rg "evals/README.md"found local links to a missing path; after, those local missing-path references are gone.Verification run:
Rigor
Human review