docs: add dogfood report for v3.16.0 - #2143
Conversation
…s from v3.16.0 session - benchmark scripts are .ts, not .js; note npm run benchmark alias and git fetch --tags requirement for correct version labeling - warn against running registry prune --ttl 0 against the real global registry; recommend CODEGRAPH_REGISTRY_PATH isolation instead - warn about update-graph.sh hook contamination risk from scratch files with tracked extensions - recommend copying .codegraphrc.json aside before editing it in place and testing config-driven commands from a different cwd than the target - recommend full edge-set diffs (not just counts) for incremental rebuild verification — this is what caught #2138 in the v3.16.0 session
| - **Fix applied:** none this session (repo-tooling, not product code). | ||
|
|
||
| ### BUG 2: incremental rebuild loses 10 receiver-dispatch edges via an unrelated file (Medium) | ||
| - **Issue:** [#2138](https://github.com/optave/ops-codegraph-tool/issues/2138) | ||
| - **PR:** open — too complex for this session (native Rust incremental orchestrator) |
There was a problem hiding this comment.
Unexplained file-count discrepancy between cold-start build and benchmark
The cold-start build in §2 and the engine-comparison table in §5 consistently report 993 files, but the build benchmark here shows 741 files. A reader consulting these numbers across sections will find them irreconcilable without explanation. If the benchmark script uses its own fixed input set or a different --db target, a brief parenthetical (e.g. "741 files — benchmark uses its own fixtures, not the full source tree") would prevent confusion when future sessions compare figures.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Fixed — added a parenthetical to the build-benchmark row explaining that scripts/benchmark.ts excludes tests/benchmarks/resolution/fixtures/** (per resolveBenchmarkExcludes() in scripts/lib/bench-config.ts), which is why it sweeps 741 files instead of the 993 used in the cold-start and engine-comparison sections.
…2143) Build benchmark scripts exclude tests/benchmarks/resolution/fixtures/** (hand-annotated resolution fixtures), unlike the full-repo cold-start and engine-comparison scans elsewhere in the report. Addresses Greptile review feedback flagging the unexplained discrepancy.
|
Addressed Greptile's review feedback:
|
Summary
Dogfooding report for v3.16.0 — full command sweep, incremental/staleness testing, native-vs-WASM engine comparison, release-specific feature validation, and performance benchmarks. Also includes a few self-improvements to the
/dogfoodskill itself based on lessons from this session (correct benchmark script paths, registry-isolation guidance, hook-contamination warning, full-edge-diff methodology for catching parity bugs).Bugs found during this session were filed as #2134-#2140; #2135 and #2136 were fixed in #2141 and #2142.
See
generated/dogfood/DOGFOOD_REPORT_v3.16.0.mdfor full details.