Add benchstat regression gate to CI bench job#41813
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a regression gate to the bench job in .github/workflows/cgo.yml so benchmark results are compared against a previous baseline using benchstat, failing the job when regressions exceed a threshold.
Changes:
- Add
actions: readpermission and installbenchstatin the bench job. - Download the most recent
benchmark-resultsartifact as a baseline and conditionally run a regression check. - Increase bench job timeout to accommodate the new steps.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/cgo.yml | Adds baseline artifact download + benchstat comparison to gate >10% benchmark regressions on main, and increases job timeout. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 3
- Review effort level: Low
| artifact_id=$(gh api \ | ||
| "repos/$GITHUB_REPOSITORY/actions/artifacts?name=benchmark-results&per_page=10" \ | ||
| --jq '[.artifacts[] | select((.workflow_run.head_branch // "main") == "main")] | .[0].id // empty' 2>/dev/null) |
| benchstat_out=$(benchstat baseline/bench_results.txt bench_results.txt 2>&1) | ||
| echo "$benchstat_out" |
| - name: Install benchstat | ||
| run: go install golang.org/x/perf/cmd/benchstat@latest |
|
@copilot please run the
|
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
🤖 PR Triage
Score breakdown: Impact 25 · Urgency 10 · Quality 12 Adds benchstat regression gate to CI bench job (+76/-1, 1 file). Ready, MERGEABLE, 2 reviews. Prevents silent perf regressions. Medium risk: modifies bench job gate logic (10% threshold via awk). Batch with other CI tooling changes for coordinated review.
|
|
🎉 This pull request is included in a new release. Release: |
The
benchjob incgo.ymlranmake benchand saved results as an artifact but never compared against a baseline — a 2× slowdown would merge silently.Changes
actions: readpermission — required to read artifacts from previous runs viagh apigolang.org/x/perf/cmd/benchstat@latestusing the existing Go toolchainbenchmark-resultsartifact filtered tomain-branch runs; setshas_baselineoutput; skips gracefully on first run or download failurebaseline/bench_results.txtvsbench_results.txt, writes thebenchstatdiff to the Job Summary, and fails withexit 1if any benchmark regresses >10%timeout-minutes: 20— bumped from 15 to accommodate the new stepsKey implementation notes:
awkfor float threshold comparison (nobcdependency)+15%/+15.23%)benchstatconvention:+= slower (regression),-= faster (improvement)github.ref == 'refs/heads/main', so any downloaded artifact is inherently from a main run; the jq filterselect((.workflow_run.head_branch // "main") == "main")adds an extra safeguard with a safe default