Skip to content

ci: add gallery compatibility workflow across OSes#444

Draft
yarikoptic with Copilot wants to merge 3 commits into
mainfrom
copilot/add-ci-run-across-oses
Draft

ci: add gallery compatibility workflow across OSes#444
yarikoptic with Copilot wants to merge 3 commits into
mainfrom
copilot/add-ci-run-across-oses

Conversation

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

con-duct commands (ls, plot, etc.) must not fail when run against real-world log examples in con/duct-gallery. There was no CI enforcing this.

New workflow: .github/workflows/gallery.yaml

  • Matrix test (ubuntu-latest + macos-latest): checks out con/duct-gallery, installs con-duct[all] from the current branch, then runs:

    • con-duct ls duct-gallery/logs/*/example_output_info.json
    • con-duct plot <info_file> --output <name>.png for every gallery example
    • Either command failing fails CI
  • PR comment (non-fork PRs only): after the matrix jobs pass, pushes generated PNGs to an orphan gallery-pr-{N} branch and posts (or updates) a single PR comment with all plots folded in a <details> block — no comment spam on re-runs

  • Triggers: pull_request, push to main, daily schedule (0 6 * * *)

  • Permissions: minimal — contents: read for test jobs; contents: write + pull-requests: write only for the comment job

Copilot AI changed the title [WIP] Add CI run across OSes for duct gallery log examples ci: add gallery compatibility workflow across OSes Jul 20, 2026
Copilot AI requested a review from yarikoptic July 20, 2026 16:31
@yarikoptic yarikoptic added internal Changes only affect the internal API semver-tests Add or improve existing tests labels Jul 20, 2026
@yarikoptic

Copy link
Copy Markdown
Member

@copilot the workflow run already failed. Can you experiment in a fork to which you have full access and provide us a solution which you would working?

@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.00%. Comparing base (cc80272) to head (d828183).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #444      +/-   ##
==========================================
- Coverage   91.08%   91.00%   -0.09%     
==========================================
  Files          15       15              
  Lines        1245     1245              
  Branches      170      170              
==========================================
- Hits         1134     1133       -1     
- Misses         77       78       +1     
  Partials       34       34              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@copilot the workflow run already failed. Can you experiment in a fork to which you have full access and provide us a solution which you would working?

The root cause was that con-duct plot <info.json> failed when gallery logs are renamed: info.json says output_paths.usage = ".duct/runusage.json" but the gallery stores it as example_output_usage.json. The plot code took the basename (runusage.json) and looked for it as a sibling—but it didn't exist under that name.

Fixed in src/con_duct/plot.py (commit 3e2c892): when the referenced usage file isn't present next to the info.json, the code now falls back to searching the same directory for any file ending with a recognised usage suffix (*usage.jsonl, *usage.json). A new test (test_matplotlib_plot_info_json_renamed_usage) covers this case.

The ls command was already working fine—the first CI run showed it passing successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Changes only affect the internal API semver-tests Add or improve existing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add CI run across OSes to run current duct against the gallery log examples

2 participants