Skip to content

ci: report frontend test coverage on pull requests#10273

Merged
mscolnick merged 1 commit into
mainfrom
ci/frontend-coverage-comments
Jul 22, 2026
Merged

ci: report frontend test coverage on pull requests#10273
mscolnick merged 1 commit into
mainfrom
ci/frontend-coverage-comments

Conversation

@mscolnick

Copy link
Copy Markdown
Contributor

What & why

Frontend test coverage was never measured or reported anywhere. The backend already publishes coverage via Codecov (test_be.yaml); per maintainer decision the frontend gets PR comments only — no Codecov wiring for the frontend. This lands as part of the marimo-team engineering-excellence initiative to make code-health signals visible on every PR.

What was added

  • frontend/vitest.config.ts — a coverage block (v8 provider, include: ["src/**"], reportOnFailure: true, reporters text, html, json-summary, json). It is opt-in: enabled is left unset so coverage only runs when --coverage is passed. This matches the pattern used across the org's codemirror repos.
  • frontend/package.json — new test:coverage script (vitest run --coverage) and dev dep @vitest/coverage-v8 pinned to 3.2.4 to exactly match the locked vitest (avoids a peer-dependency mismatch). Lockfile updated (additive only).
  • turbo.json — a test:coverage task mirroring test (dependsOn: ["build"], outputs: ["coverage/**"]).
  • .github/workflows/test_fe.yaml — the test step now runs pnpm turbo test:coverage, and a SHA-pinned davelosert/vitest-coverage-report-action@…v2 step posts the coverage comment. The test_frontend job gains pull-requests: write.
  • frontend/.gitignore — ignore coverage/.

Design choice: coverage in CI only

pnpm test / pnpm turbo test are unchanged — they still run plain vitest, so local runs and the default CI test path keep their current speed and behavior. Coverage instrumentation (which is not free on a repo this size) is gated behind the explicit --coverage flag via the separate test:coverage script, which CI invokes. reportOnFailure: true means the comment still appears when tests fail.

Fork-PR handling

marimo gets many PRs from forks, where GITHUB_TOKEN is downgraded to read-only and posting a comment 403s. The report step uses if: always() + continue-on-error: true so external contributors' checks never go red because of the comment (same pattern just adopted in marimo-lsp#634).

Local verification

  • pnpm install — clean; only @vitest/coverage-v8 (+ its transitive deps) added to the lockfile; pnpm dedupe --check passes.
  • Ran the CI coverage command in frontend/ on a subset (vitest run --coverage src/utils/__tests__/arrays.test.ts) — the full frontend suite is too slow to run to completion locally, so a subset was used to validate wiring and output path.
  • Confirmed coverage/coverage-summary.json and coverage/coverage-final.json land in frontend/coverage/ — exactly where the workflow points (working-directory: ./frontend + coverage/coverage-summary.json, which the action resolves via path.resolve).
  • oxlint and oxfmt clean on the changed config; git tree clean (coverage/ is now ignored).

Scope: only test_fe.yaml, the vitest/turbo config, package.json (+lockfile), and .gitignore. Backend workflows and Codecov config are untouched.

Copilot AI review requested due to automatic review settings July 22, 2026 15:06
@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Jul 22, 2026 3:07pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@mscolnick mscolnick added the internal A refactor or improvement that is not user facing label Jul 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report for ./frontend

Status Category Percentage Covered / Total
🔵 Lines 76.82% 76787 / 99951
🔵 Statements 76.82% 76787 / 99951
🔵 Functions 71.42% 655 / 917
🔵 Branches 80.23% 4483 / 5587
File CoverageNo changed files found.
Generated in workflow #19787 for commit ef86898 by the Vitest Coverage Report Action

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Architecture diagram
sequenceDiagram
    participant Dev as Developer (PR)
    participant GHA as GitHub Actions (test_fe.yaml)
    participant Turbo as Turbo (monorepo runner)
    participant Vitest as Vitest w/ Coverage
    participant FS as Coverage Files (coverage/*.json)
    participant ReportAction as vitest-coverage-report-action
    participant GH_API as GitHub API (PR Comment)

    Note over Dev,GH_API: PR Coverage Report Flow

    Dev->>GHA: Push PR / update branch
    GHA->>GHA: Trigger test_frontend job
    Note over GHA: permissions: pull-requests: write

    GHA->>Turbo: NEW: pnpm turbo test:coverage
    Turbo->>Turbo: dependsOn: ["build"] (cached or fresh)
    Turbo->>Vitest: vitest run --coverage
    Note over Vitest: Coverage only runs with --coverage flag (opt-in)
    Vitest->>FS: Write coverage/coverage-summary.json, coverage-final.json, etc.
    Note over Vitest: reportOnFailure: true → written even on test failure

    alt Tests pass or fail (always())
        GHA->>ReportAction: NEW: Run davelosert/vitest-coverage-report-action
        Note over ReportAction: continue-on-error: true
        ReportAction->>FS: Read coverage JSON files
        FS-->>ReportAction: summary & final coverage data

        alt Fork PR (GITHUB_TOKEN read-only)
            ReportAction->>GH_API: POST PR comment
            GH_API-->>ReportAction: 403 Forbidden
            Note over ReportAction: continue-on-error → check not marked red
        else Non-fork PR (GITHUB_TOKEN write)
            ReportAction->>GH_API: POST PR comment
            GH_API-->>ReportAction: 201 Created
            Note over GH_API: Coverage report appears on PR
        end
    end

    Note over GHA: Optional: Summarize typecheck failure (unchanged)
Loading

Re-trigger cubic

@mscolnick
mscolnick merged commit cb0c935 into main Jul 22, 2026
31 of 32 checks passed
@mscolnick
mscolnick deleted the ci/frontend-coverage-comments branch July 22, 2026 16:59
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Development release published. You may be able to view the changes at https://marimo.app?v=0.23.15-dev55

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies internal A refactor or improvement that is not user facing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants