test: fix failing npm test after gh-aw v0.81.0 recompile by lpcox · Pull Request #5450 · github/gh-aw-firewall

lpcox · 2026-06-23T22:13:27Z

Summary

Fixes the failing npm test caused by stale version assertions in the CI workflow tests. The v0.81.0 recompile (#5447) bumped pinned versions, but three workflow tests still asserted the old values.

Failing tests fixed

scripts/ci/security-guard-workflow.test.ts
scripts/ci/export-audit-workflow.test.ts
scripts/ci/test-coverage-improver-workflow.test.ts

Changes

Updated the stale assertions to match the recompiled lock files:

github/gh-aw-actions/setup: @c20f9e75… # v0.80.6 → @3c7f3b6f423dd721e2f115b7c8fda65287e1f137 # v0.81.0
ghcr.io/github/github-mcp-server: v1.3.0 → v1.4.0
COPILOT_DUMMY_BYOK: now the offline placeholder dummy-byok-key-for-offline-mode (gh-aw v0.81.0 changed this; it is distinct from AWF's own COPILOT_PLACEHOLDER_TOKEN). The assertions were flipped, keeping a meaningful negative check that the old ghu_ placeholder is gone.

Verification

The three suites pass; full npm test is green apart from src/services/agent-volumes-mounts.test.ts, a pre-existing flaky test (passes in isolation; fails only under full-suite parallelism due to env leakage) — unrelated to this change.
tsc build passes.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

The v0.81.0 recompile (#5447) bumped the setup action to v0.81.0 (SHA 3c7f3b6f), github-mcp-server to v1.4.0, and changed the offline COPILOT_DUMMY_BYOK placeholder to 'dummy-byok-key-for-offline-mode'. Update the stale assertions in the security-guard, export-audit, and test-coverage-improver workflow tests to match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-06-23T22:15:16Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	98.04%	98.08%	📈 +0.04%
Statements	97.97%	98.01%	📈 +0.04%
Functions	99.51%	99.51%	➡️ +0.00%
Branches	93.74%	93.77%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/workdir-setup.ts`	92.7% → 94.5% (+1.82%)	92.7% → 94.5% (+1.82%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot

Pull request overview

Updates CI workflow unit tests to align with the gh-aw v0.81.0 recompiled workflow lock files, fixing failures caused by stale version/token assertions.

Changes:

Update pinned github/gh-aw-actions/setup assertion to the v0.81.0 commit SHA in relevant workflow tests.
Update ghcr.io/github/github-mcp-server image assertion from v1.3.0 to v1.4.0.
Update COPILOT_DUMMY_BYOK assertion in the security-guard workflow test to expect the new offline placeholder value.

Show a summary per file

File	Description
scripts/ci/test-coverage-improver-workflow.test.ts	Updates lock-file assertions for setup action pin and github-mcp-server image version.
scripts/ci/security-guard-workflow.test.ts	Updates lock-file assertions for COPILOT_DUMMY_BYOK placeholder and pinned dependency versions.
scripts/ci/export-audit-workflow.test.ts	Updates lock-file assertion for github-mcp-server image version.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 3/3 changed files
Comments generated: 0

github-actions · 2026-06-23T22:22:39Z

✅ Smoke Claude passed

github-actions · 2026-06-23T22:22:40Z

✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

github-actions · 2026-06-23T22:22:43Z

✅ Contribution Check completed successfully!

Contribution guidelines review complete: PR #5450 follows the applicable CONTRIBUTING.md requirements; no comment needed.

github-actions · 2026-06-23T22:22:43Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-06-23T22:22:48Z

🚀 Security Guard has started processing this pull request

github-actions · 2026-06-23T22:22:49Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-06-23T22:22:49Z

✅ Build Test Suite completed successfully!

github-actions · 2026-06-23T22:22:49Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-06-23T22:22:55Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-06-23T22:22:57Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-06-23T22:22:57Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-06-23T22:23:00Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-06-23T22:23:28Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-06-23T22:23:38Z

✅ Smoke Gemini completed. All facets verified. 💎

github-actions · 2026-06-23T22:26:21Z

🔬 Smoke Test Results

Test	Status
GitHub MCP connectivity	✅
github.com HTTP	✅ (200)
File write/read	❌ (pre-step template vars unresolved)

Overall: FAIL

PR: test: fix failing npm test after gh-aw v0.81.0 recompile — author @lpcox

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-23T22:26:58Z

🔥 Smoke Test: Copilot PAT — PASS

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP (200)	✅
File write/read	✅

Overall: PASS · Auth mode: PAT (COPILOT_GITHUB_TOKEN)

/cc @lpcox

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-23T22:27:18Z

🧪 Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌ NO
Node.js	v24.16.0	v22.22.3	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environment.

Tested by Smoke Chroot

github-actions · 2026-06-23T22:27:20Z

🔬 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Details
Module Loading	✅	`otel.js` loads cleanly; exports 14 symbols: `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled`, plus internal test helpers
Test Suite	✅	59 tests passed, 0 failed across `otel.test.js` + `otel-fanout.test.js`
Env Var Forwarding	✅	`src/services/api-proxy-service-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME` via `pickEnvVars()`
Token Tracker Integration	✅	`onUsage` callback present in `token-tracker-http.js` (line 283); `otel.js` `setTokenAttributes()` is documented as called from this hook
OTEL Diagnostics	⏭️	No live api-proxy process during smoke test; when no endpoint is configured spans fall back to `/var/log/api-proxy/otel.jsonl` (expected behaviour)

All scenarios pass. OTEL tracing integration is healthy on this branch.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-23T22:28:38Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #5450 · 64.2 AIC · ⊞ 7.7K · ◷

github-actions · 2026-06-23T22:29:26Z

Smoke Test Results — Services Connectivity

Check	Result
Redis PING (`host.docker.internal:6379`)	❌ Timeout (RC=124)
PostgreSQL pg_isready (`host.docker.internal:5432`)	❌ No response
PostgreSQL SELECT 1	❌ No response

host.docker.internal resolves to 172.17.0.1, but all service connections failed (timeout/no response).

Overall: ❌ FAIL

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-23T22:29:34Z

@lpcox

GitHub MCP PR data ✅
GitHub.com connectivity ✅
File write/read ✅
Direct BYOK inference ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

github-actions · 2026-06-23T22:30:03Z

@lpcox

Smoke Test Results:

test: fix failing npm test after gh-aw v0.81.0 recompile: ✅
GitHub MCP connectivity: ✅
github.com HTTP: ✅
File I/O: ✅
Direct BYOK mode (Azure OpenAI, Foundry): ✅

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)

Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

github-actions · 2026-06-23T22:31:32Z

Smoke Test: Claude Engine

API check: ✅ PASS
gh CLI check: ✅ PASS
File check: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for issue #5450 · 37 AIC · ⊞ 3.1K · ◷

github-actions · 2026-06-23T22:33:21Z

chore: upgrade gh-aw to v0.81.0 and recompile workflows ✅
refactor: extract buildAgentSecurityConfig from buildAgentService ✅
Overall status: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-23T22:36:19Z

✅ Smoke Test: Copilot BYOK (Direct Mode) — PASS

Tests:

✅ MCP connectivity (PR fetch verified)
✅ github.com HTTP 200
✅ File write/read passing
✅ BYOK inference active (api-proxy → api.githubcopilot.com)

Mode: Direct BYOK via COPILOT_PROVIDER_API_KEY

All systems nominal.

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-23T22:37:04Z

Gemini Engine Smoke Test Results

GitHub MCP Testing: ❌ (Tools not found in environment)
GitHub.com Connectivity: ❌ (SSL error 35 / Squid intercept)
File Writing Testing: ✅ (Success)
Bash Tool Testing: ✅ (Success)

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

Copilot AI review requested due to automatic review settings June 23, 2026 22:13

Copilot started reviewing on behalf of lpcox June 23, 2026 22:14 View session

Copilot AI reviewed Jun 23, 2026

View reviewed changes

lpcox added the ready-for-aw label Jun 23, 2026

lpcox temporarily deployed to aoai-model June 23, 2026 22:22 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-pat label Jun 23, 2026

github-actions Bot added the build-test label Jun 23, 2026

lpcox temporarily deployed to aoai-model June 23, 2026 22:29 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-entra label Jun 23, 2026

lpcox temporarily deployed to aoai-model June 23, 2026 22:29 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jun 23, 2026

github-actions Bot added the smoke-claude label Jun 23, 2026

github-actions Bot mentioned this pull request Jun 23, 2026

[aw] No-Op Runs #5368

Open

github-actions Bot added the smoke-codex label Jun 23, 2026

github-actions Bot added the smoke-copilot-byok label Jun 23, 2026

lpcox merged commit c7fabe6 into main Jun 23, 2026
89 checks passed

lpcox deleted the fix/npm-test-version-pins branch June 23, 2026 22:43

github-actions Bot mentioned this pull request Jun 23, 2026

Refactor host-access validation tests to remove duplicated exit/assert scaffolding #5462

Merged

Uh oh!

Conversation

lpcox commented Jun 23, 2026

Summary

Failing tests fixed

Changes

Verification

Uh oh!

github-actions Bot commented Jun 23, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 23, 2026

🔬 Smoke Test Results

Uh oh!

github-actions Bot commented Jun 23, 2026

🔥 Smoke Test: Copilot PAT — PASS

Uh oh!

github-actions Bot commented Jun 23, 2026

🧪 Chroot Version Comparison Results

Uh oh!

github-actions Bot commented Jun 23, 2026

🔬 Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jun 23, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented Jun 23, 2026

Smoke Test Results — Services Connectivity

Overall: ❌ FAIL

Uh oh!

github-actions Bot commented Jun 23, 2026

Uh oh!

github-actions Bot commented Jun 23, 2026

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading