Split config-assembly validator tests by concern group by Copilot · Pull Request #5348 · github/gh-aw-firewall

Copilot · 2026-06-21T15:58:59Z

src/commands/validators/config-assembly.test.ts had grown into a single 900+ line test file covering unrelated validation paths. This change separates the coverage by concern so security-sensitive and model-detection cases are easier to navigate and target independently.

Test file split
- Moved the original nested describe blocks into focused files:
  - config-assembly-docker-host.test.ts
  - config-assembly-host-access.test.ts
  - config-assembly-api-proxy.test.ts
  - config-assembly-model-detection.test.ts
  - config-assembly-flags.test.ts
- Removed the original monolithic config-assembly.test.ts
Shared test harness
- Extracted the common mocks, fixture builders, temp-dir lifecycle, and assembleAndValidateConfig helpers into config-assembly.test-utils.ts
- Preserved the existing mock behavior and assertions while eliminating duplication across the new files
Concern-oriented coverage layout
- Isolated Docker host and chroot path validation into a dedicated security-focused test file
- Grouped host access / host port perimeter checks together
- Kept API proxy and rate-limit validation together
- Separated COPILOT_MODEL env-file detection into its own file
- Left general flag validation and the happy-path assembly case in a smaller flags-focused file

describe('config-assembly', () => {
  setupConfigAssemblyTestSuite();

  describe('docker-host validation', () => {
    it('should reject non-loopback tcp:// docker host URIs', () => {
      mockBuildConfigOnce({ awfDockerHost: 'tcp://192.168.1.100:2375' });
      expect(() => callAssembleWith()).toThrow('process.exit(1)');
    });
  });
});

Copilot

Pull request overview

This PR refactors the config-assembly validator unit tests to improve navigability by splitting a large, multi-concern test file into smaller, concern-focused suites while centralizing shared setup/mocks in a reusable test harness.

Changes:

Split src/commands/validators/config-assembly.test.ts into multiple focused test files grouped by validation concern (docker host, host access, API proxy, model detection, and general flags).
Extracted shared mocks, fixtures, and helpers into config-assembly.test-utils.ts to reduce duplication across the new suites.
Removed the original monolithic config-assembly.test.ts.

Show a summary per file

File	Description
src/commands/validators/config-assembly.test.ts	Removed the previous monolithic test suite.
src/commands/validators/config-assembly.test-utils.ts	Added shared test harness (mocks, temp-dir lifecycle, helper builders) used across the split suites.
src/commands/validators/config-assembly-docker-host.test.ts	Contains docker host / path-prefix / chroot-related validation tests.
src/commands/validators/config-assembly-host-access.test.ts	Contains host access + host port perimeter validation/warning tests.
src/commands/validators/config-assembly-api-proxy.test.ts	Contains API-proxy and rate-limit validation tests.
src/commands/validators/config-assembly-model-detection.test.ts	Contains env/env-file COPILOT_MODEL detection and model validation tests.
src/commands/validators/config-assembly-flags.test.ts	Contains general flag validation and happy-path assembly tests.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 7/7 changed files
Comments generated: 1

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-06-21T17:27:07Z

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

github-actions · 2026-06-21T17:27:37Z

✅ Smoke Gemini completed. All facets verified. 💎

Smoke test completed with partial failures. Results reported to PR #5348.

github-actions · 2026-06-21T17:27:37Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-06-21T17:27:37Z

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

github-actions · 2026-06-21T17:27:37Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-06-21T17:27:38Z

✅ Smoke Claude passed

github-actions · 2026-06-21T17:27:38Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-06-21T17:27:39Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-06-21T17:27:39Z

✅ Build Test Suite completed successfully!

github-actions · 2026-06-21T17:27:39Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-06-21T17:27:40Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-06-21T17:27:42Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-06-21T17:27:43Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-06-21T17:28:34Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	97.85%	97.89%	📈 +0.04%
Statements	97.78%	97.82%	📈 +0.04%
Functions	99.50%	99.50%	➡️ +0.00%
Branches	93.55%	93.58%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/workdir-setup.ts`	92.7% → 94.5% (+1.82%)	92.7% → 94.5% (+1.82%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-06-21T17:30:43Z

Smoke Test: Copilot BYOK (Direct Mode) — PASS ✅

✅ MCP connectivity verified
✅ GitHub.com reachable (HTTP 200)
✅ File write/read test passed
✅ BYOK inference path working (agent → api-proxy → api.githubcopilot.com)

Running in direct BYOK mode via COPILOT_PROVIDER_API_KEY.

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-21T17:30:49Z

Smoke Test: Claude Engine Validation

Check	Result
API Status	✅ PASS
gh Check	✅ PASS
File Status	✅ PASS

Overall result: PASS

Generated by Smoke Claude for issue #5348 · 36.8 AIC · ⊞ 3.1K · ◷

github-actions · 2026-06-21T17:30:49Z

🔐 Smoke Test: Copilot PAT Auth — PASS

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	✅
File write/read	✅ (`smoke-test-copilot-pat-27911993321.txt` verified)

Overall: PASS | Auth mode: PAT (COPILOT_GITHUB_TOKEN)

cc @lpcox @Copilot

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-21T17:30:58Z

🔬 Smoke Test Results — PASS

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP	✅ 200
File write/read	✅

PR: Split config-assembly validator tests by concern group
Author: @Copilot | Assignees: @lpcox @Copilot

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-21T17:31:37Z

🔭 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Notes
S1: Module loading	✅	`otel.js` loads cleanly; exports `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled` + internal test helpers
S2: Test suite	✅	59 tests passed, 0 failed (`otel.test.js` + `otel-fanout.test.js`)
S3: Env var forwarding	✅	`api-proxy-service-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME`
S4: Token tracker integration	✅	`onUsage` callback present in `token-tracker-http.js` (lines 283/324/374) — OTEL hook point confirmed
S5: OTEL diagnostics	✅	No OTLP endpoint configured → falls back to file export (`/var/log/api-proxy/otel.jsonl`); module stays enabled with graceful degradation

All 5 scenarios passed.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-21T17:31:45Z

Smoke test report

Last merged PRs: perf(security-guard): prioritize security-relevant files in PR diff, fix(smoke-claude): raise turn budget to 8 and fix add_comment usage
GitHub title: ✅
Safe inputs GH query: ✅
Playwright title: ✅
File write/read: ✅
Discussion comment: ✅
Build: ✅
Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-21T17:31:50Z

Gemini Engine Smoke Test Results

GitHub MCP Testing: ❌ (Tools not found: github:list_pull_requests)
GitHub.com Connectivity: ❌ (Status 000/35/400 - Squid Proxy ERR_INVALID_URL/SSL Version Error)
File Writing Testing: ✅
Bash Tool Testing: ✅

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-06-21T17:31:56Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌ NO
Node.js	v24.16.0	v22.22.3	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot

github-actions · 2026-06-21T17:32:31Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #5348 · 35.8 AIC · ⊞ 7.7K · ◷

github-actions · 2026-06-21T17:33:56Z

Smoke Test: GitHub Actions Services Connectivity

Check	Result
Redis PING	❌ No response (timeout)
PostgreSQL pg_isready	❌ `no response`
PostgreSQL SELECT 1	❌ No response (timeout)

Overall: FAIL — host.docker.internal is not reachable from this environment. Service containers are inaccessible.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-21T17:34:24Z

@lpcox @Copilot
Split config-assembly validator tests by concern group
• MCP API: ✅
• HTTP: ✅
• File I/O: ✅
• BYOK Inference: ✅
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
Overall: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

Initial plan

34bd588

Copilot AI assigned Copilot and lpcox Jun 21, 2026

Copilot started work on behalf of lpcox June 21, 2026 16:06 View session

test: split config assembly validator coverage

ca0096f

Copilot AI changed the title ~~[WIP] Refactor config-assembly test file by validation concern groups~~ Split config-assembly validator tests by concern group Jun 21, 2026

Copilot AI requested a review from lpcox June 21, 2026 16:09

Copilot finished work on behalf of lpcox June 21, 2026 16:10

lpcox marked this pull request as ready for review June 21, 2026 17:11

Copilot AI review requested due to automatic review settings June 21, 2026 17:11

Copilot started reviewing on behalf of lpcox June 21, 2026 17:11 View session

Copilot AI reviewed Jun 21, 2026

View reviewed changes

Comment thread src/commands/validators/config-assembly.test-utils.ts

Potential fix for pull request finding

49eadc9

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

lpcox added the ready-for-aw label Jun 21, 2026

lpcox had a problem deploying to aoai-model June 21, 2026 17:27 — with GitHub Actions Failure

github-actions Bot added the smoke-copilot-byok label Jun 21, 2026

lpcox temporarily deployed to aoai-model June 21, 2026 17:30 — with GitHub Actions Inactive

github-actions Bot added smoke-claude smoke-copilot-pat labels Jun 21, 2026

github-actions Bot added the smoke-copilot label Jun 21, 2026

lpcox temporarily deployed to aoai-model June 21, 2026 17:31 — with GitHub Actions Inactive

github-actions Bot added the smoke-codex label Jun 21, 2026

github-actions Bot added the build-test label Jun 21, 2026

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jun 21, 2026

lpcox enabled auto-merge (squash) June 21, 2026 17:46

lpcox merged commit 58ced40 into main Jun 21, 2026
84 of 88 checks passed

lpcox deleted the copilot/refactor-split-config-assembly-tests branch June 21, 2026 17:46

Uh oh!

Conversation

Copilot AI commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 21, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

github-actions Bot commented Jun 21, 2026

Smoke Test: Copilot BYOK (Direct Mode) — PASS ✅

Uh oh!

github-actions Bot commented Jun 21, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Jun 21, 2026

🔐 Smoke Test: Copilot PAT Auth — PASS

Uh oh!

github-actions Bot commented Jun 21, 2026

🔬 Smoke Test Results — PASS

Uh oh!

github-actions Bot commented Jun 21, 2026

🔭 Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jun 21, 2026

Uh oh!

github-actions Bot commented Jun 21, 2026

Gemini Engine Smoke Test Results

Uh oh!

github-actions Bot commented Jun 21, 2026

Chroot Version Comparison Results

Uh oh!

github-actions Bot commented Jun 21, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented Jun 21, 2026

Copilot AI commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading

github-actions Bot commented Jun 21, 2026 •

edited

Loading