fix(chroot): mount binaries overlay at /host/tmp/awf-runner-bin to avoid read-only /host/usr collision on ARC/DinD

[Copilot]

On ARC/DinD with --docker-host-path-prefix /tmp/gh-aw and chroot.binariesSourcePath: /tmp/gh-aw, Docker applies bind mounts sequentially: the /tmp/gh-aw/usr:/host/usr:ro mount makes /host/usr read-only, then the attempt to create /host/usr/local/bin as a mount point fails with mkdirat: read-only file system. This blocks the chrooted agent from ever seeing runner-installed binaries.

Changes

src/services/agent-volumes/system-mounts.ts

• Before: chrootBinariesSourcePath was mounted at /host/usr/local/bin:ro — a path nested inside the read-only /host/usr parent mount.
• After: mounted at /host/tmp/awf-runner-bin:ro. Since /host/tmp is always mounted writable (/tmp:/host/tmp:rw), Docker can unconditionally create the subdirectory mount point, even when the staged /usr tree lacks a local/bin directory.

containers/agent/entrypoint.sh

• After the staged-binary handling block, detects /host/tmp/awf-runner-bin and prepends /tmp/awf-runner-bin to AWF_HOST_PATH, making the overlay visible on the chroot PATH.
• Updated the preflight-binary error message to list /tmp/awf-runner-bin among the standard PATH directories.

Tests

• Updated two existing test assertions (/host/usr/local/bin → /host/tmp/awf-runner-bin).
• Added regression test covering the collision case where binariesSourcePath == dockerHostPathPrefix:

// binariesSourcePath = /tmp/gh-aw === dockerHostPathPrefix — the reported collision
expect(volumes).toContain(`${sharedPrefix}/usr:/host/usr:ro`);
expect(volumes).toContain(`${sharedPrefix}:/host/tmp/awf-runner-bin:ro`);
expect(volumes).not.toContain(`/host/usr/local/bin`);

[Copilot]

Pull request overview

This PR fixes an ARC/DinD chroot-mode volume mount collision where Docker can’t create /host/usr/local/bin as a mount point after /host/usr has been mounted read-only, preventing runner-installed binaries from being visible inside the chroot.

Changes:

• Move the chroot runner-binaries overlay mount target from /host/usr/local/bin to /host/tmp/awf-runner-bin (under the writable /host/tmp).
• Update the agent entrypoint to detect the new mount and ensure /tmp/awf-runner-bin is included in the chroot PATH.
• Update existing tests and add a regression test for the binariesSourcePath == dockerHostPathPrefix collision case.

Show a summary per file

File	Description
src/services/agent-volumes/system-mounts.ts	Changes the runner-binaries overlay mount target to /host/tmp/awf-runner-bin to avoid read-only /host/usr mount-point creation failures.
src/services/agent-volumes-mounts.test.ts	Updates assertions for the new mount target and adds a regression test for the ARC/DinD collision scenario.
containers/agent/entrypoint.sh	Prepends /tmp/awf-runner-bin to the chroot PATH when the overlay is present and updates the preflight PATH error message.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 3/3 changed files
• Comments generated: 1

[github-actions]

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

[github-actions]

🔌 Smoke Services — All services reachable! ✅

[github-actions]

✅ Build Test Suite completed successfully!

[github-actions]

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

[github-actions]

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

[github-actions]

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

[github-actions]

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

[github-actions]

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

[github-actions]

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

[github-actions]

✅ Smoke Gemini completed. All facets verified. 💎

[github-actions]

✅ Smoke Claude passed

[github-actions]

❌ Security Guard failed. Please review the logs for details.

[github-actions]

✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

[github-actions]

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

[github-actions]

✅ Contribution Check completed successfully!

Contribution guidelines review complete - PR follows all applicable CONTRIBUTING.md guidelines; no comment needed.

[github-actions]

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	98.06%	98.10%	📈 +0.04%
Statements	98.00%	98.03%	📈 +0.03%
Functions	99.52%	99.52%	➡️ +0.00%
Branches	93.71%	93.75%	📈 +0.04%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
src/workdir-setup.ts	92.7% → 94.5% (+1.82%)	92.7% → 94.5% (+1.82%)

---

Coverage comparison generated by scripts/ci/compare-coverage.ts

[github-actions]

🔬 Smoke Test: Copilot PAT Auth — PR #5482

Test	Result
GitHub MCP connectivity	✅ PASS
github.com HTTP connectivity	⚠️ N/A (step output not resolved)
File write/read	⚠️ N/A (step output not resolved)

Overall: PARTIAL — MCP test passed; pre-step outputs (SMOKE_HTTP_CODE, SMOKE_FILE_PATH, SMOKE_FILE_CONTENT) were not interpolated into the agent prompt.

Auth mode: PAT (COPILOT_GITHUB_TOKEN)

cc @lpcox @Copilot

🔑 PAT report filed by Smoke Copilot PAT

[github-actions]

Smoke Test Results for #5482

• fix(chroot): mount binaries overlay at /host/tmp/awf-runner-bin to avoid read-only /host/usr collision on ARC/DinD - ✅

• docs: sync schemas and specs with source changes - ✅

• GitHub.com connectivity - ✅

• File I/O in agent container - ✅

• Direct BYOK mode via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) - ✅

Overall Status: PASS

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

[github-actions]

fix(chroot): mount binaries overlay at /host/tmp/awf-runner-bin to avoid read-only /host/usr collision on ARC/DinD by Copilot
✅ GitHub reads
✅ Browser title check
✅ File write + readback
✅ Discussion comment
✅ npm ci && npm run build
Overall: PASS

🔮 The oracle has spoken through Smoke Codex

[github-actions]

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Notes

• Java: Maven local repository path required override (-Dmaven.repo.local) due to default ~/.m2/repository directory being owned by root in this runner environment. Tests compiled and ran successfully with alternate path.
• All other ecosystems ran without issues.

Generated by Build Test Suite for issue #5482 · 46.1 AIC · ⊞ 7.7K · ◷

[github-actions]

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	3.12.13	3.12.3	❌ NO
Node.js	v24.17.0	v22.23.0	❌ NO
Go	go1.22.12	go1.22.12	✅ YES

Overall: ❌ FAILED — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot

[github-actions]

🔬 Smoke Test Results

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP	❌ (pre-step data not substituted)
File write/read	❌ (pre-step data not substituted)

Overall: FAIL — workflow template variables were not substituted; pre-computed step outputs unavailable.

PR: fix(chroot): mount binaries overlay at /host/tmp/awf-runner-bin to avoid read-only /host/usr collision on ARC/DinD
Author: @Copilot | Assignees: @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot

[github-actions]

Smoke Test Results — PR #5482

Check	Result
Redis PING	❌ Timeout (host.docker.internal:6379 unreachable)
PostgreSQL pg_isready	❌ No response (host.docker.internal:5432 unreachable)
PostgreSQL SELECT 1	❌ Not attempted (pg_isready failed)

host.docker.internal resolves to 172.17.0.1 but both ports timed out.

Overall: FAIL

🔌 Service connectivity validated by Smoke Services

[github-actions]

Copilot BYOK Smoke Test Results

✅ All tests passed — Direct BYOK mode operational

Tests:

• ✅ GitHub MCP connectivity (PRs listed successfully)
• ✅ GitHub.com connectivity (HTTP 200)
• ✅ File write/read test (smoke-test-copilot-byok.txt confirmed)
• ✅ BYOK inference (reading & responding via api-proxy sidecar)

Configuration: Direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Status: PASS

🔑 BYOK report filed by Smoke Copilot BYOK

[github-actions]

Smoke Test: Claude Engine Validation

Check	Status
API	✅ PASS
gh CLI	✅ PASS
File access	✅ PASS

Overall result: PASS

Generated by Smoke Claude for issue #5482 · 61.3 AIC · ⊞ 3.1K · ◷

[github-actions]

@lpcox Smoke Test Results for direct BYOK mode:

• GitHub MCP connectivity: ✅
• GitHub.com connectivity: ✅
• File write/read test: ✅
• Direct BYOK inference path: ✅
  Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
  Overall: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

[github-actions]

📡 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Status	Result
1. Module Loading	✅	otel.js loaded, isEnabled: true; exports: startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown + internals
2. Test Suite	✅	59 passed, 0 failed across 2 suites (otel.test.js, otel-fanout.test.js)
3. Env Var Forwarding	⚠️	OTEL_EXPORTER_OTLP_ENDPOINT / GITHUB_AW_OTEL_TRACE_ID not yet in api-proxy-service.ts — expected during development
4. Token Tracker Integration	✅	token-tracker-http.js has onUsage callback (OTEL hook point present)
5. OTEL Diagnostics	i️	No span file at logs/api-proxy/otel.jsonl — api-proxy OTEL export not yet active

All scenarios pass or are expected-pending during development. ✅

📡 OTel tracing validated by Smoke OTel Tracing

[github-actions]

Smoke Test Results

• GitHub MCP Testing: ❌ (Could not retrieve last 2 merged PRs)
• GitHub.com Connectivity: ❌ (HTTP 000 / SSL Error 35)
• File Writing Testing: ✅
• Bash Tool Testing: ✅

Overall Status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini