CI: opt tool-calling smoke tests out of the chat tool approval gate#7162
Conversation
The permission-levels feature defaults an unset permission_mode to ask on streaming requests, so the headless smoke probes hang at the approval prompt until the job timeout. Declare permission_mode full in the tool probe bodies; the gate itself is covered by unit tests.
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d443bbc1c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| content, events = post_sse("/v1/chat/completions", { | ||
| "messages": [{"role": "user", "content": prompt}], | ||
| "enable_tools": True, | ||
| "permission_mode": "full", |
There was a problem hiding this comment.
In this and the other added probe bodies, permission_mode: "full" does more than bypass the approval wait: ChatCompletionRequest._fold_full_permission_into_bypass maps it to bypass_permissions=True, and the local tool loop passes that through as disable_sandbox to execute_tool, so the python/terminal smoke jobs no longer cover the safety checks, blocklist, or rlimits they used to exercise. For this headless CI case, permission_mode: "off" avoids prompting while keeping the sandbox enabled.
Useful? React with 👍 / 👎.
1 participant
Since the permission-levels feature (#7079), an unset permission_mode on a streaming request with tools enabled defaults to the ask gate. The headless smoke probes never approve, so every Tool calling Tests job across all branches hangs at the approval wait (3600s decision timeout) until the job timeout kills it - reproduced locally and bisected to the gate default in _permission_mode_confirm.
This declares permission_mode full in the seven tool-probe request bodies across the linux, mac, and windows inference smoke workflows. Gate behavior itself stays covered by the tool-policy unit tests.
Verified locally against a live Studio: with the gate engaged the stream stalls at tool_start (awaiting_confirmation) as CI sees; with permission_mode full the same probe completes.