Studio: improve Deep Research synthesis

[oobabooga]

Follow-up to #7219, targeting feature/deep-research.

Important

This is a review-only mirror. It targets pr-7219-deep-research, an exact copy of #7219's current head, so GitHub displays only this follow-up diff.
Do not merge this PR. Review and merge alkinun/unsloth#1 instead. Once merged there, these changes will reach main through #7219.

Problem

This follow-up addresses two separate problems found while testing #7219:

1. Planning and next-search calls use extended thinking even though they only need to return small JSON objects. With Qwen3.6-27B-MTP, these calls spent much of their output budget reasoning about the decision.
2. The final report is written directly from the collected notes. Nothing checks whether its main recommendations are supported by those notes.

In the public test below, #7219 collected 38 sources but recommended a diffusion-based architecture that did not provide the required deterministic, random-access terrain streaming. The report presented that recommendation and other design choices as established conclusions.

Terrain test prompt

We need an algorithm for generating seamless, planet-scale terrain for a space
exploration game. A planet is generated deterministically from a seed and
streamed at multiple levels of detail.

The result must not be uniform noise decorated with biome colors and randomly
scattered assets. It should contain coherent regions, recognizable landforms,
caves, surface compositions and reflectance, resources, vegetation or analogous
assets, and rare features worth seeking out. These elements should be spatially
related so players can learn patterns, follow environmental clues, and predict
where discoveries may occur.

Formulate this as a mathematical and algorithmic problem. Recommend a concrete
generative model that produces structure across planetary, regional, and local
scales. Define the important variables and representations, such as fields,
graphs, regions, processes, constraints, or probability distributions, and
explain how different scales and feature types influence one another.

Compare viable approaches and select one architecture. Provide equations or
pseudocode sufficient to implement its central mechanisms. Address determinism,
boundary continuity, streaming, level of detail, computational cost,
controllable rarity, and avoiding repetition across and within planets.

Finally, propose quantitative measures and experiments for determining whether
the generated worlds are genuinely diverse, coherent, and useful for exploration
rather than merely visually different.

What changes in practice

Before this follow-up:

Planning the next search:
  extended model reasoning
    -> {"action": "search", "query": "..."}

Collected note:
  Diffusion models can generate large, coherent terrain.

State carried to the next step:
  accumulated notes only

Final report:
  Use diffusion for deterministic O(1) random access while streaming the planet.

There is no step that records the unresolved streaming requirement or checks
whether the note supports the stronger O(1) random-access claim.

With this follow-up:

Planning the next search:
  {"action": "search", "query": "..."}

Research state:
  requirement: deterministic random-access streaming
  unresolved: whether diffusion can provide it at runtime
  unsupported: O(1) random-access claim

Evidence check before the report:
  supported: diffusion can generate coherent terrain
  unsupported: deterministic O(1) random access

Final report:
  The unsupported claim must be removed, identified as a design inference with
  a validation experiment, or replaced by an architecture supported by the
  available evidence.

Extended model reasoning remains enabled for final report generation.

Changes

• Disable extended thinking for planning and next-step decisions. These calls still use the same model, but return their small structured responses directly.
• Keep a compact research state after every step: current findings, remaining gaps, unsupported claims, and the most useful next connection to investigate.
• Give each decision call that state and the list of searches already performed.
• Before writing the report, ask the model to map conclusions to evidence and identify unsupported details, contradictions, and missing areas.
• Tell the report writer to label unsupported recommendations as design inferences and include a way to test them.
• If report generation reaches the output limit before producing the report, retry it once without extended thinking.

The search engine, scraping, source limits, and initial final-report generation settings are unchanged.

Evaluation

Both versions used Qwen3.6-27B-MTP-GGUF (UD-Q4_K_XL), a 32,768-token context, temperature 0, DuckDuckGo search, automatic scraping, at most 12 research steps, and at most 40 sources.

Metric	#7219	This follow-up
Completed runs	1/1	2/2
Wall time	661 s	395-405 s
Research steps	8	12, 12
Sources collected	38	40, 40
Blind review score	12/40	23/40, 26/40

The blind review scored mathematical coherence, deterministic streaming, system integration, exploration structure, feasibility, evidence, validation, and clarity. The higher-scoring follow-up report was preferred over the #7219 report in both presentation orders.

This is a focused comparison on one public research problem, not a general accuracy benchmark.

Checks

• 130 focused backend tests passed.
• Both follow-up end-to-end runs completed with the committed code.
• Ruff, Python compilation, and diff checks passed.

[oobabooga]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d99fad30cb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5bc139c093

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33a016e511

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[oobabooga]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 👍

Reviewed commit: e6dc991353

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[oobabooga]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: f1b3231b99

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".