Make group-boundary AutoQuant scoring the default

CHANGELOG.rst

@@ -28,6 +28,7 @@ Experimental
 
 - Add the ``prepare_megatron_data_blend`` utility to prepare weighted Megatron data blends from YAML configs, including optional token-budgeted subsets for distillation workflows. See the `Megatron data preparation guide <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/dataset/MEGATRON_DATA_PREP.md#prepare-token-budgeted-data-blends>`_.
 - Add Learned Scale Quantization (LSQ) and Dual-LSQ support for quantization-aware distillation, including learnable ``amax`` parameters, tied-scale and pre-scale options, focused NVFP4 recipes, and scale-only training.
+- Add group-boundary AutoQuantize scoring and make it the default for gradient scoring. Quantizable descendants of common Hugging Face and Megatron attention parents are scored at their parent attention output without grouping recipe decisions; attention containers resolve to their invoked child, while fused, shared, and unfused MoE experts retain the established parent-MLP scoring boundary. Explicit mutable cache or recurrent-state inputs are rejected during parent replay, and configuration-level cache changes are transactional. Set ``score_boundary: local`` in an AutoQuantize recipe to restore leaf-output attention scoring. KL-divergence retains local scoring.
 - Add the **D-PACE** loss objective for DFlash speculative-decoding training (`arXiv:2605.18810 <https://arxiv.org/abs/2605.18810>`_) and make it the default (``dflash_loss_objective: dpace``). It replaces the static exponential position decay with dynamic, confidence-derived per-position weights that adapt to whichever block positions currently limit acceptance. Smoothing is controlled by ``dflash_dpace_alpha`` (default 0.5); set ``dflash_loss_objective: decay`` to restore the previous static schedule. Training-only and detached from the gradient (no architecture or inference change).
 - Add the ``day0-release`` agent skill (``.agents/skills/day0-release/``), a deterministic end-to-end driver that chains the PTQ → evaluation → comparison skills (the evaluation stage deploys the checkpoint itself) with an enforced gate after each stage and returns a publish decision (ACCEPT / REGRESSION / ANOMALOUS / INFEASIBLE). Ships three GPU-free, unit-tested gate scripts (``gate_ptq.py``, ``gate_run.py``, ``gate_compare.py``) that validate checkpoint coverage, evaluation-run completeness, and baseline-vs-candidate accuracy threshold. v1 reports and stops on regression; the recipe-search loop is deferred.
 - Add **streaming** speculative-decoding training (EAGLE3 / DFlash): the draft trains on base-model hidden states produced on the fly by a co-located ``vllm serve`` (no disk dump), moved trainer-side over NIXL RDMA, scaling to multi-node (dedicated serve replicas + DDP trainers). New launcher examples for NVFP4 Kimi-K2.5 / K2.6 on GB200/aarch64 under ``tools/launcher/examples/moonshotai/``.

examples/hf_ptq/README.md

@@ -352,19 +352,22 @@ Here is an example usage for `AutoQuantize` algorithm (Please see [auto_quantize
 `AutoQuantize` can be performed for Huggingface LLM models like [Qwen](https://huggingface.co/Qwen/Qwen3-8B) / [Nemotron](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) as shown below:
 
 `AutoQuantize` is driven by an **AutoQuantize recipe** passed with `--recipe`. The recipe defines the
-candidate formats, optional fixed PTQ baseline, `effective_bits` target, cost model, scoring method,
+candidate formats, optional fixed PTQ baseline, `effective_bits` target, cost model, scoring method
+and boundary,
 search-disabled layers, and cost-excluded layers — see
 [`AutoQuantizeConfig`](../../modelopt/recipe/config.py). Shipped recipes live in
 [`modelopt_recipes/general/auto_quantize/`](../../modelopt_recipes/general/auto_quantize); model-specific
 recipes (carrying architecture-specific disabled layers — e.g. VL vision towers) live under
 `modelopt_recipes/huggingface/<model>/auto_quantize/`.
 
 > *Migration: prefer an AutoQuantize `--recipe`. The `--auto_quantize_bits`, `--auto_quantize_method`,
-> `--auto_quantize_score_size`, `--auto_quantize_cost_model`, and `--auto_quantize_active_moe_expert_ratio`
+> `--auto_quantize_score_size`, `--auto_quantize_cost_model`, and
+> `--auto_quantize_active_moe_expert_ratio`
 > CLI flags are **deprecated but still work** — they are converted into an `AutoQuantizeConfig` on the fly
 > (with a `DeprecationWarning`) and will be removed in a future release. They map to recipe fields:
 > `--auto_quantize_bits` → `constraints.effective_bits`, `--auto_quantize_method` → `auto_quantize_method`,
-> `--auto_quantize_score_size` → `score_size`, `--auto_quantize_cost_model` → `constraints.cost_model`,
+> `--auto_quantize_score_size` → `score_size`,
+> `--auto_quantize_cost_model` → `constraints.cost_model`,
 > `--auto_quantize_active_moe_expert_ratio` → `constraints.cost.active_moe_expert_ratio`, and the
 > `--qformat fp8,nvfp4` candidate list → `candidate_formats`. When converted, the shared base
 > `disabled_layers` and `cost_excluded_layers` patterns are appended automatically. `--auto_quantize_checkpoint`
@@ -385,7 +388,15 @@ keeps the more sensitive ones at higher precision (or unquantized), so the model
 `constraints.effective_bits`, `auto_quantize_method` (`gradient` / `kl_div`), `score_size`,
 `module_search_spaces` (optional per-module candidate overrides), `disabled_layers` (excluded from
 the search), and `cost_excluded_layers` (kept out of the bit-budget accounting — e.g. VL vision
-towers). Recipes can splice a shared base `disabled_layers` set via `$import` (see
+towers), and `score_boundary` (`local` / `group`). Group scoring is the default for gradient; it
+changes the measurement boundary but does not force attention projections to share one recipe
+decision. Set `score_boundary: local` in the recipe to restore leaf-output attention scoring;
+expert projections retain their established parent MLP/mixer scoring boundary. Attention parent
+names ending in `attn` or `attention` are recognized across fused and split projection layouts.
+For attention stored in a `ModuleList` or `ModuleDict`, scoring resolves the invoked child instead
+of the uncalled container. Group scoring rejects explicit past/cache/recurrent-state inputs because
+replaying mutable state would compare candidates at different sequence positions. Recipes can splice
+a shared base `disabled_layers` set via `$import` (see
 `modelopt_recipes/configs/auto_quantize/units/base_disabled_layers`).
 
 AutoQuantize recipes support two mutually exclusive search-space styles:
@@ -450,7 +461,9 @@ The example scripts above also have an additional flag `--tasks`, where the actu
 
 > *If GPU out-of-memory error is reported running the scripts, please try editing the scripts and reducing the max batch size to save GPU memory.*
 
-> *NOTE: AutoQuantize requires backpropagation of the model. Models without backpropagation support (e.g., Llama-4) will not work with AutoQuantize when using the `gradient` method. The `kl_div` method does not require backpropagation.*
+> *NOTE: AutoQuantize requires backpropagation of the model. Models without backpropagation support
+> (e.g., Llama-4) will not work with AutoQuantize when using the `gradient` method. The `kl_div`
+> method does not require backpropagation.*
 
 ## Real Quant
 

examples/hf_ptq/hf_ptq.py

@@ -371,6 +371,7 @@ def _mtq_inputs_from_auto_quantize_config(
         "disabled_layers": aq_config.disabled_layers,
         "kv_cache_quant_cfg": kv_cache_quant_cfg,
         "method": aq_config.auto_quantize_method,
+        "score_boundary": aq_config.score_boundary,
         "score_size": aq_config.score_size,
     }
 
@@ -501,6 +502,7 @@ def forward_step(model, batch):
         verbose=True,
         disabled_layers=inputs["disabled_layers"],
         method=inputs["method"],
+        score_boundary=inputs["score_boundary"],
         checkpoint=args.auto_quantize_checkpoint,
     )
 

modelopt/recipe/config.py

@@ -259,6 +259,13 @@ class AutoQuantizeConfig(ModeloptBaseConfig):
         title="Sensitivity scoring method",
         description="'gradient' (Taylor + Fisher, needs labels) or 'kl_div' (no labels).",
     )
+    score_boundary: Literal["local", "group"] | None = ModeloptField(
+        default=None,
+        title="Sensitivity score boundary",
+        description="'local' scores attention projections at their leaf outputs; 'group' scores "
+        "them at the parent attention output. Expert projections retain their established parent "
+        "MLP boundary in either mode. Defaults to 'group' for gradient and 'local' for kl_div.",
+    )
     score_size: int = ModeloptField(
         default=128,
         title="Scoring sample count",
@@ -293,6 +300,15 @@ def _has_search_space(self):
             )
         return self
 
+    @model_validator(mode="after")
+    def _validate_scoring_configuration(self):
+        boundary = self.score_boundary or (
+            "local" if self.auto_quantize_method == "kl_div" else "group"
+        )
+        if self.auto_quantize_method == "kl_div" and boundary != "local":
+            raise ValueError("auto_quantize_method='kl_div' requires score_boundary='local'.")
+        return self
+
 
 class ModelOptAutoQuantizeRecipe(ModelOptRecipeBase):
     """Our config class for AutoQuantize recipes."""