Fix: skip fp16/bf16 validation for full finetuning in RL trainers#6813
Conversation
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
When doing full finetuning (FFT) of a bfloat16 model, the fp16/bf16 mismatch validation fires before the corrective logic runs, causing a misleading error even though the code would properly handle it downstream. Skip the validation when full_finetuning is active. Fixes unslothai#6731
b3bc2b5 to
146740d
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b3bc2b53ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex I have made some changes in the code that you suggested. Please review them once and then notify the reviewers about the PR once please |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 146740de5c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…idation Instead of entirely skipping validation (which could let mismatches through when mixed_precision_dtype is float32), auto-correct explicit fp16/bf16 settings that conflict with the model's dtype for FFT. This way the existing validation still catches real mismatches for non-FFT cases, and the corrective logic below handles the normalized settings. Fixes the issue raised in Codex review of PR unslothai#6813.
|
Reproduced #6731 to confirm your fix. I pulled the exact mixed-precision block from On Codex's open note about float32 full-finetuning: it applied to the earlier skip-validation commit, not your current auto-correct one. Because you flip Merging. |
* Fix: skip fp16/bf16 validation for full finetuning in RL trainers When doing full finetuning (FFT) of a bfloat16 model, the fp16/bf16 mismatch validation fires before the corrective logic runs, causing a misleading error even though the code would properly handle it downstream. Skip the validation when full_finetuning is active. Fixes #6731 * Fix: auto-correct fp16/bf16 mismatches for full finetuning before validation Instead of entirely skipping validation (which could let mismatches through when mixed_precision_dtype is float32), auto-correct explicit fp16/bf16 settings that conflict with the model's dtype for FFT. This way the existing validation still catches real mismatches for non-FFT cases, and the corrective logic below handles the normalized settings. Fixes the issue raised in Codex review of PR #6813. * Guard Windows ROCm torchao override skip Detect installed ROCm torch directly before applying the torchao override so Windows ROCm environments never install the crashing torchao package even if the earlier ROCm-installed flag is missing. * Update unsloth/models/rl.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update studio/install_python_stack.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Harden ROCm probe and sync RL precision flags Tolerate stray stdout noise when probing Windows ROCm torch installs by checking the last non-empty output line, matching the existing torch version probe behavior. Also keep args.fp16 and args.bf16 synchronized with the full-finetuning precision auto-corrections in the RL trainer patch so downstream eval settings see a consistent TrainingArguments state. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add MLX trainer compatibility shims Patch imported MLXTrainer and MLXTrainingConfig objects to preserve the expected dataclass field ordering and to provide a _train_dataset_for_batches fallback when older trainers or test doubles only expose train_dataset. Also add focused worker tests covering both compatibility paths. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope PR to Windows ROCm torchao guard * Restore PR scope to Windows ROCm guard * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: cover Windows ROCm torchao skip behavior * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Ayushman Paul <ayushman@HP> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com>
2 participants
Description
Fixes #6731
When doing full finetuning (FFT) of a bfloat16 model, the fp16/bf16 mismatch validation in the RL trainer patches fires before the corrective logic runs. This means if
TrainingArgumentshasfp16=True(a common default), the code raises a TypeError telling the user to setbf16=True— even though:fp16andbf16toFalsebf16=Trueas the error suggests is actually wrong for bf16 FFT (which doesn't need autocasting)Fix
Added
not full_finetuningto both validation conditions, so they are skipped during full finetuning and the downstream corrective logic handles the settings correctly.Testing
Verified the change is syntactically correct and the logic flow now properly bypasses the validation when
full_finetuning=True.AI Assistance
I have used OpenCode for generating the code but I have reviewed the code myself.