Skip to content

[Bug] [STUDIO] Context Length & KV Cache Dtype settings ignored on initial model loading #7346

Description

@rafpigna
  1. Did you update? pip install --upgrade unsloth unsloth_zoo
    Not installed via pip, but updated all to last versions
    Installed/updated on windows via "irm https://unsloth.ai/install.ps1 | iex"

  2. Colab or Kaggle or local / cloud
    Local

  3. Number GPUs used, use nvidia-smi
    1 x RTX 3080 !0G

  4. Which notebook? Please link!
    Doesnt apply

  5. Which Unsloth version, TRL version, transformers version, PyTorch version?
    Unsloth Version: v0.1.50-beta
    Package Version: 2026.7.4
    Python 3.12.10


Bug Report
When setting custom parameters for Context Length and KV Cache Dtype in the Chat sidebar before loading the model, these values are ignored during the initial load. The model starts with default values instead of the ones specified.

Environment
OS: Windows 11
Unsloth Version: v0.1.50-beta
Package Version: 2026.7.4
llama.cpp Version: b10079-mix-fb3d4ca
Python 3.12.10

Steps to Reproduce
Ensure no models are loaded.
Open the Chat sidebar in Unsloth Studio.
Highlight a model in the dropdown list and click on the "Configure run settings before loading a model" small "gear" icon. The sidebar will open.
Change the Context Length to a specific custom value (e.g., 262144).
Change the KV Cache Dtype to a specific custom value (e.g., q8_0).
Click the "Load model" button.
Observe the loaded settings.
Actual Behavior The model loads with the default values instead of the ones specified above.

Context Length defaults to 4096 (in my case)
KV Cache Dtype defaults to fp16. (in my case)
Workaround To apply the custom settings, the user must wait for the model to load, change the settings again after loading, and then click "Apply".

Expected Behavior The model should initialize using the Context Length and KV Cache Dtype values specified in the sidebar immediately upon clicking "Load model".

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions