Installer: opt-in Vulkan llama.cpp backend (and fallback when no AMD card is HIP-supported)#1
Open
Souravrajvi0 wants to merge 57 commits into
Open
Installer: opt-in Vulkan llama.cpp backend (and fallback when no AMD card is HIP-supported)#1Souravrajvi0 wants to merge 57 commits into
Souravrajvi0 wants to merge 57 commits into
Conversation
* Fix resume training crash recovery and MLX checkpoints * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review: preserve interrupted stop-and-save output_dir, verify MLX checkpoint - finish_run: add clear_output_dir flag; preserve output_dir for stopped/error unless cancel explicitly clears it (fixes pump finalization wiping persisted path). - training pump: pass interrupted stop-and-save context into finalize_run_in_db. - MLX stop-and-save: verify resumable checkpoint exists before sending complete; return bool from _write_mlx_stop_checkpoint and add regression tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address Codex review: MLX current-step checkpoint and cancel error finalize - Only skip MLX stop checkpoint write when checkpoint-{current_step} exists; stale periodic checkpoints no longer mask missing stop saves. - Pass clear_output_dir through error-event finalization so Stop-without-save cannot leave a persisted output_dir that still offers Resume. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address review * Address more reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * more reviews * clear in-memory output_dir on interrupted cancel * allow resuming errored runs at the final step * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Clear persisted output_dir in cancel watchdog path * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Write MLX stop checkpoint in stop path, keep output_dir on crash finalize * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): harden resumable run finalization * fix(studio): defer safetensors checkpoint import * fix(studio): reject stale training cancellation * fix(studio): replay null resume targets * fix(studio): serialize terminal cancellation * Harden resume checkpoint validation and fix stop-save cleanup - Reject unrecognized shard formats and keep indexed shard paths inside the checkpoint dir - Require a non-empty tensor record when validating .pt/.bin optimizer and model state - Always finalize TensorBoard and W&B on stop-save-failure exits - Refuse writing an MLX stop checkpoint through a symlinked directory - Clarify the resume rejection message to cover errored runs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten resume/checkpoint comments * Recover resumability when a valid stop checkpoint landed - Re-validate the current-step checkpoint in the dead-worker and error finalization paths so a stop-and-save that actually wrote a valid checkpoint is not wrongly marked error/resume_blocked - Accept a valid tensor-free optimizer state (e.g. SGD without momentum); the model-state check still requires real tensors - Include errored runs in the frontend resume rejection message * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Lyxot <longyixing331@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: don't resolve HF cache GGUF symlinks to blob paths * Studio: handle split GGUF symlink layouts
Co-authored-by: shimmyshimmer <info@unsloth.ai>
* Installer: report the installed Unsloth version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show HF token tick only after validation * Studio: prevent stale HF token validation state * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…othai#7277) * Installer: enable ROCm torch on RDNA2 (gfx1030-1036) on Windows repo.amd.com publishes a gfx103X-all wheel family with win_amd64 torch 2.9.1/2.10.0/2.11.0+rocm7.13.0 (cp310-313), but both Windows allowlists omitted RDNA2, so RX 6000 cards (gfx1030/1032, etc.) fell back to CPU-only torch. Map gfx1030-1036 to gfx103X-all in install.ps1 ($archFamilyMap) and install_python_stack.py (_GFX_TO_AMD_INDEX_ARCH). No torch floor (mirrors gfx110X-all: newest wheel, no _grouped_mm bug on RDNA2). NVIDIA/Mac/CPU and Linux paths untouched; gfx906 stays CPU (no wheels published). * Sync studio/setup.ps1 RDNA2 (gfx1030-1036) allowlists for PR unslothai#7277
* Studio: show the mobile sidebar trigger above the chat header * fix * correct z-index --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: wrap composer toolbar chips instead of clipping on narrow screens * Studio: collapse composer pills to icons on narrow screens * Studio: wrap compare composer pills instead of clipping --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
…es (unslothai#7291) * Add project pinning with sidebar chats, full project chat menu, and new-chat fixes * Fold pinned projects into the Pinned section with chat icons and lighter show-more * Address review: queue guard, HTTP-safe nonce, projects show-more, dialog state resets - Project new-chat composer now shows Stop instead of Queue while running, and the queue click is guarded so a disabled queue cannot enqueue. - Replace crypto.randomUUID with a helper that falls back on non-secure (HTTP LAN) contexts where it is undefined. - Projects page keeps all projects reachable: the grid caps at four with a Show more/Show less toggle instead of hiding the rest. - Reset the rename draft and the delete-files checkbox so a project dialog never opens with stale state. * Address review: dedupe pinned chats, close drawer on nested nav, gate saved-prompt queue, reset delete toggle - Pinned chats that live inside a pinned project now render only nested under the project, not a second time in the flat pinned list. - Opening a chat nested under a pinned project closes the mobile drawer, matching the other sidebar nav handlers. - The saved-prompt Run-list path now honours disableQueue, so running a prompt list from the project new-chat composer cannot queue against an unbound thread. - Reset the delete-files toggle when opening a project delete, since the Cancel button closes the dialog programmatically and skips the onOpenChange reset. * Sidebar: give nested project chats the full options menu and a pin quick-action - Chats nested under a pinned project now render through the shared chat row, so they get the same hover kebab (Rename, Pin, Move, Export, Archive, Delete) plus a pin/unpin quick-action, matching top-level chats. - Show more/less now uses the muted nav token with an override, since the sidebar-nav-btn color rule otherwise won so the label matched the chat rows; it now reads clearly dimmer in both light and dark mode. * Projects list view, pinned-chat promotion, and sidebar polish - Pinning a chat inside a project now promotes it into the pinned chats list and removes it from the project's nested list, so it shows once in the Pinned section. - Sidebar highlights only the open chat, not its parent project folder, while a chat inside the project is active. - Rename project uses the edit icon instead of the compose icon so it no longer matches New chat. - Projects page is now a list: Name and Modified columns, a pin indicator on pinned rows, and the row options menu on hover, replacing the card grid. * Redirect after deleting a project viewed via a thread-only URL commitDelete only redirected when the ?project= param matched the deleted project. On a thread-only URL the project is resolved from the thread into the runtime store, so also compare that resolved id, otherwise the user is left on a deleted thread. * Restore the unpin quick-action on pinned chats Pinned rows in Recents already reserved room for it, but only the kebab rendered. Show the unpin button on hover, left of the options button. * Projects list: alignment, spacing, no icon overlap, fit-to-height paging - Add side padding to the list and more room after the folder icon. - Column header now shares the row layout so Name and Modified line up with the values. - Pin indicator and options button swap by display so they no longer overlap. - Show more only appears once projects exceed the page height and reveals five more per click. * Projects list: align Name to the folder icon and narrow the table - Drop the header leading spacer so Name starts at the folder icon edge; the right-anchored columns keep Modified aligned. - Narrow the page to max-w-4xl so the table is less wide. * Projects list: widen slightly and add top spacing Bump the page to max-w-5xl and increase the top padding so the header sits a little lower. * Projects list: infinite scroll instead of Show more - First page fills the viewport, then a sentinel loads another batch as it scrolls into view, re-observing after each load so it keeps filling. - Search still filters the full project set and shows every match uncapped. * Projects list: drop dividers for a rounded-row hover style Remove the row and header borders and give each row a rounded hover fill so the list reads cleaner, closer to a modern file list. * Projects list: more row spacing and a tighter hover radius Increase row padding so projects sit further apart, and drop the hover corner radius so it reads as a rounded rectangle rather than a pill. * Projects list: more space below the title Increase the list top margin so the header sits further from the rows. * Project landing: narrow slightly and drop the Sources New badge Reduce the landing column to 44rem and remove the New badge from the Sources tab. * Project switcher: rounded-rectangle rows instead of pill The switcher rows are short, so the shared 12px item radius reads as a pill. Scope a smaller radius to this menu so the highlight is a rounded rectangle. * Project landing: add a header options menu Add a kebab menu next to the project title with Rename project, Pin or Unpin project, Export, and Delete project, reusing the existing project actions. * Project switcher: round the scrollbar-side corners Revert the earlier item-radius tweak. The container corners were squared on the scrollbar side because the container itself scrolled; move the scroll to an inner wrapper so the rounded container never scrolls. * Projects: keep row kebab focusable, gate off-route dialogs, refresh history on delete * Projects: fix delete copy, refresh history on landing delete, gate chat-delete dialog off-route --------- Co-authored-by: shimmyshimmer <info@unsloth.ai>
…HIP crash (unslothai#7233) * fix(rocm): prepend system ROCm libs on native Linux to avoid bundled HIP crash Prebuilt llama.cpp bundles ship their own ROCR/HIP runtime which can be incompatible with the host's amdkfd kernel driver, causing hsa_init() to crash or report zero devices. The llama-server then silently falls back to CPU while the UI reports GPU. The existing workaround (_wsl_system_rocm_lib_dirs) that prepends /opt/rocm/lib to LD_LIBRARY_PATH was gated on WSL (/dev/dxg) only, leaving native Linux AMD hosts unprotected. This commit adds _native_linux_system_rocm_lib_dirs(), a parallel helper gated on: - Linux platform (not WSL) - /dev/kfd present (bare-metal AMD compute) - Bundle contains bundled HIP libs (libggml-hip.so) - System has libhsa-runtime64.so(.1) It is called from both _llama_server_env_for_binary (serve-time) and binary_env (install-time validation), directly after the WSL block in both paths. Fixes unslothai#7208 Fixes unslothai#7208 * Add UNSLOTH_LLAMA_NO_SYSTEM_ROCM opt-out to native-Linux system ROCm preference for PR unslothai#7233 Lets a host where the bundled runtime works but system ROCm is mismatched keep the bundle. Mirrored in llama_cpp.py and install_llama_prebuilt.py. * Prefer env-configured ROCm root over /opt/rocm fallback for PR unslothai#7233 Put HIP_PATH/HIP_PATH_57/ROCM_PATH-derived roots before /opt/rocm so a stale /opt/rocm can't shadow the driver-matching install the env vars point at. Mirrored in llama_cpp.py and install_llama_prebuilt.py. * Match versioned libggml-hip.so via glob so the native-Linux ROCm fix fires for PR unslothai#7233 * Clarify native-Linux ROCm prepend uses the consistent system stack for PR unslothai#7233 * llama_cpp: tighten native-Linux ROCm prepend comments (no code change) --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>
…(gfx1201) (unslothai#7292) * studio: extend the _grouped_mm null-kernel guard to Linux ROCm RDNA4 torch._grouped_mm has a null HIP kernel on RDNA4 (gfx1200/gfx1201) at ROCm <= 7.12 (fixed in 7.13; ROCm/TheRock unslothai#5284). The existing guard that registers a Python mm/bmm fallback was win32-only, so Linux gfx1201 (e.g. R9700 Pro on Ubuntu) hits the null kernel -> illegal instruction during training. Extract the fallback registration into a module-level helper (_install_grouped_mm_cpu_fallback) and add a Linux branch that installs it, gated on gfx1200/gfx1201 AND HIP < 7.13 so NVIDIA/CUDA and every non-RDNA4 AMD arch are untouched, and it is a no-op on fixed runtimes. The Windows path now calls the same helper with identical behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve HIP version from torch.__version__ when version.hip is unset for PR unslothai#7292 AMD SDK / Radeon ROCm wheels leave torch.version.hip empty and encode the version only in torch.__version__ (e.g. +rocm7.12). The Linux gfx120X guard parsed version.hip only, so those affected installs skipped the fallback and still hit the null _grouped_mm kernel. Mirror the Windows parse: version.hip, then the embedded rocmX.Y, then assume affected unless a post-fix rocmsdk wheel. * Scan all GPUs and add RDNA4 name fallback for _grouped_mm guard in PR unslothai#7292 * worker.py: tighten gfx120X Linux guard comments (no code change) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…nslothai#7290) * Recognize Radeon 8065S (Gorgon Halo / Ryzen AI Max 400) as gfx1151 * Classify Radeon 8065S (Gorgon Halo) as unified memory in ROCm OOM guard * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…tials (unslothai#7286) * fix(install.ps1): use ordinal IndexOf when stripping index URL credentials On non-English Windows locales, culture-aware String.IndexOf can mis-locate punctuation-only markers like ://, which corrupts scheme and authority parsing and crashes Remove-IndexUrlCredentials with a Substring ArgumentOutOfRangeException (issue 7279). Force Ordinal comparison for URL scheme/host parsing. Fixes unslothai#7279 * Condense the ordinal parsing comment in Remove-IndexUrlCredentials --------- Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
) + PCI detection hint (unslothai#7293) * install.sh: route Strix (gfx1151/gfx1150) to the AMD arch index on rocm7.2, add PCI hint Two Linux install fixes for AMD Strix Halo / Strix Point: 1. unslothai#7264: Strix reverts to rocm7.2. Modern ROCm (7.3+) caps to the generic rocm7.2 index and the Radeon repo can be unavailable, so gfx1151/gfx1150 landed on a non-arch-specific build (torch 2.11+rocm7.2) instead of repo.amd.com/rocm/whl/gfx<arch> (torch 2.11+rocm7.13, AMD's real Strix fixes). The reroute to that arch index only fired on rocm7.1; broaden it to rocm7.2 too. Only acts when a gfx1151/gfx1150 is actually detected, so other arches on rocm7.2 pass through unchanged. 2. Rows about Strix not detected -> CPU-only: when no GPU is detected but an AMD display GPU is on the PCI bus, print a targeted hint (ROCm kernel stack / /dev/kfd missing) instead of only the generic docs pointer. Purely additive diagnostic; does not change the torch index decision. * Address review on PR unslothai#7293: gate PCI hint on ROCm-detection failure, fix test marker, use read builtin - Only show the 'ROCm cannot see the GPU' hint when _has_amd_rocm_gpu fails; a detected-but-too-old ROCm (rocminfo works, wheels need 6.0+) has its own path. - Update test_previous_torch_pin.sh to the stable 'Strix Halo / Strix Point:' marker after the heading reworded (the old grep broke the ordering assert). - _amd_gpu_present_via_pci: read builtin instead of spawning cat twice per device, and guard /sys/bus/pci/devices existence. * install.sh: reroute Strix on any generic index older than the arch build Generalize the Strix reroute from the hardcoded rocm7.1/rocm7.2 match to a version compare against the arch index's own build (rocm7.13): - backwards: rocm6.0-6.4 and rocm7.0 now reroute (were silently missed) - forwards: any future intermediate rocm7.x below 7.13 reroutes; rocm7.13+ is left alone so a generic index that already carries the fix is not downgraded to the arch build _rocm_index_below does an integer major.minor compare (so rocm7.2 < rocm7.13); non-rocm, arch (gfx), and unparseable URLs return false, so NVIDIA/CPU and the arch index itself are untouched. Reroute still fires only for gfx1150/gfx1151. * install.sh: tighten _amd_gpu_present_via_pci comment (no code change) * install.sh: match the index leaf in the Strix version reroute (unslothai#7293 review) Address two review points on the rocm-version reroute: - Parse the final path segment (_torch_index_leaf) instead of grepping the whole URL. A custom mirror whose base path holds its own rocm token (e.g. .../rocm7.13/cache/rocm7.2) previously matched the base and skipped the reroute; now it compares the leaf (rocm7.2) like the nearby index-family logic. Renamed the helper to _rocm_leaf_below and switched the case selector to $_torch_index_leaf. - Replace the stale test_strix_override_only_fires_on_rocm71 (which passed by matching the new rocm7.13 comment) with an executed test that runs _rocm_leaf_below and asserts rocm6.0-7.12 reroute while rocm7.13+/gfx/cu leaves do not. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install.sh: keep gfx probes non-fatal under set -e (unslothai#7293 review) The Strix reroute now matches every rocm* index, not just rocm7.1, so its gfx detection runs on all AMD installs. Each `_gfx_all=$(rocminfo|amd-smi | grep -oE gfx...)` returns 1 when grep finds no match, which under set -euo pipefail aborts the installer before the next fallback runs (e.g. rocminfo present but emitting no gfx token). Append `|| true` to the three probes, matching the display block that already guards this. Add an executed regression test (shimmed rocminfo/amd-smi) that fails if any probe becomes fatal again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: move sidebar search into the header
Put the search action as an icon button next to the sidebar toggle in
the header instead of a full-width nav row, so New Chat is the only
fixed row above the scrolling list. The search row is kept for the
collapsed icon rail only. Also add a small bottom gap under New Chat
when it is pinned during scroll.
* Studio: keep search row on custom-titlebar platforms
The header search button only renders on mac/web where the brand row
shows. On win/linux custom titlebars there's no header button, so keep
the full-width search row visible instead of hiding it.
* Studio: address review on sidebar search tooltip
- Hide the search tooltip on mobile (hidden={isMobile}), matching the
SidebarMenuButton tooltip convention.
- Show Cmd K on Mac and Ctrl K elsewhere instead of a hardcoded glyph;
the search dialog binds both meta and ctrl. Uses getClientPlatform so
it is correct on web too, not just Tauri.
…nslothai#7311) * Studio: reuse MLX prompt cache across turns instead of re-prefilling * clean up * key prompt cache on what the KV covers * skip windowed KV caches past their window * verify prefix coverage before caching KV
… model switching (unslothai#7313) * Improve unsloth start runtime lifecycle * Remove speculative Gemma prompt override * Polish model download progress output * Refine unsloth start status output * Clarify unsloth readiness banner * Clarify model reuse and switching output * Queue model switches behind active inference * Tighten unsloth start model switching * Reduce model switch bookkeeping * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Studio re-exec compatibility * Recheck sidecar reservation after inference drain * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pass start marker through child environment * Fix key redaction, switch-waiter ordering, and stop/messaging gaps for PR unslothai#7313 - Redact minted sk-unsloth keys from the startup-failure log tail: the early key marker lands in the server log before the model load finishes, so a load-phase crash printed a live key to the terminal - Deregister a finished switch waiter before releasing the swap gate so a swap on another event loop cannot count it as still queued and unload the model the finished request is about to generate against - Warn on same-repo quant switches: an explicit variant replaces the resident weights for every attached session, but the repo ids match so no switch warning was printed - Note the agent exit code when it is nonzero so the server keep-alive message does not read as a successful session - Use taskkill /T in unsloth studio stop so llama-server children stop too * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten comments in start, studio, and inference changes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
…nslothai#7325) The enforcing pip scan-packages hf-stack shard fails on two CRITICAL staged-dropper findings in unsloth-zoo test files: tests/test_mlx_save_export_regressions.py and tests/test_vision_collator_audio.py. Both are false positives: the combination heuristic matches a /tmp path literal alongside unrelated subprocess/import references in the same file, but those are mocked test fixtures (monkeypatch.setattr on subprocess, asserted /tmp path strings), not droppers. Add both to the reviewed allowlist so the gate stops red-failing on legitimate test code. The scan then exits 0 on both the hf-stack shard and a direct unsloth-zoo scan.
unslothai#7216) * studio: show system-wide VRAM in the multi-GPU System tab view on ROCm The System tab's per-GPU list comes from get_visible_gpu_utilization. When amd-smi is unavailable (always on Windows, minimal Linux installs) it fell back to torch, whose readings are process-local: on Windows WDDM hands each process its own budget, so a model held by the separate llama-server process read as ~0 VRAM used even with the GPU full (unslothai#7072). The primary-GPU endpoint already compensates with system-wide sources -- Windows Performance Counters (Task Manager's source) and Linux DRM sysfs -- but the multi-device endpoint never got those fallbacks. Add per-GPU variants of both sources and overlay them onto the torch fallback: _rocm_windows_perf_counter_vram_per_adapter_gb() attributes Dedicated Usage per physical adapter (phys_<N> in the counter instance name), and _rocm_linux_sysfs_vram_per_card_gb() reads mem_info_vram_{used,total} per DRM card. _overlay_system_wide_vram() applies them to the device list, ROCm-only, best-effort: unmatched adapters and ambiguous card counts keep the torch figures, and NVIDIA paths are untouched. Fixes unslothai#7072 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: match VRAM overlay sources by device, honor unified memory, unblock the loop Five review fixes on the multi-GPU system-wide VRAM overlay: 1. Linux: match DRM cards to devices by PHYSICAL index instead of a positional zip, so a reordering visibility mask (HIP_VISIBLE_DEVICES=1,0) no longer swaps each card's figures onto the other GPU (which would mislead auto_select_gpu_ids and the coexistence checks). An index with no matching card keeps its torch figures. 2. Linux: skip the overlay for a device whose sysfs total is below torch's -- on unified-memory APUs (Strix Halo) mem_info_vram_total is only the small dedicated slice while torch sees the GTT-backed pool, and _apply_unified_memory_correction already defines larger-total-wins. 3. Windows: group counter instances by adapter LUID, not the phys_<N> suffix -- separate adapters each read phys_0, which collapsed every GPU into key 0. LUIDs are mapped to 0-based positions by ascending value as the closest stand-in for device order. 4. Windows: pair the system-wide usage with the physical capacity from get_device_properties (as the primary-GPU fallback does) -- under WDDM mem_get_info's "total" is the process budget, which misreported capacity and pushed utilization to 100%. 5. Run get_visible_gpu_utilization off the event loop in the /hardware/visible route (asyncio.to_thread, the repo's convention): the ROCm fallbacks can shell out to PowerShell with a 5s timeout, which would stall every other request while the System view polls. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: skip the system-wide VRAM overlay for relative GPU indices The overlay matches its per-GPU sources (Windows perf counters, Linux sysfs) by physical device index, but under a UUID/MIG visibility mask the torch fallback enumerates ordinals and reports index_kind == "relative", where `index` is a visible ordinal, not a physical id. Applying the overlay there let card/adapter 0's system-wide VRAM overwrite the torch reading of a process that actually exposes physical GPU 1, misleading auto_select_gpu_ids and the coexistence checks. Gate the overlay on index_kind == "physical"; relative-index paths keep the torch fallback. * studio: drop the unreliable Windows VRAM overlay, keep the Linux one The multi-GPU system-wide VRAM overlay is now Linux-only. The Windows per-adapter Performance Counter path could not be made correct: the wildcard Get-Counter query also returns non-ROCm/iGPU adapters and LUID order is not the ROCm device order, so an adapter's usage could be overlaid onto the wrong GPU; and it read only Dedicated Usage, missing WDDM shared memory on unified-memory GPUs (Strix Halo), overstating free VRAM. Rather than misattribute VRAM and skew placement decisions, Windows keeps the process-local torch fallback (no regression vs before this PR); Linux DRM sysfs -- matched by physical index -- still fixes unslothai#7072 for the reporter's native-Linux ROCm case. Removes _rocm_windows_perf_counter_vram_per_adapter_gb and _torch_props_total_gb. * studio: key sysfs VRAM by DRM card number so filtering can't renumber cards _rocm_linux_sysfs_vram_per_card_gb dropped cards with a zero total or unreadable files and then the overlay enumerated the compacted list, so if card0 was dropped, card1's usage was assigned to physical GPU index 0 (equal-capacity GPUs slip past the unified-memory total guard). Return {card_number: (used, total)} and match a device to its card number directly: a hole stays a hole -- device 0 keeps its torch figures when card0 is absent, and card1 maps to device 1. * studio: key system-wide VRAM by ROCm ordinal, not raw DRM card number When a non-amdgpu adapter (Intel iGPU, a display-only card) owns an earlier DRM slot, DRM card numbers stop equalling ROCm device ordinals -- Intel card0 plus AMD card1/card2 gives ROCm devices 0/1, so keying the sysfs overlay by card number handed ROCm device 1 card1's data (AMD device 0) and left device 0 on stale torch figures, corrupting free-VRAM placement on equal-capacity GPUs. Only amdgpu cards expose mem_info_vram_*, so the glob already excludes foreign adapters; order the surviving cards by their PCI address (ROCm/HIP's default device order, read from each card's device symlink) and key by that position -- the ROCm physical ordinal, which is what the overlay matches against dev index. An unreadable / zero-total amdgpu card still consumes its ordinal so a later card is never renumbered onto its slot. * studio: skip the VRAM overlay under layered HIP-over-ROCR masks ROCR_VISIBLE_DEVICES filters physical GPUs at the HSA/ROCr layer, and a HIP_VISIBLE_DEVICES set on top selects WITHIN that already-filtered set (apply_gpu_ids sets HIP while leaving an inherited ROCR mask in place). When both are active _get_parent_visible_gpu_spec() prefers the HIP value, so the reported device index is a ROCR-relative ordinal, not a physical GPU id -- overlaying DRM-sysfs figures by that index would pull another GPU's usage (e.g. ROCR=2,3 + HIP=1 is physical GPU 3, but the overlay would read card 1), and equal-capacity cards bypass the total-size safeguard. Detect layered masks and keep torch's process-local figures there rather than risk misattribution; a single mask still leaves the index physical and is overlaid as before. * studio: only overlay whole-card VRAM onto 1:1 ROCm devices The overlay guard only skipped the case where sysfs total < torch total (unified-memory APUs), so a partitioned ROCm device (MI300 in CPX mode) -- where HIP exposes several logical devices per physical card but sysfs reports the whole card's aggregate -- passed the guard: the card total exceeds a partition's torch total, and the overlay overwrote the partition with whole-card usage and capacity, letting downstream selection think a partition had the entire card free. Require the sysfs card total to match the torch device total (within ~10%) so a mismatch in either direction -- unified memory (sysfs smaller) or partitioning (sysfs larger) -- keeps torch's figures. * studio: treat CUDA-over-ROCR as layered, enumerate AMD cards by driver Two remaining mismatches between the reported device index and the DRM card the overlay reads: - On ROCm the HIP layer honors CUDA_VISIBLE_DEVICES as well as HIP_VISIBLE_DEVICES, so a CUDA mask composed over ROCR layers identically: ROCR=2,3 with CUDA=1 is physical GPU 3, yet the spec reports the ROCR value [2,3] and the device was labeled index 2, overlaying card 2's usage onto GPU 3. The layered check now treats ROCR combined with either HIP or CUDA as layered. - The ROCm device set is now enumerated by bound driver (device/driver resolves to amdgpu) instead of by the presence of mem_info_vram_*. An AMD device with incomplete sysfs support (some APUs expose no VRAM files at all) was omitted by the glob entirely and shifted every later card down one ordinal, letting a similar-capacity GPU pass the total guard with another device's usage. Such a card now consumes its ordinal and simply yields no entry. * studio: honor GPU_DEVICE_ORDINAL and require an unambiguous card mapping Two remaining ways the reported device index could be matched to the wrong DRM card: - GPU_DEVICE_ORDINAL is a supported ROCm visibility variable that _get_parent_visible_gpu_spec() never consults, so GPU_DEVICE_ORDINAL=1 surfaces physical GPU 1 as torch ordinal 0 and it was mislabeled index 0, overlaying card 0's usage onto GPU 1. The mask check now covers it, and is renamed _rocm_device_index_unreliable() to say what it actually decides. - driver == amdgpu is only a SUPERSET of the ROCm-visible set: an amdgpu-bound adapter HIP cannot enumerate (an unsupported older AMD GPU beside a supported one) still took an ordinal and shifted every real compute device. There is no torch-side PCI identity to match against, so the overlay now requires the amdgpu card count to equal the device count -- exactly the condition under which position-in-PCI-order is a sound 1:1 mapping. Any disagreement keeps torch's process-local figures: less informative, never misattributed. * studio: keep the VRAM overlay working for masked GPU subsets The card-count guard compared the amdgpu card list against the VISIBLE device list, so any visibility mask disabled the overlay outright: HIP_VISIBLE_DEVICES=1,3 on a four-GPU host gives two devices against four cards. Those masked GPUs then kept reporting process-local torch usage, hiding VRAM held by llama-server and letting the training/chat placement checks overestimate free memory -- the exact problem the overlay exists to fix. The count check now applies only when no visibility mask is active, which is the case where the reported devices really are the whole host and a mismatch means an amdgpu adapter ROCm cannot enumerate is shifting the ordinals. Under a mask the subset is expected, so each device's physical index is validated individually instead: the per-card lookup bounds-checks it and the total-size guard rejects a card whose capacity does not match the device's. * studio: match GPUs to DRM cards by PCI identity, not by position Every mapping bug on this PR came from the same root cause: there was no authoritative link between a reported device index and a DRM card, so the overlay kept inferring one positionally and each heuristic broke on a new host shape -- foreign adapters on earlier DRM slots, cards with no VRAM sysfs, and most recently amdgpu-bound adapters HIP cannot enumerate, which the count guard could only catch on an unmasked host and therefore missed under any mask. Use the link ROCm itself enumerates from. KFD topology (/sys/class/kfd/kfd/topology/nodes/<N>/properties) lists exactly the GPUs HIP exposes -- GPU nodes in node-id order are HIP's device order -- and each carries its PCI location, so index N there IS physical device N with a stable identity. DRM sysfs now supplies system-wide VRAM keyed by that same PCI address, and the overlay is a join on it. Every previous skew becomes a failed join rather than a misattribution: an unenumerable adapter has no KFD node so it never takes an ordinal, a foreign adapter contributes no entry, and a masked subset resolves each physical index directly. That removes the count heuristic and its mask exception entirely. With no KFD topology there is no identity to join on, so the overlay is skipped rather than guessing positionally. * studio: require verified host visibility and AMD-only KFD nodes Three ways the identity map could still be built on a false premise: - The NVIDIA open kernel module registers KFD topology nodes with a positive SIMD count, so an earlier NVIDIA node shifted every AMD ordinal and ROCm device 1 resolved to AMD GPU 0. GPU nodes now require vendor_id 4098 (0x1002), the same filter install.sh already applies for this exact reason. - A GPU node with an unreadable properties file or no location_id was skipped, which silently shifted every later ordinal. Both now fail the whole map closed, so the overlay is disabled rather than misattributing. - A container exposing only some render devices through device cgroups sets no visibility variable, yet torch compacts what it can see to ordinals from zero while the host-mounted KFD and DRM trees still list every GPU. Nothing in the reported payload distinguishes that from a full host, and torch exposes no PCI id to check against, so the overlay now runs only when host visibility is positively verified: no visibility mask AND device count equal to the host GPU count. That also subsumes the previous layered-mask and GPU_DEVICE_ORDINAL checks, so _rocm_device_index_unreliable() is gone. This trades coverage for correctness: masked subsets and filtered containers now keep torch's process-local figures instead of a mapping that cannot be verified. * Fix the multi-GPU VRAM overlay docstring for PR unslothai#7216 The docstring claimed a reordering mask keeps each card on the right GPU, but the overlay skips any active visibility mask and keeps torch's figures. State the actual gating instead. * Tighten comments in the multi-GPU VRAM overlay and its tests Collapse the verbose docstrings and inline explanations added for the Linux ROCm system-wide VRAM overlay to succinct one-liners, keeping the non-obvious rationale (fail-closed KFD mapping, PCI-identity join, mask gating, the 10% whole-card guard). Comments only, no behavior change. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
… mode (unslothai#7218) * studio: classify embedding models from the HF cache and honor offline mode is_embedding_model() went straight to huggingface_hub.model_info() for any repo id, so in offline mode (no DNS, or HF_HUB_OFFLINE set) selecting an already-downloaded model hung on network retries that could never succeed and training/export never started (#6817). Check the local HF cache first: a sentence-transformers repo carries modules.json in its snapshot (the same marker used for local paths), so a cached model is classified with no network call. When HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE is set, anything not positively an embedding model returns False without a network call instead of retrying a doomed request. Online, uncached lookups still fall through to model_info(), so tag-only embedding models (feature-extraction) are unaffected. Adds _embedding_marker_in_hf_cache() over the existing _iter_hf_cache_snapshots. * studio: judge the active cached revision, harden the cache probe, stop stub leaks Three review fixes on the cache-first embedding detection: 1. Prefer the revision refs/main resolves to. The HF cache keeps snapshots of older revisions, so an any-snapshot scan could classify a repo by a stale revision -- e.g. a repo that used to be a sentence-transformers model would short-circuit even the online lookup. When refs/main is recorded, only its snapshot is consulted; the newest-first scan remains the fallback for caches with no ref. 2. Keep the cache probe inside the detection error boundary. The snapshot iterator stat()s entries and could raise if a cached model is deleted concurrently, propagating a 500 out of the config/check-embedding routes. _embedding_marker_in_hf_cache now catches everything and reads as not-cached, so callers keep their normal Hub/offline fallback. 3. Stub loggers/structlog in the test only when the real modules are absent (try-import, mirroring test_windows_gpu_detection_mock), so collecting this file first can no longer shadow the real packages for later tests in the same pytest process. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: treat a missing active-ref snapshot as a cache miss, don't cache offline misses Two review fixes on the cache-first embedding detection: 1. When refs/main is recorded but points at a commit whose snapshot dir is absent (partial download / cache pruning), the recorded ref is still authoritative: return None (cache miss) instead of falling through to scan older snapshots, which could report a stale historical revision's modules.json as the active one -- the same stale-cache class this helper avoids. 2. Do not cache the offline negative. When HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE is set and the repo is not positively an ST model from modules.json, is_embedding_model stored False under the (model_name, hf_token) key shared with online lookups; after the env var cleared in the same process, a tag-only (feature-extraction) embedder returned the cached False and never reached model_info(). The offline negative is now returned without caching. * studio: defer online embedding detection to the Hub, re-probe offline The local modules.json marker short-circuited is_embedding_model() even online, so a repo that dropped (or added) the marker since it was cached was judged by its stale local revision instead of the current remote one. Online now treats model_info() as authoritative and uses the cache marker only as an uncached fallback when the Hub is unreachable, so a transient failure never poisons the memo. Offline re-probes the marker on every call without consulting or populating the memo, so a model downloaded later in the session (or a cached online negative that predates the download) is detected. _embedding_marker_in_hf_cache() now treats an unreadable refs/main (a non-FileNotFoundError OSError) as a cache miss rather than scanning stale history -- only a genuinely missing ref enables the fallback scan. * studio: harden offline embedding detection against empty refs, offline flips, and cache casing - _embedding_marker_in_hf_cache: an existing-but-empty/whitespace refs/main (a partial write or in-progress truncate-and-rewrite) now reads as a cache miss (None) instead of falling through to scan stale snapshots; only a genuinely missing ref enables the historical scan. - is_embedding_model: while offline, retain a positive already confirmed online this session (model_info only ever memoizes Hub-derived results), so _hf_offline_if_dns_dead() flipping the process to offline mid-load can't downgrade a verified tag-only embedder to False. Cached negatives are still bypassed and re-probed. - resolve_cached_repo_casing + settings route: persist the embedding model in the casing its local HF cache dir uses. Validation accepts a case-insensitive cache hit, but an offline SentenceTransformer load resolves the cache by exact case, so storing the requested spelling (baai/bge-m3 vs models--BAAI--bge-m3) made the model fail to load on a case-sensitive filesystem. * studio: reuse the exact-match-first case resolver and preserve the default Replace the ad-hoc resolve_cached_repo_casing with the existing resolve_cached_repo_id_case, which already prefers the exact-case cache dir before any case variant and tie-breaks variants deterministically -- so an exact requested id is never rewritten to a differently cased directory just because iterdir() happened to yield it first. Skip the normalization entirely when the submitted model equals the default: rewriting its casing would make set_rag_embedding_model()'s exact-string default comparison treat it as a custom override, pinning it so later changes to the configured default stop taking effect. * studio: don't let a stale cache marker mask a permanent Hub error is_embedding_model's Hub-failure fallback consulted the local modules.json marker for ANY model_info() exception, so a permanent error -- a deleted repo, a gated repo without credentials, or a typo that matches stale cache casing -- could pass online validation on a stale marker instead of returning the documented 409, and the persisted model could then fail when the loader refreshes from the Hub. Classify permanent Hub errors (RepositoryNotFound, GatedRepo, RevisionNotFound, EntryNotFound) as False, matching the nearby GGUF/vision detectors, and reserve the cache fallback for transient/5xx failures. * studio: honor TRANSFORMERS_OFFLINE in the embedding preflight, skip casing for local paths - The embedding-model save reached the offline-aware is_embedding_model() only after two preflight helpers made direct huggingface_hub calls that honor just HF_HUB_OFFLINE: _st_module_subdirs() downloads modules.json and the security scan fetches Hub metadata twice. In a TRANSFORMERS_OFFLINE-only session those blocked on network timeouts before the offline return, so saving an already cached model stalled. Both now consult a canonical hf_env_offline() helper -- the download passes local_files_only, and the metadata-only security scan short-circuits to its documented fail-open instead of burning both timeouts. - Skip cache-casing normalization for local paths: a relative directory such as "org/model" is loaded from disk, so rewriting it to a case-insensitive HF cache collision ("Org/model") would stop resolving to that directory and be read as a Hub repo id instead. * studio: never skip the security scan on TRANSFORMERS_OFFLINE alone The previous commit skipped the Hub security scan whenever either offline flag was set, but huggingface_hub honors only HF_HUB_OFFLINE: under a TRANSFORMERS_OFFLINE-only session the later SentenceTransformer load still reaches the network, so the scan was being skipped while the repo's pickle could still be downloaded and deserialized -- waving through exactly what _guard_model_security exists to block. Split the flags: hf_hub_offline() (HF_HUB_OFFLINE, the only one that actually prevents a fetch) gates the security short-circuit, while hf_env_offline() (either flag, the user's intent) is used only where local-only behavior is forced explicitly. The SentenceTransformer load now passes local_files_only from that intent, so TRANSFORMERS_OFFLINE genuinely stops the loader fetching instead of merely being assumed to. * studio: short-circuit the security preflight under either offline flag With the loader now pinned to the local cache by local_files_only = hf_env_offline(), a TRANSFORMERS_OFFLINE-only session can no longer fetch anything -- yet the preflight still fell through to two model_info() attempts on 10s and 20s timeouts, stalling every save and load of an already-cached embedder for half a minute before failing open anyway. Skip the metadata-only scan whenever either flag is set. The scan's job is to stop a poisoned pickle being downloaded and deserialized, and nothing can be downloaded under that predicate; the residual case -- a model cached BEFORE it was flagged -- is the same fail-open this function has always documented for an unavailable scan, and is exactly what HF_HUB_OFFLINE already did. That safety argument depends on every loader behind the gate honoring the same predicate, so it is pinned as a test invariant instead of a comment: removing local_files_only from the SentenceTransformer construction now fails the suite. Drops the short-lived hf_hub_offline() helper, which no longer has a caller. * studio: scope the offline scan bypass to callers that load local-only The previous commit put the offline short-circuit inside _fetch_security_status, which is the malware gate shared by every loader -- so TRANSFORMERS_OFFLINE=1 disabled it for all of them, while only the RAG embedder had been changed to pass local_files_only. MLX inference (core/inference/worker.py -> FastMLXModel .from_pretrained), training and export call from_pretrained with no local-only argument, and huggingface_hub ignores that flag, so those paths could still fetch and deserialize an unscanned model with the gate switched off. The bypass is now an explicit local_only_load argument, defaulting to False, and only the two RAG embedding callers -- whose loader is pinned to the local cache by the same predicate -- opt in. Tests pin both halves: the shared gate must still scan under either offline flag by default, and no other caller may pass local_only_load without constraining its loader. * studio: capture offline state once, and probe the ST cache root Two holes in the offline embedding path: - _get() read hf_env_offline() twice: once inside _guard_model_security and again for local_files_only. _hf_offline_if_dns_dead() mutates the process-wide offline vars and restores them on exit, so a concurrent load could see True in the guard -- skipping the Hub malware scan -- and False by the time the constructor ran, fetching and deserializing the unscanned repo and breaking the very invariant that licenses the bypass. The value is now read once in _get() and passed to both; _guard_model_security takes it as an argument instead of re-deriving it. - The cache probe searched only HF_HUB_CACHE. SentenceTransformer downloads into SENTENCE_TRANSFORMERS_HOME when that is set, using the same models--org--name/snapshots layout under a different root, so a model fully present there looked uncached and was rejected with a 409 offline even though the local-only loader could load it. Snapshot lookup now covers both roots. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: probe the cache the ST loader actually uses, and require it be loadable Adding SENTENCE_TRANSFORMERS_HOME to the shared snapshot iterator was too broad in one direction and too narrow in another: - _get() builds SentenceTransformer with no cache_folder, so with ST_HOME set it searches THAT root only, never the Hub cache. Probing the union let offline validation pass on a repo cached only in the Hub cache, after which the loader looked in ST_HOME and failed. The Sentence-Transformers probe now resolves to exactly one root: ST_HOME when set, the Hub cache otherwise. - The shared iterator is also used by the GGUF detectors, whose downloads go through hf_hub_download with no cache_dir and therefore really do use the Hub cache. It is back to Hub-cache-only so detection cannot pick a snapshot the GGUF load will not find. - Casing normalization ran through resolve_cached_repo_id_case, which scans the Hub cache, so with ST_HOME set the requested spelling was persisted unchanged and the exact-case offline load missed the differently cased directory that detection had just accepted. It now resolves against the same roots detection uses, exact match first. - A snapshot carrying only modules.json no longer counts as cached: the online security preflight downloads that single file itself, and a partial download leaves it behind, so validation passed for a snapshot with no weights and the first RAG load then failed. A hit now requires the marker plus a config and at least one weight file. * studio: thread the captured offline state into the module probe, fix the gate shard - _st_module_subdirs() re-read the process env for its local_files_only. With _hf_offline_if_dns_dead() flipping those vars from another thread, a load that captured local_only=False could still force this probe local-only, get () back because modules.json is not cached, and leave the scan with NO module load roots -- a Hub-flagged pickle under 0_Transformer/ would then pass as an unreferenced nested artifact while the loader fetched and deserialized it. It now takes the captured predicate as an argument, and the settings route reads the state once and uses that single value for both the probe and the scan. - Skip ST-cache casing on the llama-server backend. Nothing there loads through SentenceTransformer: the embedder derives a GGUF companion from the saved spelling and fetches it from the HUB cache, so normalizing to an ST_HOME spelling would point it at a repo _hf_gguf_backend_error() never validated (BAAI/bge-m3-GGUF instead of the checked baai/bge-m3-GGUF). - Fix the security-gate shard, which the signature change had broken: the direct _guard_model_security / _st_module_subdirs callers now pass the new argument (they were raising TypeError before reaching any assertion), and the casing tests patch utils.models.resolve_st_cached_repo_id_case, which the route actually calls, instead of the Hub-only resolver it no longer uses -- those patches were being silently ignored. * studio: accept only torch-loadable weights in the offline ST probe; fix re-export lint _snapshot_is_loadable_st_model accepted a cached snapshot whose only weights were .onnx (or .pt), but the RAG loader builds SentenceTransformer with the default torch backend, so such a snapshot passed offline validation and then failed on the first load, the exact validate-then-fail this helper exists to prevent. Restrict _ST_WEIGHT_SUFFIXES to .safetensors and .bin and add a regression test for an ONNX-only snapshot. Also teach scripts/verify_import_hoist.py that names listed in a module-level __all__ are uses, so the legitimately added resolve_st_cached_repo_id_case re-export in utils/models/__init__.py no longer trips HOISTED-IMPORT-UNUSED. Covered by two new self-test cases. * studio: probe the exact repo dir and revision an offline load resolves The cache probe modelled the cache loosely rather than modelling what SentenceTransformer actually does with local_files_only=True: - It merged snapshots across every case-variant repo dir and then read refs/main from whichever held the newest one. With both models--baai--bge-m3 and models--BAAI--bge-m3 present, a complete embedding snapshot in the directory the loader opens could be judged by a newer partial snapshot in the other, failing validation for a usable model. It now selects the ONE directory the loader opens, by the same exact-case-first rule resolve_st_cached_repo_id_case uses to choose the spelling that gets persisted. - It fell back to scanning historical snapshots when refs/main was absent. With local_files_only the default revision is resolved THROUGH that ref, so a snapshot directory alone is not discoverable: the settings request succeeded and the loader then failed at first indexing. A missing, empty or unreadable ref is now a cache miss, and the historical scan is gone. The tests exercise the real lookup against a built cache tree instead of patching the snapshot iterator, so they now cover the directory selection and ref resolution the loader depends on. * studio: record refs/main in the ONNX-only probe test The ONNX-only regression test predates the refs/main requirement, so after that change it returned None (a cache miss for want of a ref) before ever reaching the weight-format check it exists to make. Recording the ref restores its intent: the snapshot resolves, and the answer is False because an ONNX export is not loadable by the RAG loader's default Torch backend. * studio: recognize base-model weight files and gate the offline positive on a materialized snapshot _snapshot_is_loadable_st_model matched any .safetensors/.bin by suffix, so a partial cache carrying only a commonly published non-weight bin such as training_args.bin (or an adapter-only artifact) passed offline validation and then failed the local_files_only load at first indexing. Match recognized Torch base-model weight filenames (model / pytorch_model, including sharded) by name. is_embedding_model retained an online-confirmed positive offline even when no files were cached, so a metadata-only /check-embedding result let an uncached repo be saved and then fail at first indexing. Retain the positive only when the active revision is materialized locally, which still covers a downloaded tag-only embedder whose snapshot carries no modules.json. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: require a complete weight set offline and persist embedder verdicts across restarts Two follow-ups to the offline embedding-model classifier: - _snapshot_is_loadable_st_model now requires a COMPLETE Torch base-model weight set in one snapshot directory, not just any single recognized weight file. A partially downloaded sharded model (model-00001-of-00002 without its sibling) no longer passes offline validation and then fails at first indexing under local_files_only. Weight files are grouped by directory and a directory counts only when it holds a single model.safetensors / pytorch_model.bin or a full shard set whose indices cover 1..total. - Online-confirmed embedder verdicts are now recorded under the resolved Studio home (embedding_verdicts.json). The session memo is lost on exit, so a downloaded tag-only feature-extraction embedder (snapshot present but no modules.json) was misclassified as non-embedding the first offline call after a restart. The offline branch consults this durable allowlist in addition to the memo, still gated on the active revision being materialized on disk, so an uncached repo is never trusted. Writes are best-effort and only positive verdicts are stored. * studio: require complete weights (with shard index) and resolve default casing offline Follow-ups to the offline embedding-model classifier from the latest review: - Trust a recorded embedder verdict (session memo or persisted allowlist) offline only when the active snapshot carries a COMPLETE, loadable weight set, not merely that it is materialized. A partial download (config present, weights missing or an incomplete shard set) makes _embedding_marker_in_hf_cache read False rather than None, so the previous marker-is-not-None gate wrongly returned True and the local_files_only load then failed. Split out _snapshot_has_complete_weights (config plus complete weights, modules.json aside) and _active_snapshot_dir, and gate the known-embedder positive on the weight set. - Require a sharded checkpoint's index map (model.safetensors.index.json / pytorch_model.bin.index.json) in addition to every shard before accepting it: transformers discovers and wires shards through that index, so a complete shard set without it fails the local-only load. - Resolve the embedding model name to its exact cache casing in the RAG loader before constructing SentenceTransformer. The settings route persists that spelling for a custom override but deliberately leaves the configured default verbatim, so a default whose casing differs from the cache dir would miss it and fail offline. Resolving at load time covers the default too; a no-op for a local path or when nothing case-matching is cached, and idempotent for an already-normalized override. Adds regression tests for the partial-snapshot verdict, the missing shard index, and the loader casing resolution; updates the offline-invariant source assertion to the resolved-name variable. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: require a tokenizer, case-fold verdict ids, and serialize verdict writes Three follow-ups to the offline embedding-model classifier from the latest review: - _snapshot_has_complete_weights now also requires a tokenizer asset. A SentenceTransformer Transformer module builds an AutoTokenizer, so a snapshot with a complete weight set but no tokenizer.json / tokenizer_config.json / vocab still fails the local_files_only load. The check is a permissive union over the common fast-tokenizer, config, and WordPiece/BPE/SentencePiece assets, so an unusual but valid layout is not rejected -- only a genuinely tokenizer-less partial download. - The persisted embedder allowlist is now keyed case-insensitively. model_info() is queried under the requested casing while the settings route saves the cache-resolved casing, so an exact-string lookup missed the persisted positive after a restart (baai/model recorded, BAAI/model looked up) and a loadable tag-only embedder was rejected. Both persist and lookup case-fold the id. - _persist_embedder serializes its read-modify-write under a lock and writes through a per-thread temp file, so concurrent confirmations of different embedders no longer drop each other's entry or collide on the temp path. Cross-process writers stay best-effort (os.replace is atomic; a dropped verdict is only an optimization miss a later online re-confirmation heals). Adds regression tests for the missing-tokenizer reject, alternate tokenizer assets, cross-casing verdict match, and concurrent verdict writes; updates the snapshot test helpers to materialize a tokenizer alongside config and weights. * studio: tighten comments in the offline embedding-model classifier Comment-only pass over the PR's changed files. Collapse the long block comments and docstrings around is_embedding_model, the cache-snapshot and weight-completeness helpers, the embedder-verdict persistence, the offline security gate, and the offline/casing tests to short one- or two-line forms. Preserve the rationale (issue #6817, the local_files_only invariant, the casing and weight-gate reasons) in far fewer words. No code changes. * studio: drop redundant comments in the offline embedding-model classifier Second comment-reduction pass over the offline embedding-model cache work: delete comments and trailing notes that restate the adjacent code or an assertion, and trim the remaining docstrings and rationale comments to their load-bearing invariants. Comments and docstrings only; no code changes. * studio: pin embedder verdicts to a revision, canonicalize default aliases - A persisted verdict recorded that the Hub tagged ONE revision an embedder, but was stored per repo. Once refs/main advanced to a complete but non-embedding Transformer snapshot, the offline path still returned True: the settings route accepted the updated model without force and RAG could silently load it as an embedder. Verdicts now carry the commit they were confirmed at and are trusted only while the active revision matches. One confirmed before the repo was cached has no revision to compare, so the first revision observed afterwards is pinned then -- which is what lets a later advance be caught. The persisted file gains a {id: commit} form and still reads the previous list format. - tokenizer_config.json no longer counts as a tokenizer asset. It only DESCRIBES a tokenizer, so a snapshot with config, weights and just that file passed validation and then failed AutoTokenizer.from_pretrained(local_files_only=True) at first indexing for common BERT/GPT-style models. - A casing-only alias of the default is canonicalized to the default up front. Repo ids are case-insensitive but every gate here compares exact strings, so saving "Unsloth/bge-m3" against a default of "unsloth/bge-m3" ran the verification and scan for a custom model and then persisted an override -- after which later changes to the configured default stopped applying. - verify_import_hoist.py replays __all__ assignments in order instead of unioning them. Only the final value exports anything, so a later plain "=" that drops a name must leave its import counted as unused; "+=" still extends, and an unreadable rebind keeps the earlier names rather than flagging real re-exports. * studio: validate the real ST load root, and pin verdicts to the Hub revision Four ways the offline probe still disagreed with what the loader does: - Verdicts were pinned to the LOCAL refs/main, but model_info() describes the current HUB revision. With a stale cache the two differ, so an older snapshot nobody verified was allowlisted. The pin is now info.sha, taken from the ModelInfo that produced the positive. A verdict carrying no revision (a legacy entry) is no longer trusted at all -- trusting it meant pinning whatever happened to be cached, which is the same bug; the next online check re-records it properly. - config, tokenizer and weights had to exist somewhere in the snapshot, not together. modules.json can send SentenceTransformer at 0_Transformer/, which is loaded FROM that directory, so a cache with the config at the root and only 0_Transformer/model.safetensors passed and then failed the local-only load. Each directory is now checked as a complete load root, which covers both the plain HF layout and the ST module layout. - vocab.json and merges.txt counted independently, but BPE needs the pair unless a serialized tokenizer.json is present, so half a pair validated and then failed AutoTokenizer.from_pretrained(local_files_only=True). - A slashless short name like all-MiniLM-L6-v2 is a supported ST alias that the loader resolves through the sentence-transformers/ organization, so its snapshot is cached under that full id. Probing only the bare name reported a miss and 409'd a model that was cached and loadable; the bare id is still tried first, matching the loader's own order. * studio: fail closed for an offline security scan instead of failing open A local_only (offline) load cannot fetch Hugging Face's malware scan, and the previous behaviour skipped the scan and failed OPEN, so a cached repo with a poisoned pickle weight could deserialize under SentenceTransformer(local_files_only=True). Evaluate it fail-CLOSED against the cached files instead: block a base-model pickle weight the load would deserialize (pytorch_model.bin and its shards, in a directory with no safetensors alternative) and allow a pickle-free (safetensors / gguf are inert) cache. A cached pickle model must be reloaded online once to be scanned, or shipped as safetensors. Nothing cached is not a security event. _fetch_security_status no longer needs the local_only_load skip (the offline branch is handled in evaluate_file_security). Adds a regression test covering the safetensors-allow and pickle-block paths with no Hub call. * studio: only suppress an offline pickle when a loadable safetensors weight exists The offline security gate treated any .safetensors in a directory as covering a pickle weight, so a cache with pytorch_model.bin beside a bare adapter_model.safetensors (or an orphan shard with no index) passed the fail-closed check even though from_pretrained still selects and deserializes the pickle. Require a genuinely loadable safetensors weight -- an unsharded base file or a complete indexed shard set -- before treating the pickle as covered. Also make the import-hoist analyzer preserve uncertainty when __all__ is extended by a value it cannot read statically (__all__ += dynamic()), matching how it already handles an unreadable rebind, so a dynamically-supplied re-export is not flagged HOISTED-IMPORT-UNUSED. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: scope the offline pickle scan to load paths; reset __all__ opacity on rebind Address three review follow-ups on the offline security gate and the import-hoist analyzer: - The offline pickle scan walked the whole snapshot, so a stray pickle in a non-load subdirectory (archive/, nemo/) that SentenceTransformer never deserializes was blocked. Scope it to real from_pretrained load roots -- the snapshot root, or a subdir that holds its own config.json -- matching the online scan's load-path scoping. - _collect_dunder_all kept a sticky opaque flag: a readable replacing assignment after an unreadable extend (__all__ += dynamic(); __all__ = []) still credited every import, so a genuinely unused hoist went unreported. A replacing assignment now resets opacity. - A bare __all__: list[str] annotation has no runtime value; it was treated as an unreadable assignment and marked the export set opaque. Skip annotation-only declarations. * studio: recase slashless ST aliases and accept a pinned embedder after a transient failure Two offline-detection gaps on well-formed input: - resolve_st_cached_repo_id_case bailed on every slashless name, so a differently-cased short alias (all-minilm-l6-v2) validated case-insensitively but was loaded verbatim; the SentenceTransformer loader rewrites it to sentence-transformers/all-minilm-l6-v2 and looks it up case-sensitively, missing the canonical sentence-transformers/all-MiniLM-L6-v2 cache dir. Resolve through _st_cache_repo_dir, which follows the same org alias, and hand back the on-disk casing. - On a transient (non-permanent) Hub failure, is_embedding_model only accepted a cached modules.json marker, so a downloaded tag-only embedder (no modules.json) with a verdict pinned to the active revision was rejected even though the offline branch accepts the identical cache. Mirror the offline branch's pinned-verdict acceptance. * studio: scan modules.json-declared module roots in the offline pickle gate The offline pickle scan treated only the snapshot root and config.json-bearing subdirs as load roots, so a pickle in a non-Transformer SentenceTransformer module directory that has no config.json (e.g. a 0_WordEmbeddings/ module: wordembedding_config.json + pytorch_model.bin) was skipped even though the loader deserializes it. Parse modules.json (and thread through load_subdirs) to treat every declared module directory as a load root, so such a pickle is scanned and fail-closed offline. * studio: classify cached non-Transformer SentenceTransformer models offline _snapshot_has_complete_weights recognized only a Transformer-shaped load root (config + tokenizer + weights co-located), so a fully-cached model built from a non-Transformer module (0_WordEmbeddings uses wordembedding_config.json + embedding weights and its own tokenizer, no HF config.json; BoW keeps its vocab in config.json) was classified non-embedding offline and the settings endpoint returned 409. Add _snapshot_modules_all_loadable, which parses modules.json and accepts a snapshot when every declared module's path directory carries the files that module class's own load() reads (a Transformer/root module still needs the full HF load root; a WordEmbeddings module needs its config plus a complete weight set; other modules need their *_config.json), and at least one embedding-producing module is present. It is OR-ed after the Transformer check, so it only ever accepts more and cannot regress the existing path or reject a pruned cache. * studio: scan PEFT adapter pickle weights in the offline security gate from_pretrained auto-detects an adapter_config.json in the load root and deserializes the adapter weights on top of the base model, so adapter_model.bin is a separate pickle RCE vector that a safetensors base weight does not cover. The offline scan matched only base-model pickle names, so an offline local-only load with safetensors base weights plus a cached adapter_model.bin was allowed despite the live adapter pickle. Scan adapter pickles too, scoped to a load root where adapter_config.json is present and no adapter_model.safetensors exists. * studio: require weights for Dense/CNN/LSTM SentenceTransformer modules offline _module_dir_is_loadable accepted a Dense, CNN, or LSTM module dir with only its config, but those modules' load() hard-load model.safetensors else pytorch_model.bin (verified against sentence-transformers source: no fallback, raises if neither exists) -- exactly like WordEmbeddings. A cache with such a module's config but no weights would validate and then fail the local_files_only load. Require a complete weight set for every weighted module, not just WordEmbeddings. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: scan root-index subdir pickle shards offline; handle __all__.append/.extend - The offline pickle scan followed only load-root directories, so a shard mapped by a root pytorch_model.bin.index.json into a non-root subdirectory was skipped even though from_pretrained follows the index weight_map and deserializes it (a layout an attacker can craft to evade the scanner). Read the local index and scan its referenced pickle shards, covered by a loadable base safetensors at the index root -- mirroring the online scan. - The import-hoist analyzer ignored __all__.append("X") / __all__.extend([...]) runtime re-export mutators, so an import added solely for one tripped HOISTED-IMPORT-UNUSED. Read their string args like +=, and treat any other __all__ method call as opaque. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: classify StaticEmbedding offline, require WordEmbeddings tokenizer, bound model_info - A StaticEmbedding module (e.g. sentence-transformers/static-retrieval-mrl-en-v1's 0_StaticEmbedding/) holds tokenizer.json + weights and NO config, so the config-gated non-Transformer path 409'd it offline. Recognize it by what StaticEmbedding.load() reads: a tokenizer.json plus a complete Torch weight set. - WordEmbeddings.load() rebuilds its tokenizer via the configured tokenizer_class.load() from the module dir, so a WordEmbeddings module now also requires a tokenizer artifact (whitespacetokenizer_config.json / phrasetokenizer_config.json, or a shared HF tokenizer asset), not just its config + weights. - With neither offline env var set, an unbounded model_info() could hang on connect/DNS retries for networkless users (the #6817 symptom). Bound it with a 15s timeout so a dead network fails fast and the existing transient-failure cache fallback resolves a cached model, while a reachable Hub still wins. (Documented caveat: a stalled DNS getaddrinfo may exceed this.) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Resolve indexed safetensors shards relative to their index _safetensors_index_complete compared shard basenames against the flat set of files in the index directory, so an index whose weight_map names shards in a subdirectory was treated as incomplete whenever a legacy pytorch_model.bin sat beside it. That falsely blocked a snapshot whose pickle weights are fully covered by a complete, loadable safetensors shard set. Resolve each shard path relative to the index directory instead, and add a regression test for the subdir-mapped shard case. * Restrict offline weight-completeness check to declared load roots _snapshot_has_complete_weights scanned every directory in a snapshot and accepted it when ANY directory was a complete Transformer load root. When modules.json is present a SentenceTransformer load only opens the declared module paths, so a snapshot whose declared modules are incomplete but which happens to contain an unrelated complete directory was accepted offline and then failed at the first local_files_only load. Restrict the candidate directories to the roots a load actually opens: the snapshot root plus each modules.json module path. For a well-formed snapshot the verdict is unchanged; only a complete directory at an undeclared path no longer vouches for an otherwise-incomplete snapshot. * Scan SentenceTransformer Router child module weights offline A Router (legacy Asym) snapshot declares its child sub-modules only in router_config.json, not the top-level modules.json, and Router.load() deserializes each child's weights from its own subdir. A config.json-less child such as query_0_WordEmbeddings (wordembedding_config.json plus a pickle pytorch_model.bin loaded via torch.load) was therefore neither a modules.json-declared load root nor a config.json-bearing dir, so the offline gate skipped its pickle even though the loader deserializes it. Parse router_config.json at each load root and treat every declared child subdir as a load root (bounded BFS, so nested routers are covered), so those child pickles are scanned. Add Router regression tests: a pickle child blocks, a safetensors child is allowed, and a Router in a declared subfolder is followed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Do not treat an unreferenced config subdir as an offline load root The offline pickle gate skipped a directory only when it was neither a declared load root nor held a config.json. Because _st_load_roots already resolves every real load root (snapshot root, modules.json / load_subdirs dirs, Router children), the config.json fallback only ever promoted an UNREFERENCED subdir -- a nested checkpoint-500/ or archive/ that ships its own config.json + pytorch_model.bin -- to a load root. from_pretrained never descends into such a subdir and the online scan ignores the same unindexed pickle, so offline mode wrongly blocked a model the loader reads from a clean safetensors root. Scope the pickle to directory in roots only, and add a regression test (a stray checkpoint-500/ no longer blocks; a modules.json-declared module dir still does). * Classify a root Router (Asym) model as loadable offline _module_dir_is_loadable applied Transformer root requirements (config + tokenizer + weights) to every root module, so a Router saved at the snapshot root -- which carries only modules.json + router_config.json and loads its weights from child subdirs -- was classified not loadable offline, and is_embedding_model missed a cached Router embedder. Dispatch on the module class before the root Transformer fallback: a Router/Asym dir is loadable when router_config.json parses and every declared child subdir is loadable (validated recursively through _module_dir_is_loadable, so nested routers and every child type are covered) with at least one embedding-producing child. This also tightens a non-root Router, which previously validated on the mere presence of router_config.json without checking its children. Add Router regression tests (root and declared subfolder, complete and incomplete-child). * Require every declared module before accepting an offline cache _snapshot_is_loadable_st_model returned has_complete_weights OR modules_all_loadable, so a complete 0_Transformer short-circuited the or and vouched for the whole snapshot even when a declared sibling module was missing its serialized weights; SentenceTransformer builds every module in modules.json, so that snapshot passed offline validation and then failed the local-only load. When modules.json declares a non-empty list it is now authoritative (modules_all_loadable validates every declared module); has_complete_weights stays the fallback only for an empty/non-list modules.json (the plain from_pretrained root). Also add the weight-bearing modules whose load() hard-loads via load_torch_weights and previously fell to the config-only path -- LayerNorm, WeightedLayerPooling, SparseAutoEncoder -- to _ST_WEIGHTED_MODULE_NAMES, with source citations and the deliberate exclusions (Pooling/Normalize/BoW/WordWeights read no weights on load). Add parametrized regression tests over LayerNorm/WeightedLayerPooling/Dense (a weightless sibling rejects, a complete sibling accepts). * Reject self-referential Router children instead of recursing forever _router_dir_is_loadable validates each router_config.json child through _module_dir_is_loadable, which re-enters _router_dir_is_loadable for a Router child. A malformed types entry naming the router's own directory (a key of ".", which normalizes to the same dir) made that recursion never descend, so it looped until RecursionError -- breaking the documented never-raises contract and turning a crafted/corrupted cached model into a 500 from is_embedding_model instead of a graceful unverifiable result. A real child reference is a subdir and always resolves deeper, so reject any child whose resolved path is the router dir itself. Add a regression test (a router_config naming "." as a Router child returns False without raising). * Treat a destructuring __all__ assignment as opaque _collect_dunder_all detected __all__ only as a direct ast.Name assignment target, so a binding through a destructuring target (__all__, meta = [...], v -> an ast.Tuple) was skipped entirely, leaving an empty, non-opaque export set. A newly hoisted import re-exported only through that assignment was then falsely flagged HOISTED-IMPORT-UNUSED. Its value cannot be mapped statically, so mark the export set opaque when __all__ is reached only through a destructuring / item / attr target, matching how the collector already handles other unreadable __all__ forms. Add a self-test case. * Canonicalize declared module paths before scoping the offline pickle gate A repo could declare a traversing module path such as 0/../evil in modules.json (or a router_config child), which SentenceTransformer resolves to evil/ and deserializes evil/pytorch_model.bin. _st_load_roots recorded the raw snap/"0/../evil", which never equals the snap/evil that rglob yields, so the offline pickle gate skipped that directory and a malicious repo slipped a pickle past the newly added gate. Add _canonical_load_dir to collapse ./ and ../ components lexically and reject an upward escape, and route the modules.json paths, load_subdirs and router children through it so the gate scopes the same normalized directory the loader opens. Add regression tests for a traversing modules.json path and router child. * Close offline embedding-classification completeness gaps Five real offline misclassifications, each a false negative (the #6817 hang recurs) or false positive (accepted then 409s at the local_files_only load). Dispatch _module_dir_is_loadable on the module class before the root Transformer fallback. A module with save_in_root=True (every InputModule: WordEmbeddings, StaticEmbedding, SparseStaticEmbedding, Transformer, Router) is saved at the snapshot root, so a root WordEmbeddings was wrongly held to Transformer requirements (an HF tokenizer it never writes) and classified not loadable. CLIPModel is Transformer-shaped: CLIPModel.load() reads AutoModel weights plus AutoProcessor, so a config-only CLIP dir must not validate. SparseStaticEmbedding needs a tokenizer plus either idf.json or a complete torch weight set (conditionally weight-bearing); a config alone is not enough. A present but empty or malformed modules.json is not loadable and does not fall back to a root Transformer: with modules.json present the loader never takes the plain-Transformer path (base/model.py _load_config_modules). The tag-only no-modules.json embedder is classified separately via _snapshot_has_complete_weights. Validate a sharded weight index against its weight_map (every mapped shard present, resolved relative to the index dir) instead of trusting the index file's mere existence, mirroring the security-side check. Add regression tests for all five. * Close case-folding and online-traversal holes in the offline pickle gate Two gate bypasses where the security scan credited or scoped a path differently from what the loader actually resolves: The safetensors credit was case-folded. _cached_pickle_weight_files lowercases every filename, and the loadable-safetensors and adapter checks tested those folded keys against the exact-lowercase names. On a case-sensitive filesystem (Linux, the Studio default) a crafted repo shipping Model.SafeTensors plus a malicious pytorch_model.bin makes transformers and sentence-transformers miss the exact-name model.safetensors and deserialize the pickle, while the gate credited an inert safetensors and did not block. Credit safetensors case-sensitively against real filenames, and drop pytorch_model.safetensors from the credit set (transformers loads only model.safetensors, never that name). Pickle matching stays case-insensitive (over-blocking a mis-cased pickle the loader would not load is the safe direction). The online scan did not canonicalize traversing paths while the offline gate did. A repo-controlled modules.json path (threaded into the online scan via the RAG guard) or a weight_map shard entry like 0/../evil / ../evil was compared verbatim, so a flagged evil/pytorch_model.bin never matched and evaded the online scan though the loader resolves and deserializes it. Canonicalize the repo-controlled load-subdir prefixes and weight_map shards the same way the offline gate does, so offline and online agree. Add regression tests for both bypasses. * Treat a conditional __all__ mutation as opaque in the import-hoist linter _collect_dunder_all replayed only top-level module statements, so an __all__ assignment or mutation inside a module-level if / try / for / while / with / match (or a deeper scope) was ignored, leaving the export set understated. A newly hoisted import re-exported only through such a conditional __all__ was then falsely flagged HOISTED-IMPORT-UNUSED, blocking a valid change. A conditional value cannot be replayed statically, so mark the export set opaque when __all__ is bound or mutated anywhere other than a top-level statement. Add a self-test case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope Router child sub-modules as load roots in the online embedding scan The RAG embedding security guard unions the SentenceTransformer module dirs from modules.json into the load roots it scopes for the Hub scan, so a flagged pickle directly under a Transformer module blocks. A Router (legacy Asym) module declares its child sub-modules only in router_config.json, not in modules.json, and Router.load() deserializes each child from its own subdir. The online scan therefore dropped a flagged child pickle (for example query_0_WordEmbeddings/pytorch_model.bin) as an unreferenced nested shard while the loader still deserialized it, the counterpart to the offline gate which already expands router children via _router_child_dirs. _st_module_subdirs now reads router_config.json for any Router-typed module and adds each declared child (joined onto the module path, canonicalized so a traversing entry is dropped) to the load roots. The config is read only for a Router-typed module, so a plain embedder pays no extra fetch, and every failure path still returns () so the guard never bricks the embedder. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Allow a recorded-clean pickle embedder to load offline The offline embedding security gate is fail-closed: with no network to reach Hugging Face's scan, a cached pickle weight cannot be verified, so it is blocked and a model the user already downloaded and used online will not load offline. This adds a persistent cache of clean Hub verdicts so that exact content can load offline, without weakening the gate for an unknown or never-scanned pickle. When an embedding repo is loaded online and HF's scan returns a completed clean verdict, the load roots are hashed and recorded under the scanned commit as an exact map of snapshot-relative pickle name to sha256, in a per-user JSON store at studio_root()/security/embedding_scan_verdicts.json (atomic write, 0600, thread and cross-process locked, 30-day TTL). Offline, a cached pickle model loads only when the active cached commit and every load-root pickle's sha256 match the recorded verdict; a missing record, moved commit, changed or added pickle, expired record, or any error keeps blocking. Online loads always re-query the Hub and an authoritative unsafe verdict deletes any stale record, so a now-flagged commit cannot keep loading on an old clean record. The store binds repo id, full commit, and a per-file sha256 map so a locally swapped pickle at the same commit, a branch advance, or an added load-relevant pickle is detected. A same-user attacker who can rewrite the model cache or the store is outside the enforceable boundary and this is documented; the sha256 is computed just before load, so a narrow verify-to-load window remains, and a Hub scanner false negative is recorded faithfully (safetensors stays the stronger defense). Recording is triggered post-load in the RAG embedder because the settings route only validates and the pre-load guard runs before the constructor downloads; recording is skipped when the loaded commit differs from the scanned commit. The blocked-pickle enumerator now returns snapshot-relative Paths so two module dirs that ship the same pickle basename are hashed and reported distinctly. * Harden the embedding verdict cache against review findings Tighten the offline verdict cache and its enumeration so every uncertain or malformed input fails closed and the recorded hashes always match the files the loader reads: - Hash every case-colliding pickle in a load root, not one representative. On a case-sensitive filesystem pytorch_model.bin and PYTORCH_MODEL.BIN are distinct files; keying by lowered name dropped one and could hash a decoy instead of the loader's target. The enumerator now returns every variant Path. - Only persist a clean verdict for a COMPLETED, entirely-benign scan. Require scansDone to be the boolean True (not a truthy string), filesWithIssues to be a well-formed list, and every flagged file to be a definitively-safe level; a pending, error, unknown, or malformed entry no longer records as clean. The online block decision is unchanged. - Fail closed when the offline cache cannot be inspected: an rglob error now propagates and blocks instead of reading as pickle-free, and a snapshot that errors on resolution (vs a clean not-cached) blocks. The offline guard also raises instead of returning when its own inspection throws, so the constructor never deserializes an unverified cached pickle. - Expand online Router children recursively (bounded BFS with a seen set), mirroring the offline load-root expansion, so a flagged grandchild pickle is scoped online and cannot be recorded clean. - Reject absolute and drive/UNC declared paths in the load-root canonicalizers; the loader would resolve them outside the snapshot, so collapsing them to an in-snapshot relative dir scoped the wrong place. - Pin verdict recording to the scanned commit's snapshot and take the offline verify commit from the snapshot directory name, removing a second refs/main read and the skew it allowed. - Drop the now-unused pickle-name wrapper. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten offline embedding classification and the pickle gate Close a set of offline edge cases where validation accepted a cache the local_files_only load then rejects, and one gate bypass: - Credit a sharded model.safetensors.index.json for a pickle sibling only at a from_pretrained root. A non-Transformer SentenceTransformer module (Dense, WordEmbeddings, StaticEmbedding) loads via Module.load_torch_weights, which reads model.safetensors then pytorch_model.bin and never the index, so a sharded safetensors index in such a module dir must not vouch for its pytorch_model.bin. - Stop counting pytorch_model.safetensors as loadable in the offline classifier: the loader probes model.safetensors (then its index) or pytorch_model.bin, never pytorch_model.safetensors, matching the gate that already treats it as a decoy. - Treat a present but unreadable weight index as incomplete: transformers opens and parses any present index, so a malformed one or one without a weight_map fails the load rather than falling back to filename-numbered shards. - Require the CLIP image-processor config (preprocessor_config.json) for a CLIP module: CLIPModel.load builds a CLIPProcessor that needs it, so a tokenizer alone is not enough. - Require a SparseStaticEmbedding config to actually select idf.json (a path ending .json) or ship loadable weights; a bare idf.json the config does not name falls through to load_torch_weights and raises. - Do not use the tag-only recorded-verdict fallback when modules.json is present: with the file present the loader takes the modules.json path, so a present but empty or malformed manifest must not be validated as a plain root Transformer. - Import-hoist linter: only a module-level conditional mutation or a function that declares global __all__ makes the export set opaque; a __all__ bound as a local in a nested function or class no longer masks a genuinely unused hoisted import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope Router-child pickles to their deepest load root and gate the ST offline kwarg The online scan stripped the first matching load-subdir prefix from a flagged file, so a nested Router child pickle (0_Router/query_0_WordEmbeddings/pytorch_model.bin) matched the parent 0_Router root, looked like an unreferenced nested shard, and slipped the gate even though Router.load() deserializes that child directly. Match the deepest (longest) load subdir instead, so the child becomes root-level under its own load root and blocks. pyproject sets no lower bound on sentence-transformers and the local_files_only constructor arg is absent on older releases, so always forwarding it broke every embedder warm on those installs. Pass it only for an offline load; an online warm never forwards it and works as before, while the offline capability still requires a version that supports it. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reject snapshot-escaping shard paths and credit Transformer submodule safetensors The offline pickle enumerator joined a weight-index weight_map value straight to the load root and followed it, so a repo-controlled index mapping "../.." into a sibling snapshot made an offline from_pretrained deserialize an out-of-snapshot pickle, and an online load would then hash and record that external file as the scanned commit's clean content. Reject any shard path that escapes the snapshot root and fail closed, mirroring the canonical-root check the online shard scan already applies. A complete model.safetensors.index.json was credited over a sibling pickle only at the snapshot root, but a Transformer module subdirectory (0_Transformer/) is loaded via AutoModel.from_pretrained, which honors that shard set and never reads the pickle. Credit the sharded index for Transformer-typed modules declared in modules.json so a cached model that ships both a sharded safetensors checkpoint and an unused PyTorch checkpoint is no longer falsely blocked offline. Non-Transformer modules (Dense, WordEmbeddings, StaticEmbedding) read a flat weight with no index and keep their pickle blocked. Limit the import-hoist verifier's global __all__ scan to the declaring function's own scope so a nested inner-scope local __all__ no longer marks the module export set opaque and mask an unused hoisted import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope Router children against the snapshot and mirror the ST alias rewrite Router.load resolves each child at Path(subfolder, model_id) relative to the Router dir, so a nested 1_Router with a "../evil" child points at evil/ inside the snapshot and the loader deserializes evil/pytorch_model.bin. The offline enumerator canonicalized the child against the Router dir alone and dropped anything with "..", so that pickle was never scanned and the gate reported the cache pickle-free. Canonicalize router children against the snapshot, retaining in-snapshot siblings as load roots and failing closed on a child that escapes the snapshot itself, matching the online scan which already joins the prefix before normalizing. The security gate resolved a slashless model id by probing the bare cache dir first, but the SentenceTransformer constructor rewrites a non-basic slashless name to sentence-transformers/ <name> and loads THAT snapshot (only the basic ORIGINAL_TRANSFORMER_MODELS load bare). With both models--<name> and models--sentence-transformers--<name> cached, the gate inspected the bare dir while the loader read the namespaced one, so a pickle there bypassed the local-only gate. Mirror the constructor: try the namespaced candidate first for non-basic slashless names. Add the same not (snapshot / modules.json).is_file() guard to the transient-Hub-failure tag-only fallback that the offline branch already carries, so a cache whose present manifest is empty or malformed is no longer reported as a loadable embedder. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten root shard credit, module-path escapes, and weight-set probe order Credit the sharded safetensors index at the snapshot ROOT only when the root is actually loaded through an AutoModel/from_pretrained path. A modules.json root module of a non-Transformer type (StaticEmbedding / WordEmbeddings / Dense) loads via load_torch_weights, which reads pytorch_model.bin and ignores the index, so crediting a root shard index there suppressed a live root pickle and let the offline gate report the cache pickle-free. Recognize the Transformer subclasses CLIPModel and MLMTransformer as index-honoring load roots (they load via from_pretrained), so a sharded-safetensors CLIP/MLM submodule with a legacy pytorch_model.bin sibling is no longer falsely blocked offline. Mirrors the classifier dispatch. Fail closed on an absolute or snapshot-escaping modules.json module path (or load_subdirs entry) instead of silently dropping it: SentenceTransformer resolves such a path outside the snapshot and would deserialize an external pytorch_model.bin the gate cannot scan. On the classifier side, walk the weight set in the exact from_pretrained probe order (model.safetensors, its index, pytorch_model.bin, its index) so a pickle behind a malformed safetensors index is no longer accepted as complete, and restrict shard names to the loader-probed stem/ext pairs so a decoy model-*.bin / pytorch_model-*.safetensors set is not treated as loadable. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore scripts/verify_import_hoist.py to main The offline embedding cache fix does not depend on the __all__ scope handling that had accumulated in this linter, so revert the file to its main version and keep the PR focused on the feature. The feature modules still pass the existing import hoist check unchanged. * Reuse a shared HF cache skeleton in the offline classification tests Extract _mk_repo and _activate helpers for the repeated snapshot cache setup that every per-type builder duplicated, and fold the two StaticEmbedding missing-asset cases into one parametrized test. Same 125 collected items, all still passing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reclassify embedding models from the cache on every offline call is_embedding_model consulted its process memo before the offline branch, so an online lookup that memoized True from tags (without caching any weights) was returned unchanged once the session went offline -- the studio flips HF_HUB_OFFLINE in-process on a dead DNS, and the ungated check-embedding route can populate the memo. Settings would then accept a repo the offline loader cannot open. Run the offline cache-marker reclassification ahead of the memo and never record it, so an offline verdict always reflects the local cache and a later cache materialization is not masked by a stale negative. Add regression tests. * Tighten comments on the offline embedding path Condense the offline-embedding helper docstrings and inline comments added in this PR to fewer, clearer lines, keeping the non-obvious security and offline rationale. Comments and docstrings only; no code change. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <unslothai@gmail.com>
…nd Pi (unslothai#7326) Bring the local-subagent support onto main. The original change (unslothai#7316) merged into the stacked pr/daniel-unsloth-start-audit branch rather than main, and unslothai#7313 reached main via squash, so these files never landed on main. Adds --as-subagent for claude, codex, opencode and pi: the parent agent keeps its own cloud model while a locally served GGUF is registered as a delegated subagent, using ephemeral per-session config that never touches the user's real agent config.
* fix(install): route Strix to AMD gfx index on ROCm 7.14 When ROCm 7.3+ caps to the generic pytorch.org rocm7.2 index (or the Radeon repo is unavailable), gfx1150/gfx1151 hosts were left on torch 2.11+rocm7.2 instead of AMD's arch-specific wheels. Broaden the Strix reroute in install.sh and studio/install_python_stack.py so `studio update` repairs the same path as fresh installs (unslothai#7280). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
…unslothai#5988) * Faster safetensors weight loading on unified-memory (integrated) GPUs On unified-memory GPUs (AMD APUs / "Strix Halo", NVIDIA GB10 "Spark", Intel iGPUs) the GPU shares the system memory pool. PyTorch's fast pinned-DMA host->device path does not recognize the Rust-allocated, mmap-backed buffers that safetensors hands back, so a direct safetensors GPU load (`safe_open(..., device=<cuda>)`) drops onto a slow per-tensor copy that, on unified memory, additionally triggers page-attribute changes and page faults. Wrap `transformers.modeling_utils.safe_open` so that, when transformers asks it to load a shard directly onto a CUDA/HIP device, the shard is opened on CPU and each tensor is `.clone()`-d into a normal torch allocation before `.to(device)`. This restores the fast DMA path. Data, dtype and final device are unchanged, so outputs are bit-identical -- only *how* the bytes reach the GPU changes. Strictly gated to integrated/unified-memory GPUs via the standard `is_integrated` device property (every visible device must be integrated): a hard no-op on discrete NVIDIA/AMD GPUs, CPU, XPU and MLX, where the pinned-DMA path already works. Only intercepts `framework="pt"` CUDA-device targets; CPU / disk-offload loads are left untouched. Accuracy-neutral, idempotent, opt out with UNSLOTH_DISABLE_UMA_CLONE_LOAD=1 (force the gate for tests with UNSLOTH_FORCE_UMA=1/0). This is the AMD/universal-UMA counterpart to the NVIDIA DGX Spark work in unslothai#5945 (which deliberately left the H2D clone-then-move out): gating on `is_integrated` covers AMD Strix Halo, Intel iGPUs and Spark-class parts alike. Verified on an AMD Radeon 8060S (gfx1151, Strix Halo) Windows ROCm box with in-process, ordering-cancelled A/B benchmarks: - H2D mechanism (safe_open device=0 vs cpu->clone->.to(0)): 2.08x faster (1.076s -> 0.518s for a 988MB bf16 shard) - full `from_pretrained`: 1.56x faster (1.552s -> 0.996s), saving 0.555s -- matching the H2D delta exactly - max|logit diff| stock vs patched == 0.0 (bit-identical), generate + a LoRA train step both verified The absolute/relative win grows with bf16/fp16 weight volume (the same trick is reported as ~2.3-2.75x on NVIDIA GB10 Spark for larger models). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: evaluate the integrated-GPU gate lazily, not at import (Gemini review) patch_unified_memory_safetensors_load() called is_integrated_unified_memory_gpu() at install time, and the gate queries torch.cuda.get_device_properties() for every visible device -- initializing the CUDA context during `import unsloth` on every CUDA machine (discrete included). That (a) breaks fork-based multiprocessing, (b) runs BEFORE patch_dgx_spark_memory_config can set PYTORCH_CUDA_ALLOC_CONF on Spark, defeating that patch's expandable_segments config in the very environment this PR targets, and (c) charges a CUDA context to CPU-only imports. The gate now runs lazily inside the wrapper, ordered AFTER the framework/device check so non-CUDA loads never trigger the property query; a CUDA-target safe_open means the caller is initializing CUDA anyway, and the gate is lru-cached so it is evaluated once. The wrapper installs unconditionally (opt-out and idempotency unchanged) and passes through when the gate is off. Tests: install-time no-eval guarantee (gate raises if called during install), wrapper passthrough with the gate off, all previous gating / passthrough / CUDA correctness tests kept -- 16/16 pass. Verified on the N1X (WSL2): module exec + patch install leave torch.cuda.is_initialized() unchanged; CPU loads pass through; forced CUDA-target loads intercept and land bit-identical on the GPU. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Compress PR comments to essentials (comment-only; AST-verified) Docstrings and the _utils hook comment trimmed to their load-bearing content (lazy-gate rationale, gating scope, opt-out env). AST dumps with normalized docstrings are identical before/after for all three files; the module's 16 unit tests pass unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: tighten the UMA-load import comment (no code change) * Tighten and trim code comments * Drop unused is_integrated_unified_memory_gpu import from _utils.py The UMA hook only needs patch_unified_memory_safetensors_load(); the gate symbol is imported and used from ._uma_safetensors directly, so the hoisted alias here was dead and tripped the import-hoist safety-net lint. * Scope the UMA loader docstring to CUDA/HIP direct-device loads The module text claimed Intel iGPU coverage, but the gate and device check are CUDA/HIP only, and the clone path only wraps safe_open calls that carry a CUDA device. State the actual scope and name the deliberate exclusions (Intel XPU, CPU-open + .to() flows like bnb/HQQ) until they can be validated on real hardware. Comment-only change. * Tighten UMA safetensors loader comments Trim the inline comments in the UMA clone-then-move path and the _utils.py install site to be shorter and clearer. No code changes. * uma: fall back to the direct move when the clone cannot allocate The clone-and-move fast path transiently doubles one tensor's CPU footprint while the mmap source and the CUDA destination are live. On a UMA box with little free shared memory a large tensor could OOM where the stock direct safe_open path would have loaded it. Both move sites now go through a helper that catches the allocation failure and falls back to the direct (slow but allocation-free) move, so the load always succeeds; a genuine non-memory error re-raises identically from the fallback. Added a test that forces the clone to fail and verifies the wrapper still lands tensors on the device with intact values (17 tests pass on a real GPU). * tests: track the moved pass-through inheritance in the gguf order check Main moved the llama_extra_args pass-through inheritance out of the GGUF branch into _resolve_inherited_extra_args, which runs before it, so the source-order assertion's "if request.llama_extra_args is None" anchor no longer exists inside the branch and the check failed after the main merge. The test now asserts the same property in the current shape: inheritance before the GGUF branch (a carried --no-mmproj still shapes the hub guard's companion requirement), and marker, hub guard, unload in order within the branch. Full file passes (32 tests). * tests: anchor the inheritance order check on the call, not the definition source.index("_resolve_inherited_extra_args(") matched the function definition, which always precedes the endpoint, so the ordering assertion was vacuously true. Anchoring on "= _resolve_inherited_ extra_args(" pins the first call site inside the load endpoint (line 4505), which is the statement whose position relative to the GGUF branch the test is meant to guard. 32 tests pass. * tests: align the gguf order test with main Main fixed the stale ordering assertion in PR 7252; adopting its version verbatim removes this file from the branch diff entirely and avoids a conflict on the next main merge. 32 tests pass. * uma: tighten comments * Relicense UMA safetensors module and test under AGPL-3.0 --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…nslothai#7308) * Studio: lighten chat text weight on Linux to match macOS rendering * Exclude custom interface fonts from the Linux chat weight compensation * Simplify Linux chat font weight override
* Studio: put Hub above Projects in the sidebar Swap the two nav rows so Hub sits directly under New Chat, ahead of Projects. Order only, no behavior change. * Studio: rename Hub to Models, lowercase New chat Rename the Hub nav row and its page heading to Models (localized in all locales). Use sentence case 'New chat' in the English label. * Studio: fix dataset title and stale Hub tab hints after rename Show 'Datasets' as the catalog heading in dataset mode, not 'Models'. Update the download-conflict toasts to point at the Models tab.
… llama-server (unslothai#7272) * Studio: mask AMD GPU pins via ROCR so an unsupported iGPU can't crash llama-server On a mixed AMD host (e.g. a discrete gfx1102 GPU next to a gfx1103 iGPU) the bundled rocm-gfx110X llama.cpp build segfaults during HSA device enumeration on the unsupported iGPU -- before llama-server prints a line, so every model load fails with a bare signal and empty logs. The GPU-subset pin masked visibility with HIP_VISIBLE_DEVICES, but HIP filtering runs only after the HSA runtime has already enumerated (and crashed on) every agent. Mask the subset via ROCR_VISIBLE_DEVICES (the ROCr/HSA layer) instead, so a deselected/unsupported GPU is never enumerated. Exactly one layer is masked (HIP cleared) to avoid the double-mask reindex that would otherwise drop the child to CPU. The whole-set tensor-split path and the CPU-only sentinel keep their existing HIP behavior. Also stop misreporting the resulting startup segfault as a vision projector incompatibility: when the text-only mmproj retry also hard- crashes with a signal, surface a GPU/driver init crash (with the ROCR hint) instead of blaming the projector. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tighten _emit_child_gpu_visibility comments for unslothai#7272 Comment and docstring only: condense the ROCR-vs-HIP masking rationale from ~22 to ~14 lines and the call-site note from 5 to 3, keeping every technical point (HSA enumeration segfault, physical ids, the -1 sentinel). Logic is unchanged, verified by an AST compare with docstrings stripped and by exercising _emit_child_gpu_visibility against a torch/HIP stub. * Detect AMD SDK ROCm wheels (hip=None) in _emit_child_gpu_visibility (Codex P2) The ROCm branch gated only on torch.version.hip, but AMD SDK wheels leave that unset while encoding 'rocm' in __version__ (detect_hardware handles this the same way). On such a wheel the masking was skipped entirely, leaving only CUDA_VISIBLE_DEVICES, so on a mixed AMD box the unsupported deselected iGPU still enumerated and could crash llama-server. Now the branch also treats 'rocm' in torch.__version__ as ROCm, mirroring detect_hardware. Adds tests for the hip=None SDK wheel (ROCR + default paths) and a CUDA guard so the version-string check can't false-positive. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remap CUDA_VISIBLE_DEVICES to post-ROCR ordinals on prefer_rocr for PR unslothai#7272 (Codex P1) On the prefer_rocr path _emit_child_gpu_visibility set ROCR_VISIBLE_DEVICES to the physical id and cleared HIP_VISIBLE_DEVICES, but left CUDA_VISIBLE_DEVICES at the physical id. ROCR re-indexes the visible agents from 0 and, with HIP cleared, HIP honours CUDA_VISIBLE_DEVICES -- so a non-zero pick (e.g. GPU 1) pointed out of range, HIP saw 0 devices, and the child fell back to CPU, defeating GPU-picker selections other than physical GPU 0. Remap CUDA to the post-ROCR ordinals (0..N-1); GPU 0 is unchanged, the default (HIP) path and the CPU sentinel are untouched, and non-AMD wheels never enter this branch. * Detect AMD SDK wheels in _resolve_visible_physical_ids for PR unslothai#7272 (Codex P2) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep the HIP mask on Windows ROCm in prefer_rocr for PR unslothai#7272 (Codex P2) * Ignore ROCR_VISIBLE_DEVICES in _resolve_visible_physical_ids on Windows for PR unslothai#7272 (Codex P2) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Preserve inherited ROCR masks in the tensor-split pin for PR unslothai#7272 (Codex P2) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>
…7305) * fix(install): infer Strix gfx when ROCm runtime is absent When /dev/kfd and rocminfo are missing on Linux (e.g. Arch/CachyOS Strix Halo), route to AMD per-arch wheels via cpuinfo/lspci inference instead of CPU-only PyTorch. Mirrors install.ps1 Windows behavior and fixes studio update via install_python_stack.py (unslothai#7301). * Map Radeon 8065S to gfx1151 in the Linux gfx inference (Codex P2) install.sh _infer_amd_gfx_arch_from_gpu_name missed 8065S, so a Strix Halo host that only exposes 'AMD Radeon 8065S' via lspci (no Ryzen AI Max branding in /proc/cpuinfo) was left on CPU torch. setup.sh and setup.ps1 already list 8065S -> gfx1151. Added it, and widened the cpuinfo regexes (install.sh and install_python_stack.py) from Radeon 80[0-9]0S to 80[0-9][05]S to match the 80X5S naming, consistent with the display-side check already in install.sh. Tests cover the 8065S name and the cpuinfo-only case. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate the Linux gfx inference out of WSL without the ROCDXG runtime for PR unslothai#7305 On WSL /proc/cpuinfo and lspci still see the host APU, so a standalone 'unsloth studio update' could infer gfx1151 and install per-arch ROCm wheels into a WSL env whose ROCDXG bridge (librocdxg) was never bootstrapped, i.e. one that cannot expose the GPU. Skip the cpuinfo/lspci inference on WSL unless librocdxg is present; an explicit UNSLOTH_ROCM_GFX_ARCH override still wins. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address Codex review on PR unslothai#7305 (WSL runtime gate, Linux mirror, arch guard) - install.sh _infer_linux_amd_gfx_arch: skip the cpuinfo/lspci inference on WSL unless librocdxg is present (the ROCDXG bridge), mirroring the Python fix, so a WSL box whose ROCm bootstrap was skipped keeps the CPU fallback instead of installing AMD wheels that cannot reach the GPU. The explicit UNSLOTH_ROCM_GFX_ARCH override still returns first, so it stays authoritative. - install.sh: guard the inferred-gfx reroute on x86_64|amd64. ROCm torch wheels are not published for arm64, so an inferred/overridden gfx no longer pushes an arm64 host to the AMD arch index (get_torch_index_url returns CPU there). - install_python_stack.py _amd_arch_index_url: honour UNSLOTH_AMD_ROCM_MIRROR on Linux (the same var install.sh uses) instead of the Windows mirror var, so a mirrored/air-gapped Linux 'unsloth studio update' reaches the index install.sh chose. Windows still delegates unchanged; both default to repo.amd.com. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scan all AMD display controllers in the lspci fallback for PR unslothai#7305 (Codex P2) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): keep inferred AMD wheels from being overwritten After a successful inferred-gfx install, skip the generic pytorch.org ROCm reinstall so readable ROCm userland without /dev/kfd cannot undo the per-arch repair (Codex P1 on unslothai#7305). Also merge latest main. * Only take the inferred-gfx install when the runtime sees no GPU for PR unslothai#7305 (Codex P1) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Isolate three updater tests from the host cpuinfo for PR unslothai#7305 (Strix dev box leak) * Gate the reroute on invisible ROCm and forward the inferred gfx to setup.sh for PR unslothai#7305 (Codex P2s) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Require AMD PCI display evidence for cpuinfo inference; honor gfx override with visible ROCm for PR unslothai#7305 (Codex P2s) --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
…ure Cloudflare links (unslothai#7340) * Studio: clear composer draft on send * Studio: verify the Cloudflare link is reachable before printing it * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: wait for tunnel DNS propagation before verifying the public URL * Studio: bound tunnel DNS wait and health probe by one deadline * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: keep composer draft when overlay send validation fails * Studio: retry transient DoH failures while waiting for tunnel DNS * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…+ clearer ROCm-less warning (unslothai#7314) * installer: fix Linux AMD GPU detection + actionable ROCm-less warning The rocminfo/amd-smi-less fallback in _has_amd_rocm_gpu keyed on a /gpu_id/ line inside each KFD node's properties file, but gpu_id is a separate sibling sysfs file and never appears in properties. The guard never matched, so the fallback missed every AMD host without ROCm tooling (e.g. a fresh CachyOS/Arch box) and reported 'no GPU detected' despite vendor_id 4098 being present in the KFD topology. Detect via vendor_id == 4098 directly: the KFD CPU node reports vendor_id 0, so any 4098 node is an AMD GPU, while NVIDIA's KFD nodes report 4318 and stay excluded. Also rework the 'ROCm version could not be determined' warning into an actionable message (install the ROCm/HIP SDK; Arch/CachyOS: rocm-hip-sdk) so ROCm-less users know the concrete next step instead of silently landing on CPU-only PyTorch. * tests: replace the FNR==1 KFD invariant with the per-line vendor_id check The FNR==1 reset guarded the old paired gpu_id+vendor_id awk against cross-node state leakage. The new detection is a single atomic vendor_id==4098 line condition, so there is no per-node state to reset; assert the new invariant instead (single-line vendor match, and no /gpu_id/ pattern, which never matched inside properties). tests/studio/install/test_rocm_support.py: 344 passed, 2 skipped. * installer: mirror the KFD vendor_id fix in setup.sh + honest CPU-fallback summary Codex P2 follow-ups: - studio/setup.sh carried the same dead gpu_id-inside-properties awk, so a host install.sh now routes to ROCm still failed setup's independent AMD re-probe and got a CPU llama.cpp. Use the same per-line vendor_id 4098 check. - When the AMD GPU is detected but the torch index stays CPU, the summary printed the old false diagnosis (gpu none / "No GPU detected"). Gate both on _has_amd_rocm_gpu and say what actually happened: AMD GPU present, no usable ROCm, CPU fallback. - Structure test asserting setup.sh's KFD awk stays in sync with install.sh. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep KFD-only AMD hosts on the CPU fallback (Codex P2s) The KFD-topology fix makes _has_amd_rocm_gpu / _setup_amd_detected true on hosts that expose an AMD GPU to the kernel but ship no rocminfo/amd-smi. Detection alone does not mean ROCm is usable or that the gfx arch is known, and two downstream paths wrongly assumed it did: - studio/setup.sh forwarded --has-rocm with no gfx, so install_llama_prebuilt found no per-gfx bundle and dropped to a HIP source build (slow, or a hard failure without build deps) instead of the CPU prebuilt these hosts used to get. Now --has-rocm is forwarded for a gfx-unknown host only when hipcc is present; otherwise it keeps the CPU prebuilt. - install.sh get_torch_index_url selected a generic rocmX.Y index whenever the ROCm version was readable, but the Strix reroute only learns gfx from rocminfo/amd-smi, so a Strix KFD-only host landed on the broken _grouped_mm wheels. Now, when neither rocminfo nor amd-smi is present (gfx unknowable), it stays on CPU with a hint to install them. Detection and the improved diagnostics are unchanged; only the routing for gfx-unknown KFD-only hosts is made safe. Adds tests for both gates. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden KFD-only fallback: probe gfx, accept versioned hipcc (Codex P2s) Follow-up to the previous commit's two guards: - install.sh: the KFD-only torch guard tested only 'command -v rocminfo/amd-smi', so a host where those binaries exist but do not enumerate the GPU (gfx unreadable) slipped through and, with hipconfig/rocm-core present, still got a generic rocm index -- breaking Strix. Now it actually reads the gfx (rocminfo, then amd-smi list / static --asic, the same probe the reroute uses) and falls back to CPU whenever the arch is unreadable, not just when the binaries are absent. - studio/setup.sh: the hipcc gate missed a HIP toolchain installed only under a versioned prefix (/opt/rocm-*/bin/hipcc), which the source build at setup.sh:1663 does support, so such hosts were dropped to the CPU prebuilt unnecessarily. The gate now also accepts /opt/rocm-*/bin/hipcc. Tests updated to assert the gfx-read (not binary-presence) gate and the versioned hipcc path; full test_rocm_support.py green (347 passed). Verified the gfx probe by execution: rocminfo-with-no-gfx now routes to CPU, amd-smi fallback still resolves gfx. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Honor UNSLOTH_ROCM_GFX_ARCH before the CPU fallback for PR unslothai#7314 Seed both the gfx-unknown guard in get_torch_index_url and the Strix reroute from UNSLOTH_ROCM_GFX_ARCH before probing rocminfo/amd-smi, so a host that names its arch reaches the correct rocm index instead of being forced to CPU (or to the generic wheels) when the runtime probes can't enumerate the GPU. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Probe gfx with visibility masks cleared for PR unslothai#7314 (Codex P2) rocminfo/amd-smi honor ROCR/HIP_VISIBLE_DEVICES, so a container that masks the GPU (e.g. ROCR_VISIBLE_DEVICES=-1) would make the gfx probe read nothing and force CPU torch, even though the KFD-based AMD detection is env-independent and hipconfig can still supply the ROCm version. Clear the visibility masks for the rocminfo/amd-smi arch probe only (the Strix reroute keeps them for per-GPU index selection), so a masked/container host keeps its ROCm route. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Re-probe gfx unmasked in the Strix reroute when a mask hides all agents for PR unslothai#7314 (Codex P2) * Remove leftover conflict marker from the test merge * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Report an explicit CPU pin instead of a ROCm misdiagnosis for PR unslothai#7314 (Codex P3) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trigger the reroute re-probe on a set-but-empty visibility mask for PR unslothai#7314 (subagent review) * Guard the ROCm version chain against set -e when no source exists for PR unslothai#7314 (simulation find) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Preserve the inferred-gfx reroute for KFD-only hosts (Codex P2) The gfx-unknown CPU guard in get_torch_index_url fired before the runtime-less reroute could run: with the KFD topology fix, _has_amd_rocm_gpu is true on KFD-only hosts, so the reroute's '! _has_amd_rocm_gpu' gate never let _infer_linux_amd_gfx_arch route them to AMD per-arch wheels, regressing inferable boxes (PCI/cpuinfo/ lspci) from arch-specific PyTorch to CPU-only. - Factor the override->rocminfo->amd-smi gfx probe (masks cleared) into _probe_amd_gfx_arch, shared by the guard and the reroute gate so the two can't disagree on what 'readable' means. - Reroute gate now also fires when the GPU is detected but the probe is empty (KFD-only). Deliberate CPU fallbacks (old/unreadable ROCm version) all had a readable gfx and stay excluded. - The guard defers to the reroute (no false 'installing CPU-only PyTorch' promise) only when inference yields a supported family; otherwise the actionable CPU warning is unchanged. Executed tests: KFD-only host reroutes to repo.amd.com per-arch wheels and exports UNSLOTH_ROCM_GFX_ARCH for setup.sh; readable-gfx CPU fallback stays un-rerouted; undetected-GPU reroute unchanged; the guard's three inference outcomes covered. Suite: 375 passed, bash -n clean on both scripts. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix two false diagnostics on the KFD-only paths (Codex P3s) 1. get_torch_index_url: with UNSLOTH_ROCM_GFX_ARCH set on a KFD-only host that has no ROCm version sources, the no-version endpoint printed 'falling back to CPU-only PyTorch' even though the reroute (gated on the override) then installs the per-arch wheels. When the override maps to a wheel family, defer with an accurate message; an unmappable override keeps the CPU warning since the reroute can't route it either. 2. Runtime-less reroute: the KFD-only branch reached the warning 'ROCm runtime not visible (/dev/kfd, rocminfo, amd-smi)' although /dev/kfd is exactly what detected the GPU. The diagnostic now distinguishes KFD-visible/tooling-blind hosts from truly runtime-invisible ones. Executed tests: supported override defers without the false CPU warning, unsupported override and readable-gfx no-version hosts keep it; KFD-only reroute emits the KFD wording, undetected-GPU reroute keeps the original. Version sources are shimmed so the tests hold on dev boxes with a real hipconfig. Suite: 376 passed, bash -n clean. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: reject whitespace-only passwords * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: reject any whitespace in passwords * Studio: surface whitespace error in setup form, isolate auth test import --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
…ut (unslothai#7355) * Studio: make UI font size scale all text without moving layout The UI font size setting changes the root rem base, so only rem sized text reacted. Hundreds of px text classes, px font sizes in CSS, and chart labels stayed fixed, while rem based padding, widths and radii wrongly grew. Convert all text sizes to rem so every font follows the setting, and pin spacing, radius, container widths, sidebar and thread widths to px so layout no longer follows the rem base. Library styles (streamdown, react-flow) are re-based via overrides. All conversions are exact at the default 16px root, so the default rendering is unchanged. * Studio: keep logo at fixed size and fit tight controls at large UI fonts The logo lockups (sidebar wordmark with beta badge, onboarding wizard) are branding and now keep px sizes at any UI font size. Two controls clipped their text at the largest setting: the appearance color chips (fixed w-24) and the voice tab selects (fixed w-56). Both use min widths now, so they keep the default look at 16px and only grow when the text needs the room. * Studio: keep dropdown corners rounded when the menu scrolls A scrolling dropdown lost its rounded corners on the scrollbar side: WebKit paints the surface square when the rounded element itself hosts the scrollbar, which shows up in the desktop app whenever a menu overflows, for example at larger UI font sizes. Dropdown menu and select content now clip with overflow hidden and scroll an inner viewport instead. The surface padding insets the scrollbar clear of the curve, so corners stay rounded in every engine. Submenus are unaffected since sub content is portaled. * Studio: scale the logo lockups at half the UI font size rate Rather than pinning the logo, the sidebar lockup (sticker, wordmark, beta badge) and the onboarding lockup now follow the UI font size at half the rate of the change: size = base + (root - 16px) / 2, written as calc((base - 8)px + 0.5rem). A 4px font size change moves the logo by 2px, and the default 16px root renders the exact base sizes. * Studio: address review feedback on leading, grid tracks and select scrolling Numeric leading utilities (leading-3 through leading-10) derive from --spacing, so pinning spacing to px also froze their line-heights while the paired text sizes now scale. Define them as rem theme tokens so line-height follows the UI font size again; values are identical at the 16px default. Convert the grid tracks the rem-to-px codemod missed (rem followed by an underscore escaped the word boundary): the response details label column and the on-device folder rows. Make the Radix select viewport the bounded scroller instead of a wrapper div, so Radix's scroll handling and the browser scroll the same element. Restore the app's thin scrollbar with an inline style, which beats the scrollbar hiding stylesheet Radix injects at runtime. * Studio: cap voice select widths and update CI contracts
…thai#7246) The rename input only ignored the composition-confirming Enter, so on WebKit an Escape that cancels an IME candidate also cancelled the rename. Move the composition guard ahead of the key branch so both Enter and Escape are ignored while a CJK candidate is being composed.
…i#7183) * fix(dataprep): don't emit a degenerate chunk for empty text smart_chunk_text feeds empty / whitespace-only text (which tokenizes to zero tokens) into the single-chunk branch, which unconditionally returns one chunk. That yields a lone-EOS "document" (input_ids=[eos]) or, when the tokenizer has no eos_token_id, a zero-length input_ids=[] — an invalid sample that breaks a downstream collator/trainer. load_from_file already guards against this with a ValueError, but chunk_text, smart_chunk_text and load_from_files do not, so batch-loading a directory that contains an empty file silently injects garbage rows. Return no chunks when the tokenized text is empty, so empty inputs contribute nothing instead of a degenerate sample. load_from_file keeps its explicit ValueError (its guard runs first). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard empty/whitespace text before tokenizing in raw_text Real BPE/SentencePiece tokenizers emit tokens for spaces and newlines, so the len(tokens)==0 check let whitespace-only documents through as a degenerate lone-EOS sample. Guard on text.strip() before tokenizing (mirroring load_from_file), and raise in load_from_files when every file is empty so return_tokenized mode never falls back to a text-column dataset. Test now uses a whitespace-preserving tokenizer and covers both return_tokenized modes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com>
…nslothai#7241) The API key and Connections form reused grid tracks that only collapse at the viewport width, so inside the narrower settings pane the provider selector and API key input were clipped. Switch the form to container queries so it responds to the pane width and stacks to a single column when narrow. Other settings tabs are unaffected.
…root font size (unslothai#7359) * Studio: drive UI font size through a typography scale, not the root font size Follow up to unslothai#7355. The preference now writes --ui-font-scale (selected / 16) and a data-ui-font-size attribute on the root instead of mutating the root font size, and the applier clears any stale inline root font-size left by older builds. Because the rem base never moves, every layout-only rem-to-px conversion from unslothai#7355 is reverted to its original form: the spacing, radius and container tokens, sidebar and thread widths, grid tracks, calc margins and hub.css dimensions match pre-unslothai#7355 main again, which also restores rem-based accessibility scaling for users with a larger browser default font size. Typography scales through tokens in index.css, all exact at 16px: - The named Tailwind sizes (--text-xs through --text-4xl) multiply their defaults by the scale, so standard utilities scale - One token per design px size (--text-ui-8 ... --text-ui-34) replaces every arbitrary text-[Npx] class; leading-ui-* mirrors the exact line heights and the numeric --leading-3..10 scale as well - CSS font-size and line-height declarations multiply by the scale - Chart labels scale through a .recharts-text rule; streamdown and react-flow px text is re-based via scaled overrides; KaTeX's 1px layout trick stays fixed by design - The logo lockups keep their half-rate behavior via the scale var - The explicit Code font size remains unmultiplied Keeps the unslothai#7355 behavior fixes: color chip min width, voice select min/max widths, and the select and dropdown menus scrolling an inner viewport so their corners stay rounded. The whitespace-password and IME rename guards that merged alongside are preserved. * Studio: contract and Playwright coverage for the UI font size scale test_ui_font_scale_contract.py pins the mechanism (scale var written, root font size never mutated, tokens scaled, code font size not multiplied, the Radix select viewport owning scroll state) and guards against new raw pixel typography, with a documented allowlist for the recharts fontSize props covered by the stylesheet override and the offscreen clipboard textarea. playwright_ui_font_scale.py drives the real appearance controls: root font size fixed at 12/16/20, text and line height scale by size/16, sidebar width invariant, explicit code font size stays fixed, an overflowing dictation select scrolls its Radix viewport by keyboard and wheel, and the default restores exactly. Wired into the UI smoke workflow against the second studio boot. The thinking-compact and descender contracts move back to the rem and token forms now that layout values no longer need px pinning.
…onments (unslothai#7342) * Installer: restore interrupted updates and clean stale rollback environments * CI: run POSIX rollback lifecycle tests on Linux
Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
* Studio: add Voice settings tab (dictation, dictionary, read aloud) New Voice tab in Settings, placed just before About: - Dictation: microphone picker, browser STT engine, recognition language, and an inline mic test with a live transcript - Dictation dictionary: entries rewrite matching speech to their exact spelling and casing, applied in both dictation paths - Recent dictations: last 20 final transcripts with copy and clear, so text can be recovered if it lands in the wrong place - Read aloud: optional button on assistant responses with two engines, curated system voices (novelty and legacy voices filtered, quality ranked, capped at 20) or the TTS audio model loaded in Unsloth via /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview Settings persist in localStorage (unsloth_voice_settings) and are read at call time so changes apply without reloading the runtime. Adds en keys plus the tab label for ja, zh-CN and pt-BR. * Studio: drop the single option STT engine select, rename TTS option The STT engine dropdown only had one entry, so it added noise without giving a real choice. The engine row can come back once local STT models land. Also renames the TTS engine option Unsloth TTS model to Load TTS model to make the action clearer. * Studio: harden Voice settings against edge cases found in simulation Simulated the feature across Chromium, Firefox and WebKit plus node level unit runs and backend contract checks. Fixes from the findings: - Dictionary rewrite used a replacement string, so entries containing dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected the match). Switched to the callback form of String.replace - Persisted voice settings now validate types on hydration: non string micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean ttsEnabled fall back to defaults instead of flowing into the UI - Dictionary entries are trimmed, capped at 120 chars and re-sanitized on hydration - The Test dictation panel now falls back to the default microphone when the saved device is unplugged, matching the composer adapter Test coverage: 46 unit assertions (dictionary regex edge cases across unicode, word boundaries and injection, voice curation for simulated macOS, Windows and Linux voice inventories, corrupt storage merge), 13 backend contract checks against /audio/generate on an isolated instance, and 60 browser assertions across the three engines covering rendering, degradation without SpeechRecognition, curation in a real DOM, dictionary persistence with unicode and dollar entries, the no-model preview error path and corrupt localStorage recovery. * Studio: address Voice settings review feedback Verified each review comment before acting. Confirmed and fixed: - Editing a dictionary entry was broken in two ways: the store trimmed on every keystroke so spaces could not be typed, and clearing the field deleted the entry and unmounted the input mid edit. Updates now keep the raw value and a blur commit trims or removes the entry - The unplugged mic fallback checked instanceof DOMException, but a cross browser probe showed Firefox and WebKit throw OverconstrainedError objects that are not DOMExceptions, so the fallback never fired there. Matching on the error name now - When the browser ended a dictation test on its own (silence timeout), the mic stream stayed open. All recognition end paths now stop the tracks and save the transcript through a single finalize path - The studio TTS audio element now releases its WAV data URL as soon as playback ends, fails or is cancelled - Allow microphone now reports insecure contexts (no mediaDevices) accurately instead of claiming access was blocked - Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale overlays can translate it; en is the baseline and parity passes - unsloth_voice_settings added to the Reset all local preferences key list so voice preferences obey the reset - Non default microphones note that the system default is used when the browser speech engine cannot bind a specific device, since browsers without the start(track) overload ignore the argument silently Re-ran the full simulation set after the changes: 46 unit assertions, 13 backend contract checks and 60 browser assertions across Chromium, Firefox and WebKit all pass, plus a dedicated browser probe for the dictionary editing behavior. * Studio: use the chat mic icon in Voice settings for consistency The Voice tab and its buttons used the hugeicons Mic02 glyph while the chat composer uses a custom filled mic. Extract that composer icon into a shared lib/mic-icon component, drop the duplicate inline copies in thread.tsx and shared-composer.tsx, and use it for the Voice tab icon and the tab's mic buttons so the microphone looks the same everywhere. * Studio: address second round of Voice settings review feedback Verified each new comment against the current code first. One item was already fixed in the previous round (recording transcripts when the browser ends a dictation test on its own). Confirmed and fixed: - The microphone row showed a picker with generic names when browsers enumerate unlabeled devices before permission, leaving no way to grant access from the row. It now branches on whether labels are visible and shows Allow microphone otherwise - Compare chat dictation ignored the selected microphone. It now opens the chosen device with the same fallback rules as the main adapter, passes the track to recognition where supported and releases the stream when recognition ends - Closing the Voice tab cancelled the shared speechSynthesis even when read aloud was playing a chat message. Cleanup now only cancels when the tab owns an active preview - Double clicking Start test could race two recognizers and leak the first stream. A starting flag set before the getUserMedia await makes start reentrancy safe - Turning off the read aloud setting mid playback removed the only stop control. The stop button now renders whenever a message is speaking - When an engine lacks the start(track) overload, both dictation paths now release the selected device stream before retrying with the default microphone instead of holding it open - Read aloud support no longer requires Web Speech synthesis: the Unsloth TTS engine only needs audio playback, so it stays available in WebViews without speechSynthesis, with a clear error if the system engine is chosen there Not addressed here: cancelling in flight backend TTS generation on stop. The route runs generation in a worker thread without a cancellation path, which is shared pre existing behavior with audio chat generation and belongs in a backend change. All suites re-run green: 46 unit, 13 backend contract and 60 browser matrix assertions across Chromium, Firefox and WebKit, plus probes for the unlabeled device branch and the double click race. * Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item * Studio: guard dictation mic lifecycle in Voice test and Compare composer Release a microphone opened after the component unmounts, and stop Compare dictation on a permission or security failure instead of silently recording from the default device, matching the main chat adapter. * Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings - Join final dictation chunks with a space so recorded transcripts do not merge words - Ignore a stale recognizer onend so a quick stop then restart is not torn down - Use previewingRef so a double click on TTS preview does not orphan the first request - Keep the read-aloud stop control visible when a new run starts while a message is spoken - Stop the dictionary remove button from deleting an adjacent entry on a blur then click race * Studio: trim redundant Voice settings comments * Studio: fix Voice preview and Compare dictation edge cases - Only cancel the shared speechSynthesis for a system-voice preview, so stopping a Studio preview no longer stops an unrelated chat read-aloud - Release the Studio preview audio and its WAV data URL on normal completion - Iterate every finalized result in Compare dictation so batched phrases are kept - Cap persisted recent dictations to the last 20 on hydration * Studio: use clipboard fallback for recents and release failed preview audio - Copy recent dictations via the copyToClipboard helper so the execCommand fallback works in Safari and insecure http LAN contexts - Release the Studio preview audio when play() rejects, not just on ended/error * Studio: add local speech-to-text dictation engine Add an offline dictation engine that transcribes with a local faster-whisper model, alongside the existing browser (Web Speech) engine. The browser engine streams audio to Apple or Google speech services and needs internet; the new engine runs on the server, works offline, and drives any chat model without evicting it (it loads in the backend process, separate from the model subprocess). It also gives Firefox dictation, which has no Web Speech support. Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper is torch-free, so this does not disturb the existing model stack. Frontend: a Dictation engine setting (browser or local model), a curated model picker with sizes, and MediaRecorder capture posted to the transcribe route. The model warms automatically when the engine is selected, with live status. * Studio: stream local STT transcription as you speak Local dictation showed nothing until you stopped, because the whole clip was transcribed once on stop. Now the growing recording is re-transcribed on a fast pass every second and emitted as live interim text, with an accurate final pass on stop. Partial recordings decode fine, and the model refines earlier words as more audio arrives. Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast preview pass; the final stop uses the accurate path. * Studio: make local dictation stop instant and reliable Stopping local dictation waited for a final network transcription before the session ended, so the stop button did not flip and a second click ended the session early and dropped the text. Now stop commits the live transcript immediately, releases the mic at once, and ignores a second stop while finalizing. Previews run more often so the committed text is current. * Studio: record local dictation in short clips for reliable streaming Re-transcribing a growing buffer every second got slower as it grew, flooded the backend, showed stale words, and could leave the stop button stuck waiting on a backlog. Record short independent clips instead and transcribe each once, appending the text as you speak. Work per clip is bounded, so stopping is prompt (with a hard timeout as a safety net) and long dictations stay smooth. * Studio: dictate then transcribe once on stop, ChatGPT style Local STT dictation streamed by re-transcribing the growing clip, which was quadratic and saturated the backend (multi-second lag), and stop only halted the recorder without releasing the mic, so it kept recording. Record the microphone continuously, release it the instant the user stops, and transcribe the whole clip once. Stopping is immediate and the transcript lands in about a second. Also add the tiny model for the fastest option. * Studio: surface dictation and read-aloud failures instead of failing silently - Compare dictation reports microphone and speech-recognition errors via toast, reusing the main chat adapter's describeMediaError and describeSpeechError - Read-aloud toasts genuine model or synthesis failures while ignoring cancellations * Studio: ChatGPT-style recording bar for dictation Clicking the mic now drops the composer into a dedicated recording bar with a live waveform, a discard (X) and a confirm (tick), instead of a plain stop button. The tick stops recording and transcribes the clip; the X throws the recording away and keeps whatever text was already in the composer. The model adapter taps the mic with an analyser to drive the waveform, and the router tracks the live session so the X can cancel it without transcribing. * Studio: transcribe dictation while speaking, ChatGPT layout Match ChatGPT's recording layout: the bar now renders in place of the input with the left plus button kept, the waveform in the middle, and the discard and confirm buttons together on the right. Cut the post-confirm delay by transcribing in the background as the user talks. The audio is split at natural pauses (voice-activity detection off the same analyser that drives the waveform) and each clip is transcribed as it is cut, so confirming only has to finish the short final tail. The model is also warmed when recording starts so the first run never pays a cold load. * Studio: ChatGPT waveform, hide tools while dictating, faster STT Make the recording UI read like ChatGPT: the waveform is now a dense row of round dots that rise into thin centered bars, and while dictating only the plus button shows, with the mode badge and tool toggles hidden so the bar is just the waveform and controls. Speed up transcription: decode greedily (beam_size=1), which is several times faster on CPU with negligible accuracy loss on short dictation clips, and cap background segments at 6s so the final tail after confirm stays short. * Studio: finish ChatGPT voice bar and low-latency STT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: full-width waveform with a timer that freezes on stop Use the full-width waveform for the recording bar: brighter, bigger bars that advance on a fixed cadence (keeping peaks between advances) so they glide instead of racing by, inset from the composer edges. Keep a visible timer and the green confirm button, matching the ChatGPT reference, and freeze the timer and waveform the moment the user confirms. * Studio: fix multilingual local dictation * Studio: speed up dictation and release local STT * Studio: harden dictation finalization and STT decoding * Studio: restore Firefox dictation fallback * Studio: add dictation history manager * Studio: manage speech model downloads * Studio: remove em dash from voice model label * Studio: move dictation history into Voice * Studio: source local STT from Unsloth Whisper models Point the dictation STT sidecar and its Model Hub download entries at Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3) and run them through Transformers, so Studio only ever downloads Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs repos; keep the Model Hub as the only download path via local_files_only, and keep PyAV for audio decoding. Device selection uses float16 on CUDA and float32 on MPS and CPU, since Whisper's decoder is unstable in float16 on MPS and repeats tokens. Shorten the model picker labels to name plus download size and update the STT tests for the new backend. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: smooth dictation waveform and keep pill height * Studio: align STT model dropdown width and tidy voice copy * Studio: guide to local engine when browser dictation is offline * Studio: clarify voice section and STT model copy * Studio: keep STT warm with training-aware eviction * Harden STT lifecycle and browser compatibility * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model discovery test lint * Harden cross-browser microphone errors * Harden cross-browser microphone errors * Surface voice test recognition errors and fall back to Studio TTS - Voice test now toasts non-abort speech-recognition failures instead of ending silently, matching the main and Compare dictation paths. - Read-aloud routes to the backend model when the runtime lacks Web Speech synthesis (audio-only WebView), so it no longer errors immediately. * Fix reviewed STT lifecycle races * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix read-aloud fallback controls * Guard read-aloud stop when deleting a non-speaking message aui.message().stopSpeaking() throws unless this message is the one being read aloud, so calling it unconditionally rejected the delete handler before the message was removed. Only stop speech when this message is speaking. * Cap recent dictation transcript length before persisting Recent dictations only limited entry count, so a long transcript stored the full text in the persisted voice settings and a few could exceed the localStorage quota, throwing synchronously from the uncaught dictation cleanup path. Truncate each entry on save and on hydration, matching the dictionary cap. * Studio: keep dictation mic clickable and guide to local model Register the dictation adapter unconditionally so the mic stays enabled for any engine and starts working right after switching to the local model on an already-open thread. When the browser engine cannot run (Firefox, Brave, non-secure origins), clicking the mic shows a toast that points to the local speech-to-text model instead of leaving a disabled button. The toast stacks its action below the text with a fully rounded button. * Studio: add bottom padding below the dictation guidance toast button * Studio: increase bottom padding under the dictation toast button * Studio: add bottom padding inside the dictation toast button * Studio: add five Whisper defaults and custom model search Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end. Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary. Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: use public Unsloth Whisper repositories Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests. * Studio: update Whisper download sizes Reflect the cleaned public Tiny and Base repositories in the curated model labels. * Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes - Show the download size on the right of each model row so long names like Whisper Large v3 Turbo no longer hide it - Update curated Whisper sizes to the safetensors weights actually downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB - Drive the model list scroll from a wheel handler so the mouse wheel scrolls it inside the Settings dialog, not just the scrollbar - Add a search icon and shorten the placeholder to Search model * Studio: do not search when a dictation model is picked, shrink repo label - Treat the filled-in model text as a selection, not a query, so choosing a model no longer kicks off a Hugging Face search - Make the repository line under each model name smaller * Studio: tighten dictation model and local engine descriptions * Studio: keep model display on pick instead of the query, shrink row text - Guard the combobox input so selecting a model shows its name and does not echo the typed query back or start a search - Map the item label to the friendly display so picks fill the field - Reduce the model name and size text in each row * Studio: show only the model name in the dictation field, shrink size label - Drop the download size from the search field; the name alone is shown once a model is selected, with sizes kept in the dropdown list - Reduce the size label text in each row * Studio: clarify the dictation model description * Studio: drop Hugging Face from the dictation model description * Studio: move the dictation dictionary to its own Manage subpage - Replace the inline entry list with a Manage row, matching Dictation history, so a long dictionary no longer crowds Voice settings - Add a DictationDictionaryView subpage that holds the entry editor * Studio: match STT field font, use best voice for System default - Bump the dictation model field text to text-sm so it matches the engine dropdown next to it - Resolve the System default read-aloud voice to the top curated voice instead of the browser default, which is a robotic legacy voice on macOS * Studio: rerank read-aloud voices and drop duplicate voice entries - Rank by vendor quality, then the user's locale, then a preferred list of natural voices, so the best voice leads instead of the first alphabetically - Collapse voices that macOS reports twice under one name and language * Studio: fold dictionary and recents into the dictation section - Drop the separate Dictation dictionary and Recent dictations headings; their Manage rows now sit under Dictation, split by the row divider - Shorten the custom spellings description * Studio: add search and sort to dictation history - Filter saved dictations by text with a search field - Sort by newest, oldest, or A to Z; show a no-matches message - Keep Clear all available regardless of the current filter * Studio: settle cancelled STT loads before training and fix dictation review items Wait for a cancelled STT load to exit and release its memory before reporting it freed for training, so the loader cannot still be inside from_pretrained()/.to(device) holding VRAM when the training subprocess starts. A load that finishes before observing the cancel now gets unloaded so the memory is actually reclaimed. Clear the accelerator cache before the CPU fallback in load() so a failed CUDA/MPS load does not strand reserved VRAM once the sidecar is marked CPU-resident. Send the saved Hugging Face token when polling STT download progress so a gated or private repo resolves and shows the correct Load/Downloaded state instead of reporting missing. Mark the composer Dictate button as type="button" so clicking it does not also submit the draft when the composer already has text or attachments. * Studio: pin dictation settings per session and close STT startup races Capture the STT model and language when a dictation session starts and pass them to every queued segment and the warm-up load, so changing the model or language mid-recording no longer transcribes the same clip with the wrong model or a model that is not downloaded. Check the local runtime at the top of transcribe(), before the model cache lookup and the bounded audio decode, so a server missing PyTorch or Transformers returns 501 up front instead of decoding a long clip first. Treat the training startup window as active for STT device selection. start_training frees VRAM in before_spawn but only assigns _proc later, so a concurrent STT load could take the GPU that was just cleared. A startup flag now reports training active from the free until the process is live, forcing those loads to CPU; a finally clears it on every exit. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: stub the STT runtime check in transcribe orchestration tests transcribe() now verifies the local runtime up front, so the unit tests that exercise transcription orchestration must treat the runtime as present to keep passing where PyTorch, Transformers, and PyAV are not installed. Stub ensure_stt_available in the shared fixture and restore the real check in the availability and load-rejection tests. * Harden custom Whisper dictation models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add whisper.cpp dictation engine with per-engine downloads and history rework Engines - New GGML STT sidecar that runs a managed whisper-server subprocess with idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh) - Dictation engine picker now offers Browser, Local transcription (whisper.cpp), and Local transcription (Transformers) - Both local engines serve the same five curated Whisper models and download them directly with byte-level progress reported by /audio/stt/status - Models auto load on selection and when their download finishes - Unload and training admission account for both engines Benchmarks (Apple Silicon, greedy, warm, same checkpoints) - whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in about 0.45s vs 0.86s for Whisper Small - whisper.cpp GGUF path is unchanged by the Transformers addition (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s) Voice settings UI - Plain curated model select replaces the searchable combobox - Single download progress bar with transfer rate for both engines - Dictation history now stores every dictation with Show more pagination, a top Clear history action, and links back to the chat it was spoken into - Archived chats dialog gets the same pagination - Delete dialog offers deleting a dictation together with its chat Tests: 88 backend STT tests pass, including new snapshot download coverage. Frontend typecheck, lint, i18n parity, and production build pass. * Merge local engines into one option and source GGML models from unslothai Engine selection - The dictation engine dropdown is back to two choices: Browser and Local transcription. The selected model decides the backend: curated ids run GGML checkpoints through whisper.cpp, searched Hugging Face repositories run safetensors through Transformers - Model picker lists the curated models and searches Hugging Face for other Whisper repositories, validating them before selection. The trigger is a plain button so the selection never renders inside a text input - /audio/stt/status accepts a model query param so downloaded state works for custom repositories; the engine param on load, transcribe, and download routes is derived from the model everywhere Model source - Curated GGML checkpoints now download from the Unsloth-hosted unslothai/whisper-*-GGUF repositories (one repo per model) instead of ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob tracking are per-model Fixes - Voice settings and dictation history were not persisting: the quota-safe localStorage wrapper was declared after the store that uses it, so the persist storage factory failed silently. Every settings write also threw mid-click, which kept the model picker popover from closing on selection - is_model_downloaded now verifies config, preprocessor config, and real weight files instead of trusting an offline snapshot lookup, so a partial download left by an aborted fetch shows the Download button instead of failing to load - Removed whisper.cpp mentions from user-facing text: the ready status shows Loaded instead of the runtime name, picker rows show the source repository, and runtime error messages say local transcription runtime Verified with automated browser sessions and live API checks: selection closes the picker with no page errors, persisted settings hydrate on reload, a stale partial snapshot triggers download then loads on MPS and transcribes, and curated models download from the unslothai repos. 88 backend STT tests, typecheck, lint, i18n parity, and build pass. * Skip the duplicate source line for custom models in the STT picker A custom repository's display name is its id, so search results and the appended current selection rendered the same string twice. The source line now only renders when it differs from the name; curated rows keep their name, unslothai source repository, and download size. * Verify every shard of a sharded checkpoint in the downloaded check A snapshot holding one of N shards (or a corrupt shard index) passed the downloaded check and then failed at load. When model.safetensors.index.json exists, every shard in its weight map must now be present. Found by simulation; covered by a regression test. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename stale _starting references in the pump resilience tests The startup flag on TrainingBackend was renamed to _spawn_in_progress but two tests added alongside it still asserted on the old name, failing the Python 3.11 to 3.13 CI jobs. * Make the selected model row clearly highlighted in the STT picker The current selection was a faint background tint. It now uses the accent background with a medium weight name. Two line rows use a small corner radius; single line custom repo rows keep the pill shape. * Address review feedback on STT snapshot checks, VRAM release, and dictation UX Verify snapshot completeness in the load preflight so a partial download fails before the audio is decoded, for curated and custom repos alike. Drop the failed accelerator traceback before the CPU retry so the cache clear can actually release that memory. Keep unloading the GGUF sidecar after cancelling an in-flight Transformers load; both engines can hold memory at once. Allow Auto language with English-only .en checkpoints, matching the backend which sends no forced language. Keep the discard button usable while a transcription is pending so a slow or hung request cannot trap the composer in dictation mode. Stop linking Compare and settings test dictations to the unrelated active single chat thread. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move the CPU retry out of the exception handler On Python 3.10 the interpreter exception state keeps its own reference to the traceback, so dropping it from the caught exception was not enough to release the failed accelerator load during the retry. Leaving the handler before clearing the cache works on every supported version. * Address review feedback on session handoff, chat pinning, and server lifetime Starting a dictation from a second entry point now cancels the session it replaces, so the old recording cannot keep the microphone open or save a transcript with no discard button pointing at it. The linked chat is pinned when recording starts, so switching threads while a transcription finalizes cannot relink the transcript to the newly opened chat. whisper-server is now bound to Studio's lifetime like the other long-lived children: PDEATHSIG on Linux, the parent job object on Windows, and pid adoption so the shutdown sweep reaps it; before this it survived a Ctrl+C exit as an orphan still holding the model. * Remove the dictation mic test from Voice settings The composer dictate button covers the same check, so the test row, its transcript panel, the unsupported fallback row, and their strings and search entry are gone. * Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits GGUF (whisper.cpp) sidecar: - Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed. - Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind. - Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription. - Reject a missing model before decoding audio, matching the Transformers download preflight. Voice settings: - The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it. Dictation dictionary: - Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fix curated GGUF whisper filenames to match hosted repos The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin, not ggml-<id>.bin, so every curated dictation download and cached-path lookup 404'd and the whisper.cpp engine could never load a model. Point GGML_STT_MODELS at the real filenames and guard the naming with a test. * Studio STT: validate a custom dictation repo before downloading it The Transformers STT engine accepts an arbitrary owner/model repo, but the download route handed it straight to snapshot_download, pulling a possibly large non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper checkpoint first with the existing metadata-only validate_remote_model (no weights); curated ids short-circuit and the GGUF engine (curated-only) is unaffected. A non-Whisper repo now 422s before any download. * Studio STT: preempt a still-loading GGUF server for training admission A whisper-server still in its startup window binds accelerator memory but has no loaded_model yet, so training admission could miss it and launch into an OOM. Make the GGUF startup cancellable (cancel_pending_load signals an abort event and terminates the starting process without the load lock; _wait_for_server observes it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock until the killed server is reaped), and always fold the GGUF sidecar into the resident-STT summary so a resident Transformers model cannot mask a loading GGUF server. free_stt_model_for_training now cancels an in-flight load and waits for it to settle before training claims the memory. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio STT: fall back to Transformers when whisper-server is absent A curated dictation model (including the default small) hard-pinned the GGUF engine, but standard installs do not ship whisper-server, so every recording 501'd instead of using the Transformers engine that serves the same checkpoint -- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine: a GGUF request for a curated id (the only ids GGUF accepts, all Transformers- servable) downgrades to Transformers when whisper-server is unavailable, applied consistently to download, load and transcribe (not unload, which targets a specific engine). The Voice tab likewise falls back to the Transformers status so the model is not shown unavailable and download is not blocked. * Studio STT: hide custom Whisper caches from the legacy model pickers The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with only the owner/model id, which cannot reach the config-based Whisper check, so a downloaded custom (non-curated) Whisper checkpoint was still offered as a chat model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo config and hides it, matching the discovery route. * Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction - Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat model inventory and pickers, backend and frontend. Only their Transformers safetensors companions were hidden; the GGUF repos use a different org and a -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked into chat pickers. - Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the Transformers sidecar. transcribe() holds self._lock across the whole inference call, so /audio/stt status polls and training admission previously blocked behind an in-flight transcription. - stt_unload resolves through the serving resolver: a "gguf" pick on a host without whisper-server is served by the Transformers fallback, so unload must target that engine or the resident model is never freed. Unload also attempts every engine even if one raises, so a failure freeing one backend no longer skips the other. - free_stt_model_for_training frees the Transformers and GGUF sidecars under independent exception boundaries so a failure unloading one no longer skips the other before training claims the memory. Adds tests/test_stt_review_fixes.py covering all four. * Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness - The model dictation adapter sent the raw setting (the literal "auto") to the backend, while the browser engine resolves Auto via resolveDictationLanguage. A batch of non-English voice notes came back mostly English on Auto. Add resolveModelDictationLanguage: only the literal "auto" is resolved to a concrete locale, gated so it becomes a language the model AND Whisper can honor (mirroring the backend's known-whisper-languages set); an explicit language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire it into both adapter call sites. - GgmlSttSidecar._process_alive() read self._process twice; a concurrent unload() nulls it under the lock while loaded_model/device read lock-free, so a null between the two reads called None.poll(). Snapshot once. Adds a deterministic regression test. * studio: tighten comments and docstrings in the dictation modules * studio: harden dictation model downloads, GGML readiness, and recording paths Address review findings on the STT dictation feature: - build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom Studio home unless it carries the Studio ownership marker, matching the setup.sh policy, and marks trees it creates - _snapshot_is_complete validates every shard of a sharded PyTorch (pytorch_model.bin.index.json) checkpoint like the safetensors path, and requires tokenizer assets (tokenizer.json or vocab.json + merges.txt) - custom-repo downloads pin the revision resolved at validation time and restrict snapshot_download to the model/tokenizer/config/preprocessor file classes Studio loads - the GGML sidecar holds its port reservation until just before spawning whisper-server and only accepts readiness from a responder that both looks like whisper.cpp's server and belongs to the still-running managed child, probing twice, so mic audio cannot be posted to a foreign local process - the recording adapter transcribes every non-empty segment; the RMS meter only shapes segment boundaries and can no longer discard quiet speech - Compare-pane dictation can cancel a pending transcription on second click, with the button relabeled while finalizing - localStorage quota recovery halves the dictation history until the save fits, so small histories shrink too - the System default TTS voice resolves to the platform default voice - new dictation UI imports go through the chat and hub feature barrels Regression tests cover the build-script gate, sharded PyTorch and tokenizer completeness, revision pinning and allow patterns, and the whisper-server readiness probe. * Fix STT download and voice picker follow-ups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add dictation button regression coverage * Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294) * Studio STT: add prebuilt whisper.cpp (whisper-server) installer New install_whisper_prebuilt.py downloads a per-platform whisper-server bundle published by the unslothai/whisper.cpp prebuilt CI into the managed whisper.cpp dir (build/bin/whisper-server) so local dictation needs no compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py: host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the trust anchor, staging + install lock + atomic swap, traversal-safe extract, co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired into setup yet; the pins ship empty so every asset fails closed until the first fork release is published and its digests are reviewed in. * Studio STT: install prebuilt whisper.cpp during setup and update Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so `unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server into the managed whisper.cpp dir the sidecar discovers. It skips a user-set WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL, forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence. * Studio STT: harden whisper-server child env + WSL ROCm detection - Sidecar spawns whisper-server with a scrubbed child env that prepends the binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child. - find_whisper_server_binary now requires an executable, not just a file. - Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only; gfx parsing skips the gfx000 CPU agent and generic ISA lines. - Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the executable check, and the WSL rocm detection. * Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can detect and install a newer whisper-server release from inside the app: - backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json and compare the installed release against the newest unslothai/whisper.cpp release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a (major, minor, patch, serial) key with a strict downgrade guard; 24h cache; fail-open. - backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch and atomically swap the newest bundle, unloading the warm GGUF sidecar first. - backend/routes/whisper.py mounted at /api/whisper (update-status + update). - pyproject: add whisper_prebuilt_pins.json to studio package-data so the installer's trust anchor ships in the wheel (it is a data file, not a .py module, so package discovery alone does not include it; node_prebuilt_pins.json is listed for the same reason). Without this a pip-installed wheel had no pins and the prebuilt install aborted to Transformers STT. Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade guard, marker layouts, stale decision, fail-open). * Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust model: instead of a committed whisper_prebuilt_pins.json, verify every download against the release's own whisper-prebuilt-sha256.json checksum index, fetched from the same GitHub release. - parse_release_checksums / fetch_release_checksums / expected_sha256_for replace the pins layer. The index is validated for schema/component and that its release_tag matches the resolved release; an asset absent from it, a release that does not publish it, or a manifest sha256 that disagrees with it all fail closed to a source build. - resolve_release_tag now resolves the newest published release at runtime (or an explicit --published-release-tag), matching llama and the freshness check; removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in. - Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data entry (nothing to ship now, same as llama which has no committed pins). - Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on uncovered asset, tampered-manifest guard, newest-release resolution). This is a same-origin checksum (integrity, not authenticity), identical to the llama.cpp installer; pair releases with GitHub artifact attestations for provenance. * Resolve whisper prebuilt release via the download host (no GitHub API) Mirror install_llama_prebuilt.py's fast path: resolve the release tag from the releases/latest redirect and fetch the manifest + checksum index from constructed releases/download URLs, so the common install path makes zero api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour per IP; the download host is not). Fall back to the GitHub API only on a 404, malformed asset, or tag mismatch. * Studio STT: coverage-aware whisper prebuilt selection via a shared core whisper's select_artifact returned the first os/arch/backend manifest match and ignored the SM-coverage fields the release manifest already carries, so a Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks cuda13-newer. Extract the coverage-aware selection into a shared, component-agnostic core under studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and generalised over a normalised artifact. whisper's HostInfo now records the GPU compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and select_artifact routes CUDA/ROCm through the shared selector: every visible SM must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes, and "already matches" contract are unchanged. On the B200 the installer now resolves cuda13-newer, matching llama. * Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship libcudart/libcublas -- they load the same runtime the host already has. So the driver's advertised CUDA version is only an upper bound: a cuda13 bundle still needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the shared core and intersect it with the driver-compatible lines in select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g. torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13 one; a host with no CUDA runtime at all falls back to CPU. Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator truthiness, not a match) that made every major report present; add a real filesystem test that exercises the scan. * studio: harden shared prebuilt core to full llama parity Apply the review findings on the shared coverage-aware prebuilt-consumer core so whisper.cpp selection is exactly equivalent to the llama.cpp path. hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an index/UUID selector now reports has_usable_nvidia False instead of staying usable, via supports_explicit_visible_device_matching plus the physical / explicit-match branches, and _select_visible_rows now matches rows the way llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus fallback and has_physical_nvidia. Adds parse_macos_version. runtime_libs.py: the Linux on-disk scan now requires the exact libcudart / libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare versioned file without the SONAME symlink no longer counts as loadable. Hardens the ldconfig parse against an empty left-hand side. selection.py: fix the Blackwell/torch reordering so it keys on the covering runtime lines (falls through to the torch preference when the covering lines were filtered out), matching linux_cuda_choice_from_release. Corrects the compatible_runtime_lines_for_driver docstring: the bundles do not ship the CUDA runtime, so the driver version is only an upper bound and the caller must intersect with the on-disk scan. install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new HostInfo.macos_version) so a bundle that cannot load on the host OS version is dropped. Keep resolver stdout to only the JSON line by leaving logs on stderr in --resolve-prebuilt mode, and map an unexpected probe failure to prebuilt_available False instead of a traceback. Tests: new host-probe suite for the visible-device logic, exact-SONAME runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON, exit-code mapping, and the repo key. * studio: fix whisper prebuilt selection + launch parity gaps from review A parallel review surfaced integration defects where the whisper path could select or launch a bundle that cannot run on a concrete host. Each is fixed to match install_llama_prebuilt.py. macOS min_os: the manifest labels macOS requirements as macos-<version> (e.g. macos-14.0), which the version parser could not read, so the guard was a no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the platform prefix before parsing. ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact ROCm matching treats that token as the active GPU, a mixed APU + dGPU host (gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU sections and honors the visibility vars (empty / -1 -> no AMD GPU). --rocm-gfx override: recording the arch without setting has_rocm left the host on its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies has_rocm and clears NVIDIA state, like llama's _apply_host_overrides. CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so on a host whose CUDA runtime lives only in the PyTorch wheels the selection would gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch runtime dirs to the child loader path for CUDA bundles (bundle dir still first), mirroring binary_env. Also normalize a manifest artifact's supported_sms defensively (parity with llama's parser) and document that blackwell_min_toolkit_for_caps is retained for the Phase B llama Windows path. Not changed (verified parity, not defects): Linux/Windows min_os is enforced nowhere in llama (macOS only); the resolver is optimistic about the checksum index and the install path verifies. * studio: tighten prebuilt-core code comments * studio: lift shared prebuilt installer core out of the whisper installer * studio: reuse the llama.cpp prebuilt installer machinery for whisper * studio: unify llama and whisper prebuilt installers on a shared descriptor core * studio: consolidate prebuilt installer tests into the shared core suite Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every component-agnostic behavior runs against both descriptors: the full seven profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle stability, missing SM metadata, dotted SM normalization, no-driver fallback policy), the ROCm gfx family matrix, macOS min_os gating and its helper, backend resolution incl. cpu-fallback precedence and Intel-mac auto detect, checksum-index non-object and plain-lookup cases, the tar symlink/hardlink extraction guards moved from the llama suite, and the compute-cap, visible device, runtime-line and Blackwell helper value tables moved verbatim from the llama characterization suites. Delete only tests whose exact behavior the master now asserts for the same component: 40 pure-alias helper cases in test_selection_logic.py (replaced by value-identical master tables plus an alias-identity pin), 6 extraction moves and the master-absorbed zip-symlink case in the llama logic suite, 3 routing twins in test_rocm_support.py already pinned byte-for-byte in test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by the master whisper parameterization. Wrapper wiring pins, the llama release plan dialect, fingerprints and every llama-only behavior stay untouched. * studio: dedupe sidecar and update helpers into the backend prebuilt package * studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow * studio: consume paired slim whisper prebuilts via the llama ggml runtime * studio: serve every whisper backend from slim prebuilts * studio: drop the whisper fat per-accelerator selection chain unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan selection glue; keep slim selection + pairing, link_ggml_runtime, and one legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim release. Exit 2 now reads as prebuilt unavailable (whisper never source builds); setup already treats it that way. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire libomp runtime DLL alongside ggml in slim whisper installs llama's clang-built windows-arm64 ggml-base.dll imports libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL. Without it next to whisper-server.exe the loader fails with STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64 was affected. The empty-runtime guard still requires a real ggml library; libomp alone is not a pairing. * studio: drop whisper-side fat-selection support structure Slim whisper bundles are selected per os/arch only; all accelerator capability comes from the installed llama.cpp prebuilt, whose installer already did the coverage-aware selection. Remove the machinery that only existed to pick among fat per-accelerator whisper bundles: - prebuilt_core: delete the generic CUDA/ROCm coverage selection (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters, detected_cuda_runtime_lines, the exact-SONAME linux probe) that no shipped component routes through; llama keeps its own selection chain and whisper shadows select_artifact with the slim-only version. select_artifact is now a plain os/arch/backend first-match. - install_whisper_prebuilt: drop the HostInfo CUDA fields (compute_caps, driver_cuda_version, torch_runtime_line) and the torch runtime probe that populated them; nothing reachable reads them, and the resolver payload sources runtime_line from the artifact. - whisper_cpp_update: delete the standalone start_update job worker; whisper applies only run as the chained phase of the combined llama+whisper update. The status payload keeps its job field (idle). - routes/whisper: drop the progress logger that could never fire. - tests: remove tests of the deleted paths and tests duplicating the descriptor-parameterized core suite or the llama freshness suite. Contracts unchanged: resolver JSON keys, exit codes, marker fields, pairing logs, and the pinned pre-slim fat CPU escape hatch. * Address review feedback on the whisper prebuilt update and install paths - Pin the chained whisper phase to the release the freshness check offered, so the download-host latest pointer cannot reinstall an older build in a loop - Wire the whisper prebuilt install into setup.ps1 (Windows setup previously skipped it entirely) - Treat a non-executable server or missing wired ggml libraries as a broken install instead of reporting already matches - Keep whisper sidecar reloads out of the job-level reload flag and resync chat state after a partial chained update that unloaded llama - Repoint home and profile vars for the whisper-server subprocess at a managed scratch dir and drop credential-store pointers - Clear the prebuilt marker before the opt-in source build overwrite - Write the prebuilt marker with explicit utf-8 encoding * Tighten comments in the whisper prebuilt consumer * Harden the Windows whisper setup phase and the chained update edges - setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH / UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard before the atomic install, and forward the release-tag pin and ROCm hints like setup.sh - sidecar: a cpu-selected install launches whisper-server with --no-gpu (slim wiring links every llama backend, so the flag is what keeps a deliberate CPU choice off the GPU) - chained update: leave whisper unpinned on macOS (the llama phase can walk back there, and a newest-tag pin could be an impossible pairing on every retry) and treat installer exit 2 as kept-existing-runtime instead of failing the combined job - job.to_tag now comes only from the llama phase, so a whisper-only round cannot report a llama update that never ran * Fix slim whisper runtime follow-ups * Address remaining whisper update reviews * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address remaining prebuilt update reviews * Fix remaining chained update reviews * Fix remaining whisper runtime review edges * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: danielhanchen <unslothai@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com> --------- Co-authored-by: danielhanchen <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
unslothai#7328) * unsloth start/run: tool-call flags, positional model, grouped help Expose the existing tool-call controls as first-class CLI flags on both unsloth run and unsloth start, add positional model detection with a GGUF quant default, and group --help into rich panels. Flags (unsloth run): --enable-tool-call-healing/--disable-tool-call-healing (default on), --enable-tool-call-nudging/--disable-tool-call-nudging (default on). Resolved before any re-exec and written to the existing env controls (UNSLOTH_DISABLE_TOOL_CALL_HEALING, UNSLOTH_TOOL_CALL_NUDGE) so the in-venv server reads them at import; an omitted flag respects a value the parent already set. Flags (unsloth start): --enable-tools/--disable-tools (default off, passthrough), plus the same healing/nudging flags (default on). start conveys them to the auto-started run via the child env and the tools flag, so it stays correct even if run re-execs into an older Studio venv. Positional model: a leading org/name(:variant) token routes to --model when --model is absent, without stealing an option value or an agent passthrough arg. A bare GGUF repo with no variant defaults to UD-Q4_K_XL for the unsloth namespace and Q4_K_M elsewhere, applied only on the fresh auto-serve path so attaching to a loaded model never reloads. Help is grouped into rich panels (Model / Server / Session for start; Model / Server and network / Tool calls / Advanced for run) so --help reads cleanly. Adds unit coverage for the helpers, the start command-and-env forwarding, the positional/quant defaulting, and the run env resolution. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Positional model: reuse _is_hub_model_id so local dirs and paths are not stolen Route a bare org/name positional to --model only when it resolves as a hub id (via the existing _is_hub_model_id, which rejects local paths and existing dirs), so an OpenCode project dir like owner/repo is left for the agent. Apply the same guard to the auto-serve GGUF quant default so a local -GGUF path is not forced to a quant it may not contain. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * unsloth start: typer floor, drop redundant GGUF quant default, respect inherited tool-call env - Require typer>=0.12.0. The rich_help_panel options added here crash at import on typer<0.6, and the dependency was previously unbounded. - Stop forcing a default GGUF quant for a bare org/name-GGUF on auto-serve. The server's own quant preference already picks UD-Q4_K_XL for Unsloth uploads and Q4_K_M otherwise, and falls back when that exact quant is missing, so forcing a fixed variant broke external repos that only publish Q5_K_M/Q8_0. - Make the healing/nudging start flags tri-state so an omitted flag keeps an operator's inherited UNSLOTH_DISABLE_TOOL_CALL_HEALING / UNSLOTH_TOOL_CALL_NUDGE instead of overwriting it with the start defaults. * Fix start passthrough and inherited tool settings --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
…slothai#7362) The pip scan-packages extras shard has been red on main because openai 2.47.0 changed the code inside five previously baselined findings, so their evidence hashes no longer matched the allowlist. The hf-stack shard was about to go red the same way: unsloth-zoo 2026.7.5 changed two baselined test files. All seven reopened findings were re-verified against the exact resolved archives before re-baselining: - openai/_base_client.py: while True in SyncPage.iter_pages, the pagination iterator. - openai/auth/_workload.py: Azure IMDS and GCP metadata token providers for the documented workload identity federation feature. - openai/resources/{beta/responses,realtime,responses}: while True in websocket __aiter__ event loops; the loop bodies gained reconnect handling in 2.47.0, which is what shifted the hashes. - unsloth-zoo tests/test_vision_collator_audio.py: asserts that an inline /tmp/a.wav path is passed through by the audio collator. - unsloth-zoo tests/test_gemma4_forced_float32_ple_dtype.py: compile()/exec() of the project's own generated Gemma4 PLE cast helper source in tests. No existing entries were removed. All three shards now exit 0 locally against the same requirement sets CI uses.
…d option (unslothai#7240) The script the python tool runs was rendered inside a collapsible that closes when the run ends or the thread is reopened, so the code disappeared from the transcript and there was no way to save it. Render the script outside the collapsible so it stays visible, and add a Download button that saves it as script.py. Other tools and normal chat are unaffected.
… packing) (unslothai#7358) * feat(studio): offer MLX-supported optimizers on Apple Silicon The training form's optimizer dropdown only listed CUDA/bitsandbytes optimizers (adamw_8bit, paged variants, torch fused). On Apple Silicon the MLX trainer supports a different set (adamw, adam, lion, muon, sgd, adafactor) and remaps every bitsandbytes/torch name to plain AdamW, so the dropdown misrepresented what actually runs. Offer the MLX optimizer list when the device is a Mac, and derive the displayed value so the control is never blank: the shared CUDA default and the other bitsandbytes/torch options render as AdamW (exactly how the MLX backend normalizes them), while any other value is shown as-is so an unrecognized or non-canonical imported optimizer is never mislabeled. Non-Mac behavior is unchanged. The run-summary optimizer label now resolves from both lists. * feat(studio): show an MLX-appropriate optimizer tooltip on Apple Silicon The optimizer tooltip described "8-bit variants" and recommended "Fused" for vision models, neither of which is offered when training runs on MLX. On Apple Silicon, show a tooltip that matches the MLX optimizer set and notes that Lion typically needs a lower learning rate than AdamW. Copy-only: no change to the selected optimizer or the learning rate, and the non-Mac tooltip is unchanged. The new string is added to the English locale; other locales fall back to English until translated, matching how new keys are handled elsewhere. * fix(studio): label Mac CUDA-alias optimizers as AdamW in the run summary On Apple Silicon the run-configuration summary looked up the stored optimizer name directly, so a run that kept a CUDA/bitsandbytes default such as adamw_8bit was labeled "AdamW 8-bit" even though the picker shows "AdamW" and the MLX backend runs plain AdamW. Mirror the training form's derivation so those aliases are labeled AdamW in the summary too. Display-only: no change to the stored or submitted optimizer, and non-Mac summaries are unchanged. * feat(studio): disable LoftQ and sequence packing on Apple Silicon Neither LoftQ nor sequence packing is supported on MLX — the backend rejects LoftQ and the trainer silently forces packing off — yet the training form still offered both on Apple Silicon. Disable the LoftQ LoRA-init option (greyed and unclickable, with an inline "Not supported on Apple Silicon" note) and the "Enable packing" checkbox (greyed, with a tooltip explaining why), matching how the unsupported "Enable streaming" control is presented. Clearing effects reset a stale loftq/packing value to its default on Mac so the disabled controls never submit it. Non-Mac behavior is unchanged. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…ow-up to unslothai#7240) (unslothai#7363) * Studio: keep the executed Python script visible in chat, with download + viewport-gated highlight Always show the executed Python script under the tool card (not only inside the collapsible run/output section, which is unmounted from history), with Copy and a client-side .py Download button, so the script stays visible on reopen (unslothai#7165). The script is rendered eagerly, but shiki syntax-highlighting only runs once the block scrolls near the viewport (IntersectionObserver, 200px margin); until then a plain monospace placeholder shows the same source with matching padding, so there is no layout jump. This bounds highlighting to the cards actually on screen instead of tokenizing every script up front. Measured shiki cost is ~8 ms per typical 2 KB script, so eager highlighting of a long agentic transcript (20-50+ Python calls) would add ~170-420 ms of main-thread work on load; viewport-gating keeps it to the few visible cards (~15-35 ms) regardless of transcript length. Falls back to immediate highlight when IntersectionObserver is unavailable (SSR / tests). * Use a div for the pre-highlight placeholder so container [&_pre]:!p-0 doesn't strip its p-3 The placeholder shares the highlighted block's p-3 padding to avoid a layout jump, but as a <pre> it was caught by the container's [&_pre]:!p-0 !important rule and rendered with no padding, so the script shifted by p-3 when shiki swapped in. A plain div keeps the padding. * Match placeholder wrapping to the highlighted pre (whitespace-pre, not pre-wrap) The placeholder wrapped long lines while the highlighted Streamdown <pre> keeps them on one line and scrolls in the container's overflow-auto, so a script with a long line changed height when shiki swapped in. Use whitespace-pre so the placeholder scrolls the same way and the height stays stable. --------- Co-authored-by: danielhanchen <unslothai@gmail.com>
… (RCE fix) (unslothai#7364) * Studio STT: only load safetensors weights for custom dictation models The STT sidecar accepts arbitrary Hugging Face owner/model repos for custom dictation models and, when safetensors were absent, downloaded and loaded pytorch_model.bin through WhisperForConditionalGeneration .from_pretrained. PyTorch checkpoints are pickles that execute code during deserialization, and this path does not run the malware gate the normal model loader applies, so an authenticated client on an exposed Studio instance could load a crafted Whisper-looking repo and run code in the backend. Restrict custom STT repos to safetensors: the snapshot selector no longer falls back to pytorch_model.bin(.index.json), the cached-snapshot completeness check ignores pickle weights, and the load forces use_safetensors so a stray cached pickle still cannot execute. The five curated Whisper defaults already ship safetensors only, so this changes nothing for the built-in models. * STT: reject safetensors indexes that reference non-safetensors shards A safetensors index (model.safetensors.index.json) is attacker-supplied JSON and can name pytorch_model-*.bin shards in its weight_map. Transformers dispatches shard loading per file by extension, so those .bin shards still load through torch.load (pickle) even with use_safetensors set. Require every weight_map value to end in .safetensors in both the snapshot selector and the completeness check so no pickle shard is downloaded or reused.
…nslothai#7357) Add UNSLOTH_LLAMA_BACKEND=vulkan and --llama-backend vulkan to force the upstream Vulkan prebuilt on any host, persist llama_backend in the install marker, and re-assert it during Studio updates. On Windows AMD, auto-fallback to Vulkan when no detected gfx arch is in the upstream win-hip-radeon GPU_TARGETS set (e.g. gfx803 / RX 480). Mixed setups where at least one card is HIP-supported still default to HIP unless opted in.
Souravrajvi0
force-pushed
the
feat/vulkan-llama-backend-4af7
branch
from
July 23, 2026 17:07
c0bc95e to
c7931df
Compare
for more information, see https://pre-commit.ci
Honor ROCm family tokens (gfx110X), include fork-supported gfx1103, require a known active gfx before auto-Vulkan, and base the HIP floor check on the visible-device target instead of every physical GPU in hipinfo.
- llama_backend_from_env: stop reading UNSLOTH_LLAMA_CPP_BACKEND. That is a separate pre-existing setup variable meaning auto/cpu; setup.sh/setup.ps1 warn and ignore other values, so reading it here forced Vulkan behind that warning. Vulkan opt-in stays on UNSLOTH_LLAMA_BACKEND / UNSLOTH_FORCE_VULKAN. - _should_auto_vulkan_for_amd_windows: gate on not has_physical_nvidia (not merely has_usable_nvidia). A CUDA-masked NVIDIA card keeps has_physical_nvidia while has_usable_nvidia goes False; Vulkan ignores CUDA_VISIBLE_DEVICES and could enumerate the reserved card. Mirrors the Intel auto path. Explicit opt-in still overrides. - test fakes: validate_prebuilt_attempts/validate_prebuilt_choice gained a llama_backend kwarg; the four fake signatures in the fallback tests now accept it, clearing the TypeError that reddened Backend CI / Repo tests (CPU). Tests: UNSLOTH_LLAMA_CPP_BACKEND=vulkan no longer triggers Vulkan; hidden physical NVIDIA suppresses AMD auto-Vulkan while explicit opt-in overrides.
Keep the child process environment consistent with the cwd used to launch native POSIX coding agents. Some Node-based agents use PWD during project-root discovery, so inheriting a stale PWD can make them edit files in a parent or unrelated directory even when the wrapper process cwd is correct. Only apply this normalization for native POSIX launches. WSL-launched Windows shims stay on the existing WSLENV bridge path so path translation behavior is unchanged. Add regression coverage that launches an agent with a deliberately stale inherited PWD and asserts the child environment is normalized to os.getcwd(). Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>
The WINDOWS_HIP_PREBUILT_GFX_TARGETS allow-list omitted gfx1034, so _route_to_vulkan_prebuilt downgraded RX 6500/6400-class hosts to the upstream Vulkan prebuilt before published_rocm_choice_for_host could match the fork windows-rocm gfx103X bundle (whose members include gfx1034). Add gfx1034 to the allow-list and a regression test asserting it stays on the fork ROCm asset.
* fix composer reset * Studio: clear composer draft on send * Studio: cancel the pending draft save when clearing on send
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes unslothai#7357
Summary
UNSLOTH_LLAMA_BACKEND=vulkan(alias for legacyUNSLOTH_FORCE_VULKAN=1) and--llama-backend vulkaninstall the upstream Vulkan prebuilt on any host. The choice is persisted inUNSLOTH_PREBUILT_INFO.jsonasllama_backend: "vulkan"and re-asserted on Studio updates.win-hip-radeonGPU_TARGETS set (gfx1030+, gfx1100–1102, gfx1150/1151, gfx1200/1201), the installer selects Vulkan instead of a HIP build that cannot see the card (e.g. gfx803 / RX 480).Testing
Unit tests added/updated in:
studio/backend/tests/test_install_resolve_prebuilt.pystudio/backend/tests/test_llama_cpp_update.pyHardware validation not performed: This PR was implemented and tested with mocked host profiles only. We do not have access to a Windows AMD dual-GPU box (9070 XT + RX 480) or other HIP-below-floor hardware in this environment. Please verify on real hardware before merge:
llama-serversees the GPUUNSLOTH_LLAMA_BACKEND=vulkanopt-in works and survives a Studio updateRefs: unslothai#7356 (Studio Vulkan device visibility)