Problem
On Windows AMD, the installer only ever picks the win-hip-radeon prebuilt; Vulkan is auto-selected only for Intel/other GPUs (install_llama_prebuilt.py, direct_upstream_release_plan). Cards below the HIP prebuilt's gfx floor (e.g. RX 480, gfx803) are invisible to llama-server entirely.
A user with an RX 9070 XT + RX 480 worked around this by manually copying the official llama-bin-win-vulkan-x64.zip over the bundled build, which works (and Studio now understands it since #7356), but a studio update clobbers the swap back to HIP.
Proposal
- Explicit opt-in:
UNSLOTH_LLAMA_BACKEND=vulkan (and/or --llama-backend vulkan) makes the installer select the Vulkan asset on any host, and updates respect it. No behavior change for anyone else.
- Auto-fallback: on Windows AMD, when no detected gfx arch is supported by the HIP prebuilt, fall back to the Vulkan asset instead of shipping a build that can't see any card.
Non-goal
No auto-preferring Vulkan on mixed setups where at least one card is HIP-supported: HIP outperforms Vulkan on supported cards, so that stays the user's call via the opt-in.
Refs: #7356 (Studio-side visibility for Vulkan devices), user report: dual AMD (RX 9070 XT + RX 480) on Windows.
Problem
On Windows AMD, the installer only ever picks the
win-hip-radeonprebuilt; Vulkan is auto-selected only for Intel/other GPUs (install_llama_prebuilt.py,direct_upstream_release_plan). Cards below the HIP prebuilt's gfx floor (e.g. RX 480, gfx803) are invisible to llama-server entirely.A user with an RX 9070 XT + RX 480 worked around this by manually copying the official
llama-bin-win-vulkan-x64.zipover the bundled build, which works (and Studio now understands it since #7356), but a studio update clobbers the swap back to HIP.Proposal
UNSLOTH_LLAMA_BACKEND=vulkan(and/or--llama-backend vulkan) makes the installer select the Vulkan asset on any host, and updates respect it. No behavior change for anyone else.Non-goal
No auto-preferring Vulkan on mixed setups where at least one card is HIP-supported: HIP outperforms Vulkan on supported cards, so that stays the user's call via the opt-in.
Refs: #7356 (Studio-side visibility for Vulkan devices), user report: dual AMD (RX 9070 XT + RX 480) on Windows.