Eval bug: DeepSeek V4 Flash crash on CUDA + AMD BLIS (AOCL-BLAS), "illegal memory access" in ggml_backend_cuda_synchronize

[vbooka1]

Name and Version

build 9977 https://github.com/ggml-org/llama.cpp/releases/tag/b9977
also tested on 9950, same crash

Operating systems

Linux

GGML backends

CUDA

Hardware

AMD EPYC4 + NVIDIA RTX PRO 6000
CUDA : ARCHS = 1200 | FORCE_CUBLAS = 1 | USE_GRAPHS = 1 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | OPENMP = 1 | REPACK = 1

Models

DeepSeek V4 Flash UD-Q8_K_X from https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF/tree/main/UD-Q8_K_XL

Problem description & steps to reproduce

The model loads correctly except a few warnings:

0.42.404.971 W llama_kv_cache_iswa: using full-size SWA cache (ref: https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.42.452.100 W resolve_fused_ops: layer 2 is assigned to device CUDA0 but Lightning Indexer is assigned to device CPU (usually due to missing support)
0.42.452.106 W resolve_fused_ops: Lightning Indexer not supported, set to disabled

But crashes on the very first request, "CUDA error: an illegal memory access was encountered", "in function ggml_backend_cuda_synchronize at llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:2388"

./llama-server -m DeepSeek-V4-Flash-UD-Q8_K_XL-00001-of-00005.gguf --ctx-size 65536 --n-gpu-layers 99 --flash-attn on --no-mmap --mlock --swa-full --parallel 1 --direct-io --jinja --reasoning off --fit off -ot "long override string........" 

First Bad Commit

No response

Relevant log output

Logs

0.00.265.723 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.381.272 I srv    load_model: loading model 'DeepSeek-V4-Flash-UD-Q8_K_XL-00001-of-00005.gguf'
0.42.404.971 W llama_kv_cache_iswa: using full-size SWA cache (ref: https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.42.452.100 W resolve_fused_ops: layer 2 is assigned to device CUDA0 but Lightning Indexer is assigned to device CPU (usually due to missing support)
0.42.452.106 W resolve_fused_ops: Lightning Indexer not supported, set to disabled
0.42.716.877 W srv    load_model: swa_full is not supported by this model, it will be disabled
0.42.933.509 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 65536, kv_unified = 'false'
0.42.955.146 I srv  llama_server: model loaded
0.42.955.154 I srv  llama_server: listening on http://...:8080
1.46.950.861 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
1.46.951.156 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
1.48.044.118 E CUDA error: an illegal memory access was encountered
1.48.044.125 E   current device: 0, in function ggml_backend_cuda_synchronize at llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:2388
1.48.044.126 E   cudaStreamSynchronize(cuda_ctx->stream())
[New LWP 208793]
...
[New LWP 208678]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fffed8a69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007fffed8a69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fffed89b668 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fffed89b6ad in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007fffed9067c7 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x000000000187a8cb in ggml_print_backtrace ()
#5  0x000000000187aa18 in ggml_abort ()
#6  0x0000000000bd7cf3 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) ()
#7  0x0000000000bd9306 in ggml_backend_cuda_synchronize(ggml_backend*) ()
#8  0x000000000189ad95 in ggml_backend_sched_compute_splits(ggml_backend_sched*) ()
#9  0x000000000094d9c1 in llama_context::graph_compute(ggml_cgraph*, bool) ()
#10 0x000000000095362a in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) ()
#11 0x000000000095a485 in llama_context::decode(llama_batch const&) ()
#12 0x000000000095bcab in llama_decode ()
#13 0x0000000000604ec4 in server_context_impl::decode(int&, int, llama_batch&) ()
#14 0x0000000000607373 in server_context_impl::update_slots() ()
#15 0x00000000005b5911 in server_queue::start_loop(long) ()
#16 0x000000000055db83 in llama_server(common_params&, int, char**) ()
#17 0x000000000055f92f in llama_server(int, char**) ()
#18 0x00007fffed835ca8 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#19 0x00007fffed835d65 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#20 0x00000000005561e1 in _start ()
[Inferior 1 (process 208677) detached]

[am17an]

Can you try without --swa-full?

[vbooka1]

Can you try without --swa-full?

Same crash and same warnings about Lightning Indexer but no warning about SWA this time.

...
0.39.470.807 I llama_dsv4_comp_state: lid ratio = 4, state = 8 x 256, streams = 1, layers = 21, size =    0.33 MiB
0.39.471.251 I sched_reserve: reserving ...
0.39.480.532 I resolve_fused_ops: resolving fused Gated Delta Net support:
0.39.487.898 I resolve_fused_ops: fused Gated Delta Net (autoregressive) enabled
0.39.494.420 I resolve_fused_ops: fused Gated Delta Net (chunked) enabled
0.39.494.423 I resolve_fused_ops: resolving fused Lightning Indexer support:
0.39.500.992 W resolve_fused_ops: layer 2 is assigned to device CUDA0 but Lightning Indexer is assigned to device CPU (usually due to missing support)
0.39.500.995 W resolve_fused_ops: Lightning Indexer not supported, set to disabled
0.39.554.915 I sched_reserve:      CUDA0 compute buffer size =  4395.85 MiB
...
4.19.574.676 I srv  server_strea: conv_id=5tp9s9ccz1b (empty=0)
4.19.611.636 I srv    operator(): chat format: peg-native
4.19.614.932 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
4.19.614.935 I srv  get_availabl: updating prompt cache
4.19.614.940 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
4.19.614.943 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 65536 tokens, 8589934592 est)
4.19.614.944 I srv  get_availabl: prompt cache update took 0.01 ms
4.19.615.028 I slot launch_slot_: id  0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> ?temp-ext -> dist
4.19.615.035 I slot launch_slot_: id  0 | task -1 | sampler params:
        repeat_last_n = 64, repeat_penalty = 1.010, frequency_penalty = 0.000, presence_penalty = 0.000
        dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 65536
        top_k = 40, top_p = 1.000, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 1.000
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
4.19.615.035 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
4.19.615.044 I slot   operator(): id  0 | task 0 | new prompt, n_ctx_slot = 65536, n_keep = 0, task.n_tokens = 16287
4.19.615.050 I slot   operator(): id  0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
4.19.757.341 I slot   operator(): id  0 | task 0 | cached n_tokens = 1, memory_seq_rm [1, end)
4.19.763.225 I slot create_check: id  0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 0, n_tokens = 1, size = 11.687 MiB)
llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
4.20.699.562 E CUDA error: an illegal memory access was encountered
4.20.699.570 E   current device: 0, in function ggml_backend_cuda_synchronize at llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:2388
4.20.699.571 E   cudaStreamSynchronize(cuda_ctx->stream())
[New LWP 266670]
...
[New LWP 266556]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fffed8a69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007fffed8a69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fffed89b668 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fffed89b6ad in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007fffed9067c7 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x000000000187a8cb in ggml_print_backtrace ()
#5  0x000000000187aa18 in ggml_abort ()
#6  0x0000000000bd7cf3 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) ()
#7  0x0000000000bd9306 in ggml_backend_cuda_synchronize(ggml_backend*) ()
#8  0x000000000189ad95 in ggml_backend_sched_compute_splits(ggml_backend_sched*) ()
#9  0x000000000094d9c1 in llama_context::graph_compute(ggml_cgraph*, bool) ()
#10 0x000000000095362a in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) ()   
#11 0x000000000095a485 in llama_context::decode(llama_batch const&) ()
#12 0x000000000095bcab in llama_decode ()
#13 0x0000000000604ec4 in server_context_impl::decode(int&, int, llama_batch&) ()
#14 0x0000000000607373 in server_context_impl::update_slots() ()
#15 0x00000000005b5911 in server_queue::start_loop(long) ()
#16 0x000000000055db83 in llama_server(common_params&, int, char**) ()
#17 0x000000000055f92f in llama_server(int, char**) ()
#18 0x00007fffed835ca8 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#19 0x00007fffed835d65 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#20 0x00000000005561e1 in _start ()
[Inferior 1 (process 266554) detached]

[leoward1231ho-max]

Thanks for the update. Since the crash persists without --swa-full, the issue seems unrelated to the SWA cache settings. Could you please try running the server again with --flash-attn off to see if the illegal memory access is tied to the Flash Attention implementation on your setup?

…

On Mon, Jul 13, 2026, 3:15 AM vbooka1 ***@***.***> wrote: *vbooka1* left a comment (ggml-org/llama.cpp#25598) <#25598 (comment)> Can you try without --swa-full? Same crash and same warnings about Lightning Indexer but no warning about SWA this time. ... 0.39.470.807 I llama_dsv4_comp_state: lid ratio = 4, state = 8 x 256, streams = 1, layers = 21, size = 0.33 MiB 0.39.471.251 I sched_reserve: reserving ... 0.39.480.532 I resolve_fused_ops: resolving fused Gated Delta Net support: 0.39.487.898 I resolve_fused_ops: fused Gated Delta Net (autoregressive) enabled 0.39.494.420 I resolve_fused_ops: fused Gated Delta Net (chunked) enabled 0.39.494.423 I resolve_fused_ops: resolving fused Lightning Indexer support: 0.39.500.992 W resolve_fused_ops: layer 2 is assigned to device CUDA0 but Lightning Indexer is assigned to device CPU (usually due to missing support) 0.39.500.995 W resolve_fused_ops: Lightning Indexer not supported, set to disabled 0.39.554.915 I sched_reserve: CUDA0 compute buffer size = 4395.85 MiB ... 4.19.574.676 I srv server_strea: conv_id=5tp9s9ccz1b (empty=0) 4.19.611.636 I srv operator(): chat format: peg-native 4.19.614.932 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 4.19.614.935 I srv get_availabl: updating prompt cache 4.19.614.940 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000 4.19.614.943 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 65536 tokens, 8589934592 est) 4.19.614.944 I srv get_availabl: prompt cache update took 0.01 ms 4.19.615.028 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> ?temp-ext -> dist 4.19.615.035 I slot launch_slot_: id 0 | task -1 | sampler params: repeat_last_n = 64, repeat_penalty = 1.010, frequency_penalty = 0.000, presence_penalty = 0.000 dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 65536 top_k = 40, top_p = 1.000, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 1.000 mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900 4.19.615.035 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 4.19.615.044 I slot operator(): id 0 | task 0 | new prompt, n_ctx_slot = 65536, n_keep = 0, task.n_tokens = 16287 4.19.615.050 I slot operator(): id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end) 4.19.757.341 I slot operator(): id 0 | task 0 | cached n_tokens = 1, memory_seq_rm [1, end) 4.19.763.225 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 0, n_tokens = 1, size = 11.687 MiB) llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error 4.20.699.562 E CUDA error: an illegal memory access was encountered 4.20.699.570 E current device: 0, in function ggml_backend_cuda_synchronize at llama.cpp-b9977/ggml/src/ggml-cuda/ggml-cuda.cu:2388 4.20.699.571 E cudaStreamSynchronize(cuda_ctx->stream()) [New LWP 266670] ... [New LWP 266556] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 0x00007fffed8a69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x00007fffed8a69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fffed89b668 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007fffed89b6ad in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007fffed9067c7 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000187a8cb in ggml_print_backtrace () #5 0x000000000187aa18 in ggml_abort () #6 0x0000000000bd7cf3 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) () #7 0x0000000000bd9306 in ggml_backend_cuda_synchronize(ggml_backend*) () #8 0x000000000189ad95 in ggml_backend_sched_compute_splits(ggml_backend_sched*) () #9 0x000000000094d9c1 in llama_context::graph_compute(ggml_cgraph*, bool) () #10 0x000000000095362a in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () #11 0x000000000095a485 in llama_context::decode(llama_batch const&) () #12 0x000000000095bcab in llama_decode () #13 0x0000000000604ec4 in server_context_impl::decode(int&, int, llama_batch&) () #14 0x0000000000607373 in server_context_impl::update_slots() () #15 0x00000000005b5911 in server_queue::start_loop(long) () #16 0x000000000055db83 in llama_server(common_params&, int, char**) () #17 0x000000000055f92f in llama_server(int, char**) () #18 0x00007fffed835ca8 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #19 0x00007fffed835d65 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6 #20 0x00000000005561e1 in _start () [Inferior 1 (process 266554) detached] — Reply to this email directly, view it on GitHub <#25598?email_source=notifications&email_token=CAYCPVSGTIMYHB2UPIQ2TF35ESLB3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOJVGU4DIOJTGQYKM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#issuecomment-4955849340>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/CAYCPVQZLVENCHRZMVTRPTL5ESLB3AVCNFSNUABFKJSXA33TNF2G64TZHM3DCMRTGU2DOOBUHNEXG43VMU5TIOBWHA4DCOJQHA4KC5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/CAYCPVUNVUEMZANYSED3L535ESLB3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOJVGU4DIOJTGQYKM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSVGM33PORSXEX3JN5ZQ> and Android <https://github.com/notifications/mobile/android/CAYCPVW6YT7ZIKTDJ4VOEUD5ESLB3A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOJVGU4DIOJTGQYKM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>. Download it today! You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[vbooka1]

I have found the reason of crash. Rebuilt llama-server with no FORCE_CUBLAS, CUDA graphs and PDL, but still was getting the same crash.
Then I've rebuilt llama-server without BLIS and crash has gone away, so the reason of crash is AMD BLIS: https://github.com/amd/blis
I use version 5.2 but the latest is 5.3, perhaps this bug was fixed already, I will try to update these libraries and check again later.

cmake -B build -DGGML_CUDA=ON ....... -DGGML_CUDA_FORCE_CUBLAS=ON -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=AOCL_mt -DGGML_BLAS_USE_BLIS=ON -DBLAS_INCLUDE_DIRS=/usr/include/blis -DBLAS_LIBRARIES="-fopenmp /usr/lib/libblis-mt.so"

ldd llama-server | grep -i bl.s
        libblis-mt.so.5 => /usr/local/lib/libblis-mt.so.5 (0x00007ffff7a00000)
        libcublas.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublas.so.13 (0x00007ffff4000000)
        libcublasLt.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublasLt.so.13 (0x00007fffc9400000)

= crash

cmake -B build -DGGML_CUDA=ON ....... -DGGML_CUDA_FORCE_CUBLAS=OFF -DGGML_BLAS=OFF -DGGML_BLAS_USE_BLIS=OFF"

$ ldd llama-server | grep -i bl.s
        libcublas.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublas.so.13 (0x00007ffff4600000)
        libcublasLt.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublasLt.so.13 (0x00007fffc9a00000)

= no crash

cmake -B build -DGGML_CUDA=ON ....... -DGGML_CUDA_FORCE_CUBLAS=ON -DGGML_BLAS=OFF -DGGML_BLAS_USE_BLIS=OFF"

ldd llama-server|grep -i bl.s
        libcublas.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublas.so.13 (0x00007ffff4600000)
        libcublasLt.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublasLt.so.13 (0x00007fffc9a00000)

= no crash, so FORCE_CUBLAS is not the reason.

It is interesting that even with GGML_CUDA_FORCE_CUBLAS=OFF the CUBLAS libraries are still linked into llama.cpp, is that normal?

Please note that llama-server compiled with BLIS crashes only when inferencing DeepSeek v4 Flash, all other models I have tried work correctly with GGML_BLAS_USE_BLIS.

[vbooka1]

Thanks for the update. Since the crash persists without --swa-full, the
issue seems unrelated to the SWA cache settings.

Could you please try running the server again with --flash-attn off to see
if the illegal memory access is tied to the Flash Attention implementation
on your setup?
…

AMD BLIS + FORCE_CUBLAS=ON + flash-attn off = crash
AMD BLIS + FORCE_CUBLAS=ON + flash-attn on = crash
AMD BLIS + no FORCE_CUBLAS + flash-attn off = crash
AMD BLIS + no FORCE_CUBLAS + flash-attn on = crash
no BLIS + FORCE_CUBLAS=ON + flash-attn off = no crash (PP ~ 190 t/s)
no BLIS + FORCE_CUBLAS=ON + flash-attn on = no crash (PP ~ 195 t/s)
no BLIS + no FORCE_CUBLAS + flash-attn off = no crash (PP ~ 220 t/s)
no BLIS + no FORCE_CUBLAS + flash-attn on = no crash (PP ~ 230 t/s)
=> crash reason = AMD BLIS

PP values are strange, if I remember correctly adding FORCE_CUBLAS made prompt processing faster before, but now it is 20% slower! Should I create a separate bug report for this? I need to recheck it with the different LLMs though, maybe it is again only with DeepSeek V4.

[am17an]

No performance guarantees are provided w.r.t FORCE_CUBLAS, if it was universally faster it would be the default. So no need to file an issue

[minionprocyk]

I was having this crash and came here for help. I have a AMD Ryzen 7 5700X3D and rtx pro 6000 I fixed this crash also by building llamacpp with -DGGML_BLAS=OFF -DGGML_BLAS_USE_BLIS=OFF. Otherwise deepseek crashes either in the first or second prompt

[vbooka1]

I use version 5.2 but the latest is 5.3, perhaps this bug was fixed already, I will try to update these libraries and check again later.

I have upgraded all libraries

• https://github.com/amd/aocl-utils/releases/tag/5.3
• https://github.com/amd/blis/releases/tag/5.3
• https://github.com/amd/libflame/releases/tag/5.3

built llama.cpp with

-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=AOCL_mt -DGGML_BLAS_USE_BLIS=ON -DBLAS_INCLUDE_DIRS=/usr/include/blis -DBLAS_LIBRARIES="-fopenmp /usr/lib/libblis-mt.so" -DLAPACK_LIBRARIES="/usr/lib/libflame.so;/usr/lib/libaoclutils.so;-lstdc++" -DUSE_OPTIMIZED_LAPACK_BLAS=ON

and llama-server did not crash! @minionprocyk try updating all libraries and compiling llama.cpp with the same flags.

$ ldd llama-server |grep -i bl.s
        libblis-mt.so.5 => /usr/local/lib/libblis-mt.so.5 (0x00007ffff7800000)
        libcublas.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublas.so.13 (0x00007ffff3e00000)
        libcublasLt.so.13 => /usr/local/cuda-13.2/targets/x86_64-linux/lib/libcublasLt.so.13 (0x00007fffc9200000)
$ ls -l /usr/local/lib/libblis-mt.so.5
lrwxrwxrwx 1 root root 28 Jul 14 12:47 /usr/local/lib/libblis-mt.so.5 -> /usr/lib/libblis-mt.so.5.3.0

[vbooka1]

I did not experience crashes anymore, this seems to be resolved by updating all AMD BLIS related libraries from 5.2 to 5.3.