Eval bug: LFM2.5-8B-A1B reasoning is not captured in reasoning_content

[mattngaw]

Name and Version

$ llama-cli --version
version: 9402 (19e92c3)
built with GNU 14.2.0 for Linux aarch64

Operating systems

Linux

GGML backends

CUDA

Hardware

Raspberry Pi 5 + RTX 3060 (over Oculink)

Models

LiquidAI/LFM2.5-8B-A1B-GGUF:Q8_0

Problem description & steps to reproduce

Responses containing <think>...</think> are returned inline in message.content instead of being extracted into message.reasoning_content, even when reasoning_format is set to auto or deepseek.

This appears to be a template detection issue rather than a missing parser implementation. llama.cpp already has an LFM2.5 specialized parser in common_chat_params_init_lfm2_5, but the detection currently checks for the literal substring:

Steps to reproduce:

1. Run llama-server with LiquidAI/LFM2.5-8B-A1B-GGUF:Q8_0.
2. Send a chat completion request with reasoning_format: "auto".
3. Observe that the response has <think>...</think> inside choices[0].message.content and no choices[0].message.reasoning_content.

Example request:

curl -s http://127.0.0.1:18080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Hello there!"}],
    "stream": false,
    "reasoning_format": "auto",
    "max_tokens": 128
  }'

I verified locally that broadening LFM2.5 detection to also recognize templates containing both "List of tools: " and "tools | tojson" routes the same model through the LFM2.5 parser and returns reasoning_content separately.

First Bad Commit

No response

Relevant log output

Chat completions request

curl -s http://127.0.0.1:18080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [
      {
        "role": "user",
        "content": "Hello there!"
      }
    ],
    "stream": false,
    "reasoning_format": "auto",
    "max_tokens": 128
  }'

Actual response excerpt

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "<think>\nThe user says \"Hello there!\" ...\n</think>\nHello! How can I help you today?"
      }
    }
  ],
  "model": "LiquidAI/LFM2.5-8B-A1B-GGUF:Q8_0"
}

Expected response shape

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?",
        "reasoning_content": "The user says \"Hello there!\" ..."
      }
    }
  ]
}

/props excerpt showing content-only parsing

{
  "default_generation_settings": {
    "params": {
      "chat_format": "Content-only",
      "reasoning_format": "none",
      "reasoning_in_content": false
    }
  },
  "chat_template": "... \"List of tools: \" + (tools | tojson) ..."
}

[mattngaw]

to elaborate further, currently chat.cpp detects LFM2.5 output is detected before template rendering is done

the substring used for detection is

List of tools: [

however, the tool list is built dynamically via this line in the LFM2.5 template source

"List of tools: " + (tools | tojson)

since rendering (filling in the tools) is done after template detection, the parser fails to detect LFM2.5 and falls back to a content-only parser

thus <think> tags become visible

[mattngaw]

PR #23856 should make it in soon, for y'all waiting for the fix

[mattngaw]

Fixed by #24178