[Feature] VLM inference support

[amirvenus]

Hi,

It would be great if we could support models such as Qwen 2.5 VL 0.5B or Gemma3 3b that are multi-modal and have vision capabilities.

Thanks!

[shubham0204]

SmolChat depends on the llama.cpp project for running the LLM/SLM models. Currently, llama.cpp does not have a polished API for VLM inference, and this is being tracked here: ggml-org/llama.cpp#11292

Once the PR is merged to the master branch of llama.cpp, I can sync the submodule present in this project and make appropriate additions to SmolChat for supporting VLMs.

[amirvenus]

SmolChat depends on the llama.cpp project for running the LLM/SLM models. Currently, llama.cpp does not have a polished API for VLM inference, and this is being tracked here: ggml-org/llama.cpp#11292

Once the PR is merged to the master branch of llama.cpp, I can sync the submodule present in this project and make appropriate additions to SmolChat for supporting VLMs.

Hi,

I believe the VLM support has now been merged at llama.cpp upstream

ggml-org/llama.cpp#13050