Skip to content

server : add slot_prompt_similarity getter/setter#22393

Closed
bernardladenthin wants to merge 2 commits into
ggml-org:masterfrom
bernardladenthin:master
Closed

server : add slot_prompt_similarity getter/setter#22393
bernardladenthin wants to merge 2 commits into
ggml-org:masterfrom
bernardladenthin:master

Conversation

@bernardladenthin

Copy link
Copy Markdown
Contributor

Overview

Add public getter and setter methods for the slot_prompt_similarity field in server_context.

Changes

  • Added get_slot_prompt_similarity() const method to retrieve the current threshold value.
  • Added set_slot_prompt_similarity(float value) method to update the threshold value.

Additional information

This change exposes the slot-similarity configuration as part of the public server_context API, so callers can query and modify slot-selection behavior at runtime without reloading the model.

The motivation is API parity for embedders. I'm refactoring bernardladenthin/java-llama.cpp using Claude Code to drop its hand-ported server.hpp and link upstream's server_context directly. In the previous code path the JNI layer could update this threshold at runtime; with the new upstream API surface, that capability is no longer reachable, since slot_prompt_similarity lives inside the impl with no setter.

I don't have a concrete operational use case beyond restoring the previous embedder behavior, and I'm happy to drop the PR if you'd rather keep the field strictly load-time configurable. If so, it might also make sense to drop this feature on the Java-binding side. Please give an suggestion.

Requirements

  • YES: I have read and agree with the contributing guidelines. I've not setup a CI because changes are minimal.
  • AI usage disclosure: partial: written with Claude Code (Opus). I reviewed every line and confirmed the design against the existing on_sleeping_changed pattern.

@bernardladenthin

Copy link
Copy Markdown
Contributor Author

Hey @ngxson, @pwilkin, this is a very small change. I'd appreciate it if you could find a few minutes to review it. Ty!

@ngxson

ngxson commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

I don't think internal data should be exposed here, it is single-use-case and not thread-safe.

Instead, better to keep it on your downstream code.

@ngxson ngxson closed this Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants