Skip to content

Anthropic translator silently drops unsupported completion params (top_p, stop, frequency_penalty, ...) instead of forwarding or warning #2620

Description

@truongsontung

Bug report

Describe the bug

When calling the Anthropic chat provider with completion params that AnthropicChatConfigParams does not declare (e.g. top_p, stop, frequency_penalty, presence_penalty, logit_bias, ...), those params are silently dropped — they never reach the Anthropic API and no warning is emitted.

This is inconsistent with the OpenAI path (the OpenAI translator warns when it discards params), so users get different, surprising behavior depending on provider.

Root cause

libs/giskard-llm/src/giskard/llm/translators/anthropic.py:

class AnthropicChatConfigParams(_BaseModel):
    model: str
    messages: Sequence[ChatMessage]
    max_tokens: int = 4096
    tools: Sequence[ToolDef] | None = None
    system: str | list[SystemTextBlock] | None = None
    temperature: float | None = None
    timeout: float | httpxTimeout | None = None
    output_config: dict[str, object] | None = None

_BaseModel extends pydantic BaseModel with the default extra="ignore" policy. In AnthropicChatTranslator.to_anthropic(...):

anthropic_params = AnthropicChatConfigParams(
    model=model, messages=messages, tools=tools, **params,
)

any **params key not in the declared fields is ignored by pydantic without error or warning. So top_p="0.5" passed by a caller is discarded, and the model silently runs with the provider default.

Steps to reproduce

from giskard.llm import ...  # call anthropic chat with top_p / stop

resp = anthropic_chat(
    model="claude-...",
    messages=[...],
    top_p=0.5,        # <- silently ignored
    stop=["\n"],
)
# resp was generated WITHOUT top_p / stop applied; no warning printed

Expected behavior

Either:

  • (a) forward the param to Anthropic when Anthropic supports it, or
  • (b) emit a warning (like the OpenAI translator does) when a param is dropped, so the caller knows the request differed from intent.

Actual behavior

Param is dropped silently; request succeeds but with different sampling behavior than the caller intended.

Environment

  • giskard-oss: current main
  • pydantic: v2 (default extra="ignore")

Related: #2613

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions