fix(checks): join list[str] context in Groundedness/Contradiction judges instead of str()-ing it 🤖🤖🤖🤖

[kalra-mohit]

Groundedness and Contradiction both accept context as str | list[str] | MISSING — passing a list of source documents is the documented way to use them (see the Groundedness docstring example: context=["Paris is the capital of France.", "It's located in Europe."]). The problem was in get_inputs(): whatever context resolved to just got passed through str(...). For a plain string that's a no-op, but for a list, str() gives you Python's repr — square brackets, comma separators, and quotes around each item, with the quote character flipping per item whenever one contains an apostrophe (that's just how repr avoids having to escape it). That repr gets dropped straight into the {{ context }} slot of judges/groundedness.j2 / judges/contradiction.j2, so the judge model was reading something like:

['Paris is the capital of France.', "It's located in Europe."]

instead of the actual context text. An empty list came through as the literal string "[]".

This wasn't some obscure edge case — it was the tested, asserted behavior. Existing tests had lines like assert result.details["inputs"]["context"] == "['Direct context']", so the repr leak had effectively been locked in as "correct" output.

Root cause: get_inputs() declares the type as str | list[str] but never actually branches on it — every value falls through to the same str() call regardless of which half of the union it is.

flowchart LR
    A["context resolves to list[str]<br/>e.g. two context documents, one with an apostrophe"] --> B{"get_inputs()"}
    B -->|"before: str(value)"| C["Python repr string:<br/>brackets + comma-separated + quoted items<br/>(quote char flips per item)"]
    B -->|"after: format_prompt_text(value)"| D["items joined with '\\n':<br/>plain readable text, no syntax"]
    C --> E["rendered into {{ context }} in<br/>groundedness.j2 / contradiction.j2"]
    D --> E
    E -->|before| F["judge model sees list/repr syntax<br/>mixed into the prompt"]
    E -->|after| G["judge model sees the actual<br/>context documents"]

Loading

Fix is a small format_prompt_text() helper in judges/base.py: joins list values with \n (matching the "\n".join(...) convention already used elsewhere in this codebase, e.g. llm/translators/google_response.py, checks/export/junit.py), and falls back to plain str(value) for anything that isn't a list — strings, NoMatch, arbitrary trace payloads, all of which were already rendering correctly. Wired it into the context field of both Groundedness.get_inputs and Contradiction.get_inputs. Went looking for other built-in checks with the same str | list[str] context pattern; these two are the only ones, so the fix doesn't need to go any further.

No issue filed for this — ran into it while going through the judge prompt templates and the rendered context just looked wrong.

Testing

• Added a regression test per check, using a context list where one item has an apostrophe — that's the case that most clearly exposes the bug, since it's exactly what makes repr mix quote styles. Confirmed both fail against unmodified judges/*.py (actual output was the bracketed repr string), and pass after the fix.
• Updated the 4 existing assertions that had the buggy repr baked in as expected output (test_context_priority_over_trace in both files, test_direct_answer_and_context_are_passed_to_judge, test_custom_answer_and_context_keys).
• Full giskard-checks suite: 759 passed, 4 skipped. giskard-core, giskard-llm, giskard-agents, giskard-scan all green individually — untouched by this change.
• ruff format --check / ruff check clean on touched files; basedpyright --level error (what make typecheck runs per giskard-checks/pyproject.toml) reports 0 errors on touched files.

Coding agents

I've read AUTONOMOUS.md.

Checklist

• I've read the CODE_OF_CONDUCT.md document.
• I've read the CONTRIBUTING.md guide.
• I've written tests for all new methods and classes that I created.
• I've written the docstring in NumPy format for all the methods and classes that I created or modified.
• I've updated the uv.lock running uv lock (not applicable — no pyproject.toml changes)

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[kalra-mohit]

@henchaves whenever you get a chance to take a look, happy to address any feedback!

[ebarkhordar]

Nice cleanup, the repr leak into judge prompts is a real one.

One asymmetry worth a look: context now routes through format_prompt_text, but answer is still str(...) in get_inputs for both groundedness.py and contradiction.py. When answer is not passed explicitly it is resolved from the trace via answer_key, through the same resolve() in extraction.py, which returns a list whenever the JSONPath is multi-match or a slice/union/wildcard (len(matches) > 1 or _is_list_expression(...)). So an answer_key pointing at a list-valued path would hand the judge "['...', '...']", the same leak this PR fixes for context. The answer: str annotation does not prevent it, since JSONPath resolution is not type-coerced.

Was leaving answer on str() intentional (answer being conceptually single-valued), or worth routing it through format_prompt_text too for symmetry?