Skip to content

Fix absolute schema_path handling in Dataset.to_file#6143

Merged
DouweM merged 4 commits into
pydantic:mainfrom
VectorPeak:codex/dataset-schema-ref
Jun 30, 2026
Merged

Fix absolute schema_path handling in Dataset.to_file#6143
DouweM merged 4 commits into
pydantic:mainfrom
VectorPeak:codex/dataset-schema-ref

Conversation

@VectorPeak

@VectorPeak VectorPeak commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

This keeps Dataset.to_file() from serializing machine-local absolute schema paths when the schema file is next to the dataset file.

What Problem This Solves

When Dataset.to_file() receives an absolute schema_path in the same directory as the dataset file, it currently compares the schema path against the dataset file path itself:

elif schema_path.is_relative_to(path):
    schema_ref = str(_get_relative_path_reference(schema_path, path))

Sibling files are not relative to each other, so that branch is skipped and the generated dataset stores an absolute $schema reference instead of a portable sibling reference.

Before this change, a local reproduction wrote a machine-local path like:

"$schema": "C:\Users\MiaoXing\AppData\Local\Temp\...\cases_schema.json"

That makes checked-in datasets less portable and can expose local machine paths.

Change

  • Compare absolute schema_path against path.parent, the dataset directory.
  • Generate the $schema reference relative to that directory when possible.
  • Add a regression test for an absolute schema path next to the dataset JSON file.

Evidence

After this change, the same reproduction writes a portable sibling reference:

"$schema": "cases_schema.json"

The schema file is still written to the requested absolute path; only the serialized reference changes.

Validation

  • uv run pytest tests/evals/test_dataset.py::test_serialization_to_json tests/evals/test_dataset.py::test_serialization_to_json_with_absolute_schema_path
  • uv run ruff check pydantic_evals/pydantic_evals/dataset.py tests/evals/test_dataset.py
  • uv run ruff format --check pydantic_evals/pydantic_evals/dataset.py tests/evals/test_dataset.py
  • git diff --check

No issue; this is a narrow regression fix for an existing code path.

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • PR title is fit for the release changelog.

@github-actions github-actions Bot added size: S Small PR (≤100 weighted lines) bug Report that something isn't working, or PR implementing a fix labels Jun 30, 2026
@DouweM
DouweM merged commit 2f78dec into pydantic:main Jun 30, 2026
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Report that something isn't working, or PR implementing a fix size: S Small PR (≤100 weighted lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants