test: add property-based roundtrip tests for to_feather/from_feather#4239
Open
TaiSakuma wants to merge 1 commit into
Open
test: add property-based roundtrip tests for to_feather/from_feather#4239TaiSakuma wants to merge 1 commit into
to_feather/from_feather#4239TaiSakuma wants to merge 1 commit into
Conversation
Add three Hypothesis tests in tests/properties/operations/: - test_roundtrip: option-free arrays reconstruct exactly - test_roundtrip_masked: option-type arrays roundtrip through nullable feather columns, without comparing content classes - test_roundtrip_stable: one write/read cycle brings the conversion to a fixed point, compared as Arrow tables `to_feather` requires a real filesystem path (its destination goes through `os.fsdecode`), so unlike the parquet sibling the tests write to a module-scoped `tmp_path_factory` directory, with a per-thread filename for pytest-run-parallel. Generation excludes arrays affected by known issues, one predicate function per issue so each released fix deletes exactly one function: scikit-hep#4221, scikit-hep#4222, and scikit-hep#4229 in `has_issues`, scikit-hep#4230 and scikit-hep#4238 via strategy flags, and scikit-hep#4219 in the dtype filter. The format's own limits live in `feather_compatible` and `feather_stable`, which are all that remains when the cited issues are fixed. Validated with 10,000 examples per test (nightly profile), fixed seeds 1-8, and under pytest-xdist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TaiSakuma
added a commit
to TaiSakuma/awkward
that referenced
this pull request
Jul 23, 2026
Split the generation predicates of test_to_from_arrow.py into the structure introduced by the feather sibling (scikit-hep#4239): `arrow_compatible` and `table_compatible` hold the permanent limits of the conversion (mergeable union contents merge on read, an optioned unknown gains an option, plus the table-wrapping record clauses), while `has_issues` composes one `_has_issue_NNNN` function per known issue (scikit-hep#4221, scikit-hep#4222, scikit-hep#4228, scikit-hep#4229) so each released fix deletes exactly one function. The filters name both parts inline, and the stability test's filter is now literally `not has_issues(a)`. No behavioral change: a 40,000-example equivalence check against the original merged predicates over all four generation domains found no disagreement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR4239 |
Member
|
Just a general comment @TaiSakuma, I'd advise on focusing to property testing for the actual common awkward functions first. It's a lot more useful for the users. I would bet that not a single person has ever used this function. At least I've never seen it or used it myself and there are no instances of |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds property-based roundtrip tests for
ak.to_feather/ak.from_feather, the fifth member of the family after buffers, numpy (#4215, #4225), parquet (#4218), and arrow (#4231). Arrays are generated with hypothesis-awkward.Three tests:
test_roundtrip— option-free arrays reconstruct exactly (ak.array_equal).test_roundtrip_masked— option-type arrays roundtrip through nullable feather columns; content classes are not compared becausefrom_feathernormalizes option layouts.test_roundtrip_stable— one write/read cycle brings the conversion to a fixed point: the roundtrip may lose type information (anonymous single-field records unwrap, a zero-field record loses its length), but converting the reconstruction again produces an identical Arrow table (pyarrow.Table.equals).Unlike the parquet sibling, the tests perform real file I/O:
to_featherpasses its destination throughos.fsdecode, so an fsspecmemory://path is not accepted. The files go to a module-scopedtmp_path_factorydirectory (a function-scoped fixture would trip Hypothesis'function_scoped_fixturehealth check) with a per-thread filename for pytest-run-parallel.The generation predicates separate two kinds of exclusion, which is also a structure proposal for eventually refactoring the sibling files:
feather_compatibleandfeather_stableencode limits of the table representation itself, whilehas_issuescomposes one_has_issue_NNNNfunction per known issue (#4221, #4222, #4229) so that each released fix deletes exactly one function; #4230 and #4238 are excluded by strategy flags and #4219 by the dtype filter. When all cited issues are fixed, the filters reduce to the two format-limit predicates.#4238 (union-type arrays produce an invalid feather file; root cause apache/arrow#50623) was found while validating these tests.
Validation: 10,000 examples per test under the nightly profile (~3 min), fixed seeds 1–8, pytest-xdist, and a 20,000-example equivalence check between the decomposed predicates and the originally validated merged predicate.
🤖 Generated with Claude Code