Skip to content

Releases: scikit-hep/awkward

Version 2.11.0

Choose a tag to compare

@TaiSakuma TaiSakuma released this 20 Jul 21:29

What's Changed

New features

  • feat: add lazy IR scheduling layer by @ianna in #4173
  • feat: allow row group size in bytes as a string in the to_parquet family of functions by @ikrommyd in #4202

Bug-fixes and performance

  • fix: include kernel-specification.yml in source distribution by @5U5H4NTH006 in #4179
  • fix: default row group size is very big and not what pyarrow defaults to by @ikrommyd in #4191
  • fix: handle FutureWarning for deprecated feather by @ianna in #4198
  • fix: nan-variant reducers drop behavior and attrs by @henryiii in #4093
  • fix: validation no-ops and broken error paths in contents by @henryiii in #4102
  • fix: form and type equality bugs by @henryiii in #4101
  • perf: vectorize union flattening and cache nplike feature probes by @henryiii in #4109
  • fix: assorted operation bugs (softmax, enforce_type, merge_union_of_records, and more) by @henryiii in #4107
  • fix: categorical hashing, delayed-backend detection, and pretty-printing by @henryiii in #4106
  • fix: ensure length is a numpy scalar for CUDA stability by @ianna in #4194
  • fix: any_backend_is_delayed should only be iterating over sequences of arrays for now by @ikrommyd in #4208
  • fix: transient attrs should not be written to parquet by @ikrommyd in #4209

Other

New Contributors

Full Changelog: v2.10.0...v2.11.0

Version 2.10.0

Choose a tag to compare

@TaiSakuma TaiSakuma released this 02 Jul 20:26

Highlights

  • Much faster list reductions (ak.sum, ak.max, …) — about 5× faster on average (geometric mean), 10×+ on long lists #4056
    • Kernels rewritten: CPU (awkward-cpp), GPU (cuda.compute)
    • Exact floating-point results may differ from the previous release
    • The same arrays may have different internal layouts than before
    • Removed the undocumented ak.index.EmptyIndex / ZeroIndex / LazyIndex
  • Many bug fixes and improvements from code review by Claude Fable 5 (PRs 4089–4108) #4085

What's Changed

New features

Bug-fixes and performance

  • fix: add missing .attrs support in parquet serialisation by @albert-physics in #4079
  • fix: test_pandas_attr_serialisation should have unique parquet path per thread by @ikrommyd in #4110
  • fix: passing a non-Collection iterable to pytest.parametrize is deprecated by @maxymnaumchyk in #4144
  • fix: datetime.time conversion and binary buffer copy in bindings by @henryiii in #4089
  • fix: properly error on conversion of IndexedOptionArray with an EmptyArray content and length > 0 to a ByteMaskedArray by @ikrommyd in #4127
  • fix: named-axis context sharing and ak.where broadcasting crash by @henryiii in #4108
  • fix: require pyarrow>=17.0.0 by @henryiii in #4152
  • fix: wrong results from string sort/unique kernels by @henryiii in #4091
  • fix: int64 precision loss in sort comparators and per-comparison branching by @henryiii in #4090
  • fix: LayoutBuilder Union form JSON and Record map-type support by @henryiii in #4094
  • fix: misc integration bugs (hist, avro, numexpr, RDataFrame) by @henryiii in #4096
  • fix: modify cache key for macOS deployment in workflow by @ianna in #4158
  • fix: reject non-integer float indices instead of truncating by @henryiii in #4097
  • fix: typetracer correctness issues affecting dask-awkward by @henryiii in #4100
  • fix: BitMaskedArray mask inversion and placeholder detection by @henryiii in #4092
  • fix: Arrow conversion without pandas, empty chunked arrays, and multi-file Parquet attrs by @henryiii in #4099
  • fix: BitMasked builder use-after-free and GrowableBuffer copy bugs by @henryiii in #4098

Other

  • ci: fix deploy job by @ariostas in #4082
  • chore: add an AGENTS file by @henryiii in #4086
  • chore: don't refuse to install awkward_cpp when using uv by @henryiii in #4087
  • chore(deps): bump the actions group with 3 updates by @dependabot[bot] in #4121
  • docs: add albert-physics as a contributor for code by @allcontributors[bot] in #4122
  • ci: automate release-note categorization with type labels by @TaiSakuma in #4125
  • docs: document kernel spec invariants and CPU/CUDA equivalence in AGENTS by @henryiii in #4088
  • ci: run property-based tests nightly with a large max_examples by @TaiSakuma in #4128
  • test: add a property-based kernel test with CPU and GPU backends by @TaiSakuma in #4142
  • ci: show Hypothesis statistics in the nightly property tests by @TaiSakuma in #4146
  • chore: modernize header-only code to C++17 and remove dead code by @henryiii in #4104
  • ci: run the GPU kernel property tests nightly at the nightly profile by @TaiSakuma in #4147
  • ci: enable pyodide build by @ryanking13 in #4149
  • ci: fix docs-preview PR comment failing without a checkout by @TaiSakuma in #4155
  • chore(deps): bump the actions group across 1 directory with 3 updates by @dependabot[bot] in #4153
  • docs: describe the returned object in Returns: for already-merged operations by @TaiSakuma in #4148
  • chore(deps): update python version for mypy by @maxymnaumchyk in #4172

New Contributors

Full Changelog: v2.9.1...v2.10.0

Version 2.9.1

Choose a tag to compare

@TaiSakuma TaiSakuma released this 08 Jun 13:08

New features

  • feat: improve and test big-endian support by @ikrommyd in #3676
  • feat: implement array compatibility/interface protocols for virtual array and placeholder array (for better errors) by @ikrommyd in #3839
  • feat: reimplement reducers kernels using cccl by @maxymnaumchyk in #3840
  • feat: implement byteswap as an nplike function by @aashirvad08 in #3845
  • feat: lazy parents allocation by @ianna in #3860
  • feat: extend Python reducers with offset support (precursor to parent removal) by @ianna in #3865
  • feat: pass 'offsets' to reducer kernels (precursor to parent removal) by @maxymnaumchyk in #3883
  • feat: reducer max for axis==None by @ianna in #3884
  • feat: improve and fix delayed length calculations and add delayed length calculation for layouts that need it by @ikrommyd in #3889
  • feat: pickle in native byteorder and pass byteorder through pickling and unpickling by @ikrommyd in #3893
  • feat: customly overload the reducers that call cuda-compute kernels by @maxymnaumchyk in #3894
  • feat: add axis=None specialization for Min reducer + small refactoring for existing specializatons by @ikrommyd in #3896
  • feat: cuda.compute combinations kernel study by @ianna in #3908
  • feat: add missing_repeat kernel implementation using cuda.compute by @maxymnaumchyk in #3922
  • feat: add index_rpad_and_clip kernels implementations using cuda.compute by @maxymnaumchyk in #3923
  • feat: allow generator expressions in highlevel operations that accept iterable of arrays as input by @ikrommyd in #3938
  • feat: add awkward.* aliases to objects.inv for intersphinx by @TaiSakuma in #3958
  • feat: properly handle numpy functions that return default containers of numpy arrays in __array_function__ by @ikrommyd in #3991
  • feat: lazy concatenation of virtual arrays by @ikrommyd in #4010
  • feat: migrate awkward_index_rpad_and_clip_axis0 to cuda.compute by @danielfrg in #4012
  • feat: migrate awkward_reduce_sum_complex to cuda.compute by @danielfrg in #4016
  • feat: migrate awkward_reduce_max_complex to cuda.compute by @danielfrg in #4018

Bug-fixes and performance

  • fix: temporarily use unary_transform instead of segmented_reduce by @maxymnaumchyk in #3814
  • fix: C++ errors in ak.from_rdataframe by @bendavid in #3851
  • fix: Fix multiple CUDA out-of-bounds accesses and kernel memory management issues by @ianna in #3858
  • fix: harden CUDA/CPU kernels against out-of-bounds writes in ListOffsetArray by @ianna in #3859
  • fix: add offsets for UnmaskedArray reducer by @ianna in #3874
  • fix: GrowableBuffer cleanup compiler warnings by @ianna in #3881
  • fix: fix typetracer's maximum implementation by @ikrommyd in #3895
  • fix: fix ASAN detected buffer overflows in awkward-cpp by @ikrommyd in #3898
  • fix: Numba-CUDA refactoring -- CUDATargetContext is spinned off to FunctionDescriptor by @ianna in #3902
  • fix: return shallow copy of fields list when .fields is being accessed from highlevel array/record by @ikrommyd in #3906
  • fix: handle union array simplification returning back indexed arrays in ak.almost_equal by @ikrommyd in #3909
  • fix: ak.array_equal with datetimes and timedeltas with NaTs by @ikrommyd in #3921
  • fix: consistent byteorder settings by @ikrommyd in #3954
  • fix: couple of fixes in tests that arose from running them in parallel threads by @ikrommyd in #3961
  • fix: proper comparison in ak.almost_equal between record array fields by @ikrommyd in #3962
  • fix: restore docstring cross-references lost in autoapi migration by @TaiSakuma in #3972
  • fix: race condition in record array CUDA kernel by @ianna in #3982
  • fix: ensure jax backend uses arrays on cpu only by @ikrommyd in #3990
  • fix: array attrs not being validated at creation and being of inconsistent type by @ikrommyd in #3996
  • perf: optimize CPU kernels and fix segfault when concatenating arrays with None values by @ianna in #3875
  • perf: restructure CUDA unit tests by @ianna in #3935

Other

  • chore: ignore jax deprecation warning in pytest by @ikrommyd in #3864
  • chore: Update Python version from 3.10 to 3.14 in ROOT workflow by @ianna in #3876
  • chore: pin ROOT to 6.36.06 in workflow by @ianna in #3886
  • chore: Increase coverage target from 20% to 98% by @ianna in #3897
  • chore: update pre-commit hooks by @pre-commit-ci[bot] in #3900
  • chore: delete nvidia channel from GPU ci as it is outdated by @ikrommyd in #3903
  • chore: upgrade hypothesis-awkward to >=0.10 by @TaiSakuma in #3936
  • chore: pin hypothesis-awkward to exact version by @TaiSakuma in #3942
  • chore: update pre-commit hooks by @pre-commit-ci[bot] in #3951
  • chore: prepare for numpy 2.5 compatibility by @ikrommyd in #3976
  • chore: update pre-commit hooks by @pre-commit-ci[bot] in #3993
  • chore: Use kw-only arguments in all cuda.compute invocations by @shwina in #3994
  • chore: bump hypothesis-awkward from 0.16.0 to 0.18.0 by @TaiSakuma in #3995
  • chore: update pre-commit hooks by @pre-commit-ci[bot] in #4076
  • chore(ci): harden ci config by @ariostas in #4075
  • chore(deps): bump aws-actions/configure-aws-credentials from 5 to 6 in the actions group by @dependabot[bot] in #3857
  • chore(deps): bump the actions group with 3 updates by @dependabot[bot] in #3899
  • chore(deps): bump pypa/cibuildwheel from 3.3 to 3.4 in the actions group by @dependabot[bot] in #3905
  • chore(deps): bump the actions group across 1 directory with 3 updates by @dependabot[bot] in #3940
  • chore(deps): bump hypothesis-awkward from 0.10.0 to 0.12.0 by @dependabot[bot] in #3945
  • chore(deps): bump hypothesis-awkward from 0.12.0 to 0.13.0 by @dependabot[bot] in #3952
  • chore(deps): bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.0 in the actions group by @dependabot[bot] in #3963
  • chore(deps): bump hypothesis-awkward from 0.13.0 to 0.14.0 by @dependabot[bot] in #3968
  • chore(deps): bump hypothesis-awkward from 0.14.0 to 0.15.0 by @dependabot[bot] in #3974
  • chore(deps): bump the actions group with 2 updates by @dependabot[bot] in #3979
  • chore(deps): bump hypothesis-awkward from 0.15.0 to 0.16.0 by @dependabot[bot] in #3984
  • chore(deps): bump the actions group with 2 updates by @dependabot[bot] in #3987
  • chore(deps): bump aws-actions/configure-aws-credentials from 6.1.1 to 6.1.3 in the actions group by @dependabot[bot] in #4077
  • ci: windows ARM build by @henryiii in #3855
  • ci: remove xfail from tests that work with cupy 14, pin cupy >= 14 in CI, and split cuda tests with virtual arrays in a separate CI task by @ikrommyd in #3890
  • ci: use self-hosted CI for codecov by @ariostas in #3901
  • ci: fix issues with self-hosted CI by @ariostas in #3933
  • ci: add dependabot config for hypothesis-awkward by @TaiSakuma in #3944
  • ci: use reusable workflows by @ariostas in https:/...
Read more

Version 2.9.0

Choose a tag to compare

@ianna ianna released this 08 Feb 21:14

New features

Bug-fixes and performance

  • fix: pass the right input type to awkward_NumpyArray_subrange_equal* kernels by @ikrommyd in #3818
  • fix: define behavior class within the test function in test_1511_set_attribute.py to make it thread-local by @ikrommyd in #3826
  • fix: properly respect requested byteorder in ak.from_buffers for ndarray buffers by @ikrommyd in #3820
  • fix: use set_mask instead of set_base_mask in to_cudf by @ikrommyd in #3847

Other

  • chore: use "M8" dtype instead of "<M8" in tests by @ikrommyd in #3819
  • chore(deps): bump actions/attest-build-provenance from 3.1.0 to 3.2.0 in the actions group by @dependabot[bot] in #3828
  • chore: move array definitions inside test functions in test_1490_jax_reducers_combinations.py by @ikrommyd in #3829
  • chore: xfail tests-cuda/test_3051_to_cuda.py only in the latest cudf by @ikrommyd in #3836
  • chore: do not skip test_recordarray_argmax_y_field and make the xfail reason consistent for test_listarray_sort in tests-cuda/test_3459_virtualarray_with_cuda.py by @ikrommyd in #3837
  • chore: update pre-commit hooks by @pre-commit-ci[bot] in #3834
  • chore: delete old VirtualArray class by @ikrommyd in #3848
  • build: drop support for PyPy by @ikrommyd in #3835
  • ci: drop 3.13t by @henryiii in #3843
  • ci: drop cp/pp build selectors by @henryiii in #3854
  • docs: add automated contributions policy to CONTRIBUTING.md by @ianna in #3831
  • docs: document named-axis support for axis in high-level operations by @X0708a in #3823
  • refactor: improve ak.to_cudf errors and documentation by @ikrommyd in #3850

Full Changelog: v2.8.12...v2.9.0

Version 2.8.12

Choose a tag to compare

@ianna ianna released this 25 Jan 14:12
0ff58cf

New features

  • feat: improve merging of NumpyArrays and simplification of UnionArrays + fix ak.almost_equal for UnionArrays by @ikrommyd in #3773
  • feat: Add ak.sort() for CUDA backend by @shwina in #3750
  • feat: Make ak.combinations faster on GPU by using cp.searchsorted to compute output list indexes by @shwina in #3795
  • feat: add argmax implementation using CCCL by @maxymnaumchyk in #3777
  • feat: make ak.combinations faster on GPU by using cp.searchsorted to compute output list indexes by @ianna in #3798
  • feat: add argmin implementation using CCCL by @maxymnaumchyk in #3811

Bug-fixes and performance

  • fix: incorrect type passed down for the mergeastable argument by @ikrommyd in #3779
  • fix: fix ak.validity_error for typetracer-backed layouts with unknown length by @ikrommyd in #3784
  • fix: make ak.enforce_type work with typetracers by @ikrommyd in #3764
  • fix: ArrayBuilder.show() formatter forwarding by @X0708a in #3796
  • fix: fix and test Form.length_zero/one_array() by @ikrommyd in #3810
  • fix: make named axis thread-safe and start testing using multiple threads in CI by @ikrommyd in #3805

Other

  • ci: use codecov token by @ariostas in #3769
  • chore(ci): use macOS to test ML dependencies by @ariostas in #3774
  • chore(deps): bump the actions group across 1 directory with 5 updates by @dependabot[bot] in #3778
  • ci: xfail failing cudf tests until fixed by @ikrommyd in #3781
  • chore: update pre-commit hooks by @pre-commit-ci[bot] in #3785
  • chore: request for tensorflow in requirements only if python < 3.14 by @ikrommyd in #3787
  • chore: fix noxfile on Intel macOS by @henryiii in #3815
  • docs: add X0708a as a contributor for code by @allcontributors[bot] in #3797
  • docs: add a comparison of different argmax implementations (using unary_transform, segmented_reduce and cuda-kernels) by @maxymnaumchyk in #3800
  • docs: Document GPU backend and cuDF compatibility limitations by @X0708a in #3813

New Contributors

  • @X0708a made their first contribution in #3796

Full Changelog: v2.8.11...v2.8.12

Version 2.8.11

Choose a tag to compare

@ianna ianna released this 15 Dec 14:05
c4a2e57

New features

  • feat: add behavior cls validation hook by @pfackeldey in #3710
  • feat: add a playground for cccl studies by @pfackeldey in #3734
  • feat: add cuda.compute implementation to CCCL study by @shwina in #3748
  • feat: ability to optionally disable the caching of virtual arrays on themselves by @ikrommyd in #3741

Bug-fixes and performance

  • fix: string array numpy conversion fails with int32 offsets from parquet by @DylanModesitt in #3697
  • fix: only check length against empty form in ak.from_buffers if it is known by @ikrommyd in #3714
  • fix: typestr respects now limit_cols for .show and repr_mimebundle by @pfackeldey in #3712
  • fix: ArrayBuilder.__bool__ raises TypeError when builder length is 1 by @T90REAL in #3738
  • fix: prevent teardown from crashing GPU test suite by @ianna in #3747
  • fix: hide awkward.ak_* and awkward.str.akstr_* from 'awkward' and 'awkward.str' module by @pfackeldey in #3752
  • fix: align multi-level indices before merge in to_dataframe() by @TaiSakuma in #3733
  • fix: numba test failures on Python 3.14 by @ianna in #3757
  • fix: incorrect passing of Index inside nplike function in to_IndexedOptionArray64 by @ikrommyd in #3727
  • fix: weights should not be raised to the power n in the n-th moment calculation by @ikrommyd in #3729
  • fix: ak.Record creation from iterable with nested dicts by @ikrommyd in #3728
  • fix: keep track of buffer key instead of field path for VirtualNDArrays & PlaceholderArrays by @pfackeldey in #3751
  • fix: typetracer nplike's all function leads to infinite recursion for axis=None by @ikrommyd in #3765

Other

New Contributors

Full Changelog: v2.8.10...v2.8.11

Version 2.8.10

Choose a tag to compare

@ianna ianna released this 28 Oct 10:33
36824fd

New features

Bug-fixes and performance

  • fix: update image cell in thinking-in-arrays.ipynb by @ianna in #3660
  • fix: prefix with an underscore unused variables by @ianna in #3672
  • fix(ci): fix issues with dependencies for GPU tests by @ariostas in #3673
  • fix: convert integer-like objects to python integers in shapes by @ikrommyd in #3662
  • fix: avoid recursion limit when concatenating EmptyArrays by @ikrommyd in #3664
  • fix: only check VirtualNDArray's shape versus count in from_buffers if both are known by @ikrommyd in #3667
  • fix: preserve the type when indexed by an empty array by @TaiSakuma in #3678
  • fix: ensure masked string/bytestring dtypes can hold 'nan' fill value… by @JamesBrofos in #3692
  • fix: add missing units to datetime/timedelta in a test by @TaiSakuma in #3688
  • fix: pass attrs parameter to wrap_layout in from_* by @DylanModesitt in #3687
  • fix: use jax's __setitem__ syntax in ak.merge_union_of_records to make it work with the jax backend by @ikrommyd in #3596

Other

New Contributors

Full Changelog: v2.8.9...v2.8.10

Version 2.8.9

Choose a tag to compare

@ianna ianna released this 20 Sep 16:02

New features

  • feat: cuda reducers - add combinations for n>2, argmin and argmax, and fix min. by @ianna in #3645
  • feat: add axis=None reducer specializations by @pfackeldey in #3653
  • feat: add 'how=' to ak.unzip by @pfackeldey in #3655

Bug-fixes and performance

  • fix: virtual array deep copy generator reference + better error message for assert_never by @ikrommyd in #3644
  • fix: SVG bug in diagram depicting ragged aggregation by @jpivarski in #3648
  • fix: json signedness fixes by @ianna in #3651
  • fix: cartoon combinations by @ianna in #3654
  • perf: detect & fastpath no-op range slices in all layout types by @pfackeldey in #3642

Other

Full Changelog: v2.8.8...v2.8.9

Version 2.8.8

Choose a tag to compare

@ianna ianna released this 06 Sep 14:15

New features

  • feat: adapt __array__ according to numpy guidelines by @ikrommyd in #3592
  • feat: ensure ability to pickle ak.Array and ak.Record with VirtualArray buffers by @ikrommyd in #3612
  • feat: skip materialization of placeholders in to_packed by @ikrommyd in #3637

Bug-fixes and performance

  • fix: catch off-by-one out of bound error by @agoose77 in #3621
  • fix: avoid unnecessary recursion if not needed by @pfackeldey in #3611
  • fix: VirtualArray's deep copy + accidental .data access by @ikrommyd in #3599
  • fix: allow arrays with Nones in single argument usage of ak.where by @ikrommyd in #3563
  • fix: properly handle VirtualArray in nplike.asarray by @ikrommyd in #3597
  • fix: do not materialize before custom pickling by @ikrommyd in #3633
  • fix: do not force LE data-order in form.py by @nileshpatra in #3629
  • fix: make to_packed work for typetracer backed IndexedOptionArray, BitMaskedArray, and UnionArray by @ikrommyd in #3608
  • perf: add slots to virtual array to reduce instance alloc size by @pfackeldey in #3613
  • perf: avoid method chaining for some shape_generators by @pfackeldey in #3615

Other

New Contributors

Full Changelog: v2.8.7...v2.8.8

Version 2.8.7

Choose a tag to compare

@ianna ianna released this 08 Aug 09:18
5ef8ad5

New features

(none!)

Bug-fixes and performance

(none!)

Other

  • refactor: add jax DeprecationWarning by @ianna in #3609

Full Changelog: v2.8.6...v2.8.7