feat: add a dtype= accumulator to ak.sum (overflow-safe, copy-free integer sums)#4232
Open
ianna wants to merge 5 commits into
Open
feat: add a dtype= accumulator to ak.sum (overflow-safe, copy-free integer sums)#4232ianna wants to merge 5 commits into
dtype= accumulator to ak.sum (overflow-safe, copy-free integer sums)#4232ianna wants to merge 5 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
|
|
The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR4232 |
ianna
force-pushed
the
ianna/copy_free_integer_sums
branch
from
July 23, 2026 13:30
ba0b2d5 to
1a19ee4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Integer reductions in the descriptive statistics overflow because the sum accumulates at the input dtype in the segmented kernel. The Python-level fix (#4223) casts integer input to
float64up front, which allocates a full-size temporary — the copy raised against #3527. NumPy avoids this by casting inside the reduction (np.add.reduce(x, dtype=np.float64)); awkward couldn't, because it had nofloat64-accumulator / integer-input reduce kernel.This PR adds that primitive:
ak.sum(x, dtype=np.float64)sums integer/bool input directly intofloat64, matching NumPy and avoiding a promoted copy.Changes
awkward_reduce_sum_float64_{int,uint}{8,16,32,64}_64specializations inkernel-specification.ymland registered inawkward-cpp/src/cpu-kernels/awkward_reduce_sum.cpp. The kernel body is unchanged —awkward_reduce_sumalready casts per element viastatic_cast<OUT>, so afloat64accumulator over integer input is a pure specialization. Generatedkernels.hand the signature tables are regenerated from the spec.Sumreducer (_reducers.py):Sum(dtype=None). When set, the accumulator/output dtype flows into the segmented kernel selection and theaxis=Nonepath; the bool path casts its small result.dtype=added tosumon all four backends (array_module,cupy,typetracer, and the protocol innumpy_like), forwarding to NumPy/CuPysum(dtype=).ak.sum: newdtype=keyword + docstring.tests/test_4223_sum_dtype_accumulator.py.Scope / follow-ups
ak.mean/ak.varare intentionally not rewired here. Doing so would hard-couple the pure-Python package to the new kernel — integer mean/var over an axis would break against any releasedawkward-cpppredating it. They should adoptdtype=(and drop the fix: overflow-safe stats without needless copies #4223 promotion) in a follow-up, once these kernels ship in a releasedawkward-cpp.var/stdadditionally need a fused sum-of-squares reducer (Phase 2).Release ordering
Kernels live in
awkward-cpp; the Python wiring is inawkward. Theaxis=Nonepath works against anyawkward-cpp(it maps onto NumPy/CuPysum(dtype=)), but the segmentedaxis != Nonepath requires a rebuiltawkward-cpp— soawkward-cppmust be released with these kernels beforeawkwardrelies on them. The two segmented tests skip when the kernels are absent.Non-goals
ak.sum/ak.proddefault dtype semantics are unchanged (NumPy preserves integer dtype); thefloat64accumulator is opt-in. Complex/datetime unaffected.AI disclosure
Parts of this PR were developed with AI assistance (per CONTRIBUTING.md).