fix: BitMasked builder use-after-free and GrowableBuffer copy bugs

[henryiii]

🤖 AI text below 🤖

This PR fixes several memory-safety and correctness bugs in the standalone header-only/ C++17 library (LayoutBuilder and GrowableBuffer), surfaced by an automated multi-agent code review. These changes are AI-assisted (Claude Code).

Fixes

1. BitMasked use-after-free (heap-use-after-free, confirmed with ASan).
   BitMasked stored a uint8_t& current_byte_ref_ bound to element 0 of the mask buffer's first panel. append_begin()/clear() did current_byte_ref_ = mask_.append_and_get_ref(...), which assigns through the reference rather than rebinding it, so it permanently aliased the original panel; after clear() freed that panel, every mask-bit write was a UAF, and completed mask bytes past the first landed in the wrong place. The reference member is removed: current_byte_/current_index_ are now plain state, and each completed byte is appended to mask_ when it fills (in natural order), with the trailing partial byte emitted at finalization via a new write_mask() helper. Touches the constructors, clear(), length(), buffer_nbytes(), to_buffers()/to_buffer()/to_char_buffers(), and append_begin()/append_end() (header-only/layout-builder/awkward/LayoutBuilder.h).

2. BitMasked<valid_when=false> wrong mask bytes. Inversion previously happened only through the broken reference; it now happens where each completed byte lands in the buffer (and for the trailing byte in write_mask()), fixing wrong bytes once more than 8 elements are appended (LayoutBuilder.h, append_end/write_mask).

3. GrowableBuffer multi-panel hole. Panel::concatenate_to_from recursed with offset + length_ instead of offset + (length_ - from), leaving a from-sized hole of uninitialised bytes when the source spanned multiple panels (hit by BitMasked masks larger than one panel) (header-only/growable-buffer/awkward/GrowableBuffer.h:135).

4. GrowableBuffer memcpy size mixed bytes and elements. length * sizeof(PRIMITIVE) - from is corrected to (length - from) * sizeof(PRIMITIVE) in Panel::append and Panel::concatenate_to_from (previously only correct for 1-byte types or from == 0) (GrowableBuffer.h:119, :133).

5. Indexed/IndexedOption zero-size extend underflow. Indexed::extend_index(size) / IndexedOption::extend_valid(size) computed stop - 1 in size_t; for size == 0 on empty content this underflowed to SIZE_MAX and poisoned max_index_/last_valid_, making is_valid() falsely fail. Both now guard size == 0. The dead else if (i < 0) branch (always false on a size_t) in append_index was removed (LayoutBuilder.h:1367, :1378, :1594).

Tests

Adds header-only/tests/test_4087-bitmasked-growablebuffer.cpp covering: builder reuse after clear(), BitMasked<valid_when=false>, more than 8 elements, a mask spanning more than one panel (20000 elements), zero-size extend_index/extend_valid, and a multi-panel GrowableBuffer concatenate with non-zero from and sizeof(PRIMITIVE) > 1. Built against the pre-fix headers, the new test reproduces the heap-use-after-free under AddressSanitizer; with the fixes it passes both normally and under ASan. Also adds enable_testing() to the top-level header-only CMakeLists.txt so ctest discovers the suite.

Notes

The header-only/ library is standalone. The generated copies under awkward-cpp/ and src/awkward/_connect/header-only are produced by dev/copy-cpp-headers.py at nox -s prepare time and are not tracked in git, so no generated copies are committed here.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.80%. Comparing base (e679c5f) to head (6128648).
✅ All tests successful. No failed tests found.

Additional details and impacted files

[TaiSakuma]

🤖 This comment was written by Claude Code.

Why this PR matters for the next release: CI on main is currently red, which blocks cutting the next release. The failing check is header-only-tests / Run Tests (macos-latest): test_1494-layout-builder dies with SIGSEGV. AddressSanitizer pins it to a heap-use-after-free in BitMasked::clear() — writing through the dangling current_byte_ref_ after mask_.clear() frees its slot. (Latent since 2022; a recent macos-latest toolchain bump exposed it.)

This PR fixes that. The rework removes the current_byte_ref_ reference member. Validated locally on macOS/arm64 (Apple clang 21) under ASan against this branch's headers:

• test_1494-layout-builder: passes, no ASan error (mask output 157 3 preserved)
• test_4087-bitmasked-growablebuffer: passes, no ASan error

However, this PR is itself red on the same header-only-tests job — for a different reason. It is not the segfault; it is a CMake configuration error from a typo in header-only/tests/CMakeLists.txt. The test_4087 target_link_libraries is missing its closing line, so it swallows the next command and the header-only suite fails to configure (before any test runs):

target_link_libraries(test_4087-bitmasked-growablebuffer
                      PRIVATE awkward::layout-builder)
target_link_libraries(test_4088-layoutbuilder-form-fixes
                      PRIVATE awkward::layout-builder)

Adding the PRIVATE awkward::layout-builder) line for test_4087 should let the suite configure and go green — after which this PR unblocks the release.

[TaiSakuma]

I made a PR on this PR's branch that implements the fix described in the comment above: #4171.
If this fixes the error, we can make the next release.

[github-actions]

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR4098

[ianna]

@henryiii and @TaiSakuma -- Thanks! It looks good. I'll enable auto-merge.

[TaiSakuma]

It looks promising. Header-only and pre-commit passed already.