fix: transient attrs should not be written to parquet

[ikrommyd]

#4079 introduced attrs serialization to parquet. However, transient attrs should not be serialized. Users are seeing errors writing root files loaded with coffea to disk. This fails with awkward 2.10.0

from coffea.nanoevents import NanoEventsFactory
import awkward as ak

file = "foo.root"
events = NanoEventsFactory.from_root(
file, treepath="Events").events()
ak.to_parquet(events.Jet.pt, "pt.parquet")

with

TypeError: Object of type NanoEventsFactory is not JSON serializable
when serializing dict item '@events_factory'

This error occurred while calling

    ak.to_parquet(
        <Array [[89.9, 41, ..., 16.6, 16], ...] type='146910 * var * float3...'>
        'pt.parquet'
    )

A patch release is needed ASAP as users cannot write to parquet in all analysis frameworks if they use awkward 2.10.0.

[ianna]

In general we almost certainly do not want to save transient attributes to Parquet.

[ariostas]

Thank you, @ikrommyd!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.97%. Comparing base (d24404d) to head (b356fd8).

Additional details and impacted files

Files with missing lines	Coverage Δ
src/awkward/operations/ak_from_parquet.py	94.01% <100.00%> (ø)
src/awkward/operations/ak_to_parquet.py	68.21% <100.00%> (+0.42%)	⬆️

[TaiSakuma]

Can we manually start the integration test when this is merged?

[ikrommyd]

The integration tests did not catch this. A user did

[TaiSakuma]

I see. Can we still run the integration test? I'll make a release from a commit that passes the nightly tests.

[ikrommyd]

Claude noticed that table.schema.metadata can be None so I fixed that too here just in case

[ianna]

@ikrommyd - great! All looks good. I'll enable auto-merge so that it makes in to a release asap.

[ikrommyd]

I plan to make a follow-up which has nothing to do with the fix to enable attrs writing when one uses to_parquet_row_groups too. In that case the attrs willk be picked up from the first array in the same way the schema is created from the first array

[ikrommyd]

I see. Can we still run the integration test? I'll make a release from a commit that passes the nightly tests.

@TaiSakuma I did not notice this and yes the answer is we can. A CI can always be started manually.

[TaiSakuma]

@ikrommyd - That is great. Can we run it when this is merged? Or should #4210 be released together?

[ikrommyd]

I mean I don't really care. The release can be tomorrow. #4210 is also ready and just needs a rebase after this one is merged. So if you want to put it in too that's cool. The bottleneck is the GPU CI here.

[github-actions]

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR4209

[ikrommyd]

https://github.com/scikit-hep/integration-tests/actions/runs/29772491297 @TaiSakuma so either review and merge the other PR or don't. I'm fine either way. It just fits adding it now as it deals with parquet attrs.

[TaiSakuma]

I confirmed this passed the nightly integration test and the two nightly property tests (all manually started).
This is in v2.11.0.