Skip to content

feat: to/from TensorFlow Tensor#3292

Merged
jpivarski merged 3 commits into
scikit-hep:mainfrom
maxymnaumchyk:maxymnaumchyk/add-to-from-tensorflow-functions
Nov 7, 2024
Merged

feat: to/from TensorFlow Tensor#3292
jpivarski merged 3 commits into
scikit-hep:mainfrom
maxymnaumchyk:maxymnaumchyk/add-to-from-tensorflow-functions

Conversation

@maxymnaumchyk

Copy link
Copy Markdown
Collaborator

No description provided.

@maxymnaumchyk

Copy link
Copy Markdown
Collaborator Author

Note: as discussed here when converting a TensorFlow Tensor into Numpy array with tensor.numpy() it creates a copy since Numpy is mutable and TensorFlow Tensor is not.

When converting Awkward Array to TensorFlow (while on CPU), this function: tf.convert_to_tensor(numpy_array) also makes a copy. I suppose for the same reason.

@codecov

codecov Bot commented Nov 6, 2024

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 30.15873% with 44 lines in your changes missing coverage. Please review.

Project coverage is 82.06%. Comparing base (b749e49) to head (fbcf571).
Report is 182 commits behind head on main.

Files with missing lines Patch % Lines
src/awkward/operations/ak_to_tensorflow.py 28.12% 23 Missing ⚠️
src/awkward/operations/ak_from_tensorflow.py 27.58% 21 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/operations/__init__.py 100.00% <100.00%> (ø)
src/awkward/operations/ak_from_tensorflow.py 27.58% <27.58%> (ø)
src/awkward/operations/ak_to_tensorflow.py 28.12% <28.12%> (ø)

... and 158 files with indirect coverage changes

@maxymnaumchyk
maxymnaumchyk marked this pull request as ready for review November 7, 2024 11:51

@jpivarski jpivarski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks perfect! Thank you again! This looks complete, and if you agree, I'll merge it into main.

On copying data from TensorFlow to NumPy: it might not be_technically_ necessary, since NumPy arrays have flags and can be marked as not writable. Thus, a NumPy array could zero-copy view a TensorFlow tensor in RAM (not GPU memory), if TensorFlow has stable pointers. Some systems, such as Java, copy data to different locations to defragment their memory use, which causes simple pointers (static integers) to get out of date. If TensorFlow does that, then it has to copy when converting to NumPy, even with NumPy's "writable" flag.

But this is probably out of our hands, since we have to rely on what functions TensorFlow gives us. They might have stable pointers that could be viewed as a non-writable NumPy array and yet just don't give us a function to do that—all we could do is ask them to add such a thing to their library (which we won't, because it's not such a big deal).

@jpivarski

Copy link
Copy Markdown
Member

(If you agree and it's ready to be merged, write a comment, rather than an emoji, since I get emails for comments but not emojis. Thanks!)

@jpivarski jpivarski linked an issue Nov 7, 2024 that may be closed by this pull request
7 tasks
@jpivarski

Copy link
Copy Markdown
Member

It looks done to me. I'm going to merge it. Thanks!

@jpivarski
jpivarski merged commit fa7be40 into scikit-hep:main Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add interoperability between Awkward Array and ML libraries

2 participants