cuda.core: add graph definition node updates by Andy-Jost · Pull Request #2395 · NVIDIA/cuda-python

Andy-Jost · 2026-07-20T23:11:20Z

Summary

Add subclass-specific update() methods for mutable graph definition nodes. The new API supports kernel, memcpy, memset, child-graph, event-record, event-wait, and host-callback nodes while leaving inspection properties read-only.

Updates replace one complete CUDA parameter structure through cuGraphNodeSetParams. Optional keyword arguments preserve existing values where coherent, coupled values such as kernels and arguments are replaced together, and existing executable graphs continue using their previous parameters and retained resources. This feature requires CUDA driver and cuda.bindings versions 12.2 or newer.

Changes

Add a shared graph-definition node update path with version checks, zero-initialized generic parameters, and failure-aware attachment prepare/commit handling.
Add update() methods for KernelNode, MemcpyNode, MemsetNode, ChildGraphNode, EventRecordNode, EventWaitNode, and HostCallbackNode.
Make partial kernel, memcpy, and memset updates keyword-only; changing a kernel requires explicitly supplying its arguments, including args=() for a no-argument kernel.
Preserve or replace retained kernels, arguments, buffers, explicit pointer owners, events, callbacks, and user data in step with the corresponding CUDA parameter update.
Stage, rekey, and publish replacement child-graph hierarchy metadata while invalidating views backed by the old embedded clone.
Add extensible parameterized behavior/failure tests, focused child-hierarchy lifetime tests, public API documentation, and release notes.

Review guide

Review cuda_core/cuda/core/graph/_subclasses.pyx and _subclasses.pyi for the public update signatures and node-specific parameter construction.
Review _set_definition_node_params for the common CUDA 12.2 gate and attachment transaction ordering.
Review resource_handles.cpp/.hpp for staged child-hierarchy metadata replacement and shared clone/rekey primitives.
Review test_graph_node_update.py for old-versus-new executable behavior and failure preservation, then test_graph_definition_lifetime.py for child-view invalidation and attachment lifetime coverage.

Related Work

Closes cuda.core: add atomic update methods for graph definition nodes #2352.
Builds on the graph attachment ownership foundation from cuda.core: retain graph attachments with per-node user objects #2357.
Executable-graph node updates remain tracked by cuda.core: add append-only attachment ownership for executable graphs #2353.
Part of CUDA graph phase N - graph updates #1330.

copy-pr-bot · 2026-07-20T23:11:23Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Andy-Jost · 2026-07-20T23:11:39Z

/ok to test

github-actions · 2026-07-20T23:33:06Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-2395/
https://nvidia.github.io/cuda-python/pr-preview/pr-2395/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-2395/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-2395/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

mdboom

The general organization and approach looks fine to me.

One small suggestion (that I can't really evaluate because I don't deeply understand the use cases).

Andy-Jost · 2026-07-23T22:33:18Z

/ok to test

Andy-Jost · 2026-07-24T21:17:56Z

/ok to test f500f26

Andy-Jost · 2026-07-24T21:40:11Z

/ok to test

juenglin · 2026-07-27T20:39:06Z

+    new_graph = case.graph_def.instantiate()
+    case.assert_exec_uses(old_graph, case.original)
+    case.assert_exec_uses(new_graph, case.replacement)
+


Suggested change

assert case.original != case.replacement

assert old_graph != new_graph

juenglin · 2026-07-27T20:49:14Z

+    case = definition_update_case
+
+    assert case.invalid_update is not None
+    assert case.invalid_exception is not None
+    with pytest.raises(case.invalid_exception):
+        case.invalid_update()
+
+    case.assert_current(case.original)
+    graph = case.graph_def.instantiate()
+    case.assert_exec_uses(graph, case.original)


Suggested change

case = definition_update_case

assert case.invalid_update is not None

assert case.invalid_exception is not None

with pytest.raises(case.invalid_exception):

case.invalid_update()

case.assert_current(case.original)

graph = case.graph_def.instantiate()

case.assert_exec_uses(graph, case.original)

case = definition_update_case

old_graph = case.graph_def.instantiate()

assert case.invalid_update is not None

assert case.invalid_exception is not None

with pytest.raises(case.invalid_exception):

case.invalid_update()

case.assert_current(case.original)

new_graph = case.graph_def.instantiate()

case.assert_exec_uses(new_graph, case.original)

assert case.original != case.replacement

assert old_graph == new_graph

I don't think old_graph == new_graph is correct here? Because they are separate instances.

Ok, so there is no equality comparison for graph instances?

There should be an equality based on the underlying CUgraphExec handle, but not a structural equality.

juenglin · 2026-07-27T21:33:01Z

+        assert_exec_uses=assert_exec_uses,
+        invalid_update=invalid_update,
+        invalid_exception=ValueError,
+        invalid_argument_update=None,


Suggested change

invalid_argument_update=None,

invalid_argument_update=lambda: None,

juenglin · 2026-07-27T21:33:54Z

+def _host_callback_ctypes_case(device):
+    """Use ctypes callbacks and copied payloads to distinguish each exec."""
+    callback_type = ctypes.CFUNCTYPE(None, ctypes.c_void_p)
+    called = []


Suggested change

called = []

invalid_callback_type = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_void_p)

called = []

juenglin · 2026-07-27T21:34:07Z

+    @callback_type
+    def replacement_fn(data):
+        called.append((replacement_fn, read_byte(data)))
+


Suggested change

@invalid_callback_type

def invalid_fn(data):

return 0

juenglin · 2026-07-27T21:34:53Z

+        assert called == [(expected[0], expected[1][0])]
+
+    def invalid_update():
+        node.update(lambda: None, user_data=b"not valid for a Python callback")


Suggested change

node.update(lambda: None, user_data=b"not valid for a Python callback")

node.update(invalid_fn, user_data=b"not valid for a Python callback")

juenglin · 2026-07-27T21:54:48Z

+            unexpected_dst = replacement_dst if expected["dst"] is original_dst else original_dst
+            assert list(as_bytes(unexpected_dst)) == [0] * 4
+
+    def cleanup():


I am surprised to see all this explicit cleanup. Is it necessary and cuda.python applications need to do this as well?

You're right, it should not be needed. This was added to work around a teardown error (below). I will continue working on that.

Warning: mr.deallocate() failed during Buffer destruction: CUDA_ERROR_INVALID_CONTEXT: This most frequently indicates that there is no context bound to the current thread. This can also be returned if the context passed to an API call is not a valid handle (such as a context that has had cuCtxDestroy() invoked on it). This can also be returned if a user mixes different API versions (i.e. 3010 context with 3020 API calls). See cuCtxGetApiVersion() for more details. This can also be returned if the green context passed to an API call was not converted to a CUcontext using cuCtxFromGreenCtx API.

juenglin · 2026-07-27T22:11:15Z

+
+    def invalid_update():
+        if replace == "kernel":
+            node.update(kernel=replacement_kernel)


Shouldn't we also test updating a kernel with wrong args?
For this case, empty_kernel does not take any args, so I would expect updating with kernel=empty_kernel, args=original_args to also fail.

cuda.core does not currently validate kernel arguments for ordinary launches either. cuKernelGetParamInfo, which enables count and size validation, requires CUDA 12.4. This should be addressed consistently across launch, graph construction, and update paths as separate work rather than only testing updates here.

juenglin · 2026-07-27T23:43:50Z

+        if c_dst_type == cydriver.CU_MEMORYTYPE_HOST:
+            c_dst = <cydriver.CUdeviceptr><uintptr_t>(
+                params.memcpy.copyParams.dstHost)
+        elif c_dst_type != cydriver.CU_MEMORYTYPE_ARRAY:


Would be safer to just make this else (same on line 824 below). Doesn't make a difference at the moment because of line 810. Otherwise follow the elif ...: with

else: raise NotImplementedError("implement me")

juenglin · 2026-07-28T15:18:55Z

+        cdef cydriver.CUdeviceptr c_dst
+        cdef unsigned int c_value
+        cdef unsigned int c_element_size
+        cdef size_t c_width
+        cdef size_t c_height
+        cdef size_t c_pitch
+        cdef OpaqueHandle dst_attachment_owner
+        cdef GraphHandle h_graph
+        cdef cydriver.CUgraphNode node = as_cu(self._h_node)
+        cdef cydriver.CUcontext ctx = NULL
+        cdef cydriver.CUDA_MEMSET_NODE_PARAMS current
+        cdef cydriver.CUgraphNodeParams params
+        cdef object queried
+
+        if dst is None and dst_owner is not None:
+            raise ValueError("dst_owner requires dst")
+        if (dst is None and value is None and width is None and
+                height is None and pitch is None):
+            return
+
+        c_memset(&params, 0, sizeof(params))
+        params.type = cydriver.CU_GRAPH_NODE_TYPE_MEMSET
+        with nogil:
+            HANDLE_RETURN(cydriver.cuGraphMemsetNodeGetParams(
+                node, &current))
+        if _check_node_get_params():
+            queried = handle_return(driver.cuGraphNodeGetParams(
+                <uintptr_t>node))
+            ctx = <cydriver.CUcontext><uintptr_t>int(queried.memset.ctx)
+        else:
+            with nogil:
+                HANDLE_RETURN(cydriver.cuCtxGetCurrent(&ctx))
+        params.memset.dst = current.dst
+        params.memset.value = current.value
+        params.memset.elementSize = current.elementSize
+        params.memset.width = current.width
+        params.memset.height = current.height
+        params.memset.pitch = current.pitch
+        params.memset.ctx = ctx
+
+        c_dst = params.memset.dst
+        c_value = params.memset.value
+        c_element_size = params.memset.elementSize
+        c_width = params.memset.width
+        c_height = params.memset.height
+        c_pitch = params.memset.pitch
+
+        if dst is None:
+            h_graph = graph_node_get_graph(self._h_node)
+            HANDLE_RETURN(graph_get_attachment(
+                h_graph, node,
+                &dst_attachment_owner, NULL))
+        else:
+            dst_attachment_owner = _resolve_memcpy_operand(
+                dst, dst_owner, "dst", &c_dst)
+
+        if value is not None:
+            c_value, c_element_size = _parse_fill_value(value)
+        if width is not None:
+            c_width = width
+        if height is not None:
+            c_height = height
+        if pitch is not None:
+            c_pitch = pitch
+
+        params.memset.dst = c_dst
+        params.memset.value = c_value
+        params.memset.elementSize = c_element_size
+        params.memset.width = c_width
+        params.memset.height = c_height
+        params.memset.pitch = c_pitch
+


Suggested change

cdef cydriver.CUdeviceptr c_dst

cdef unsigned int c_value

cdef unsigned int c_element_size

cdef size_t c_width

cdef size_t c_height

cdef size_t c_pitch

cdef OpaqueHandle dst_attachment_owner

cdef GraphHandle h_graph

cdef cydriver.CUgraphNode node = as_cu(self._h_node)

cdef cydriver.CUcontext ctx = NULL

cdef cydriver.CUDA_MEMSET_NODE_PARAMS current

cdef cydriver.CUgraphNodeParams params

cdef object queried

if dst is None and dst_owner is not None:

raise ValueError("dst_owner requires dst")

if (dst is None and value is None and width is None and

height is None and pitch is None):

return

c_memset(&params, 0, sizeof(params))

params.type = cydriver.CU_GRAPH_NODE_TYPE_MEMSET

with nogil:

HANDLE_RETURN(cydriver.cuGraphMemsetNodeGetParams(

node, &current))

if _check_node_get_params():

queried = handle_return(driver.cuGraphNodeGetParams(

<uintptr_t>node))

ctx = <cydriver.CUcontext><uintptr_t>int(queried.memset.ctx)

else:

with nogil:

HANDLE_RETURN(cydriver.cuCtxGetCurrent(&ctx))

params.memset.dst = current.dst

params.memset.value = current.value

params.memset.elementSize = current.elementSize

params.memset.width = current.width

params.memset.height = current.height

params.memset.pitch = current.pitch

params.memset.ctx = ctx

c_dst = params.memset.dst

c_value = params.memset.value

c_element_size = params.memset.elementSize

c_width = params.memset.width

c_height = params.memset.height

c_pitch = params.memset.pitch

if dst is None:

h_graph = graph_node_get_graph(self._h_node)

HANDLE_RETURN(graph_get_attachment(

h_graph, node,

&dst_attachment_owner, NULL))

else:

dst_attachment_owner = _resolve_memcpy_operand(

dst, dst_owner, "dst", &c_dst)

if value is not None:

c_value, c_element_size = _parse_fill_value(value)

if width is not None:

c_width = width

if height is not None:

c_height = height

if pitch is not None:

c_pitch = pitch

params.memset.dst = c_dst

params.memset.value = c_value

params.memset.elementSize = c_element_size

params.memset.width = c_width

params.memset.height = c_height

params.memset.pitch = c_pitch

cdef OpaqueHandle dst_attachment_owner

cdef GraphHandle h_graph

cdef cydriver.CUgraphNode node = as_cu(self._h_node)

cdef cydriver.CUcontext ctx = NULL

cdef cydriver.CUDA_MEMSET_NODE_PARAMS current

cdef cydriver.CUgraphNodeParams params

cdef object queried

if dst is None and dst_owner is not None:

raise ValueError("dst_owner requires dst")

if (dst is None and value is None and width is None and

height is None and pitch is None):

return

c_memset(&params, 0, sizeof(params))

params.type = cydriver.CU_GRAPH_NODE_TYPE_MEMSET

with nogil:

HANDLE_RETURN(cydriver.cuGraphMemsetNodeGetParams(

node, &current))

if _check_node_get_params():

queried = handle_return(driver.cuGraphNodeGetParams(

<uintptr_t>node))

ctx = <cydriver.CUcontext><uintptr_t>int(queried.memset.ctx)

else:

with nogil:

HANDLE_RETURN(cydriver.cuCtxGetCurrent(&ctx))

cdef cydriver.CUdeviceptr c_dst = current.dst

cdef unsigned int c_value = current.value

cdef unsigned int c_element_size = current.elementSize

cdef size_t c_width = current.width

cdef size_t c_height = current.height

cdef size_t c_pitch = current.pitch

if dst is None:

h_graph = graph_node_get_graph(self._h_node)

HANDLE_RETURN(graph_get_attachment(

h_graph, node,

&dst_attachment_owner, NULL))

else:

dst_attachment_owner = _resolve_memcpy_operand(

dst, dst_owner, "dst", &c_dst)

if value is not None:

c_value, c_element_size = _parse_fill_value(value)

if width is not None:

c_width = width

if height is not None:

c_height = height

if pitch is not None:

c_pitch = pitch

params.memset.dst = c_dst

params.memset.value = c_value

params.memset.elementSize = c_element_size

params.memset.width = c_width

params.memset.height = c_height

params.memset.pitch = c_pitch

params.memset.ctx = ctx

juenglin · 2026-07-28T15:30:49Z

+            commit_status = graph_commit_child_graph_update(
+                prepared, &h_replacement)
+        finally:
+            if h_replacement:


Suggested change

if h_replacement:

if commit_status == CUDATA_SUCCESS:

graph_commit_child_graph_update returns CUDA errors through the status, but C++ errors are reported by throwing, so commit_status is not correct here.

Hm, the h_replacement handle is not initialized on line 919. Why does this work, i.e., why would the if h_replacement: path not always be taken?

The function throws when a CUDA error occurred here, btw.

Use the generic node setter with failure-atomic attachment replacement, establishing the shared path for definition-level parameter mutation.

Extend definition-level mutation to event waits and both Python and ctypes host callbacks while preserving old executable state and attachment ownership.

Report unsupported driver or binding versions before preparing mutation attachments or calling the generic node setter.

Allow partial memset parameter replacement while preserving graph-owned destination lifetimes and previously instantiated graph behavior.

Support partial copy parameter replacement while preserving independent source and destination ownership across graph instantiations.

Support independent launch configuration and argument replacement while requiring explicit arguments when changing kernels.

Replace embedded child hierarchies while preserving attachment metadata, invalidating stale views, and keeping existing executables independent.

Describe supported mutation methods, CUDA 12.2 requirements, and executable graph behavior in the API and release notes.

Use type-erased shared ownership for prepared child updates so extension loading does not depend on a hidden C++ deleter symbol.

Reflect shared ownership for the opaque child update transaction in the generated stub.

Make memcpy and memset mutation calls explicit and unambiguous before the public API freezes.

Preserve memory-node contexts, reject unsupported node forms, and fail clearly when child graph metadata cannot be updated.

Resolve the CUDA 13.2 graph parameter getter dynamically so CUDA 12 binding builds remain compilable.

Clarify parameter handling and cover host/device memory transitions while exposing context-sensitive test teardown for follow-up.

Cover the public invalid-node state to ensure parameter updates fail cleanly without restoring graph membership.

Andy-Jost · 2026-07-28T20:32:04Z

/ok to test

Andy-Jost added this to the cuda.core 1.2.0 milestone Jul 20, 2026

Andy-Jost added P0 High priority - Must do! feature New feature or request cuda.core Everything related to the cuda.core module labels Jul 20, 2026

Andy-Jost self-assigned this Jul 20, 2026

Andy-Jost force-pushed the graph-definition-node-updates branch from 4ce2862 to fffa9fd Compare July 22, 2026 23:04

mdboom reviewed Jul 23, 2026

View reviewed changes

Comment thread cuda_core/cuda/core/graph/_subclasses.pyx

Andy-Jost force-pushed the graph-definition-node-updates branch 2 times, most recently from 6bafe1a to 84bcfab Compare July 23, 2026 22:22

This was referenced Jul 24, 2026

cuda.core: support multidimensional and array-backed graph memcpy nodes #2420

Open

cuda.core: support clustered and cooperative kernel graph nodes #2421

Open

Andy-Jost force-pushed the graph-definition-node-updates branch from 84bcfab to f500f26 Compare July 24, 2026 21:08

Andy-Jost requested a review from juenglin July 27, 2026 16:37

juenglin reviewed Jul 27, 2026

View reviewed changes

Comment thread cuda_core/tests/graph/test_graph_node_update.py

juenglin reviewed Jul 28, 2026

View reviewed changes

Andy-Jost added 6 commits July 28, 2026 13:27

feat(cuda.core): add event record node updates

b29df83

Use the generic node setter with failure-atomic attachment replacement, establishing the shared path for definition-level parameter mutation.

test(cuda.core): remove unused graph update import

27ba915

feat(cuda.core): add event wait and host node updates

4f40e14

Extend definition-level mutation to event waits and both Python and ctypes host callbacks while preserving old executable state and attachment ownership.

feat(cuda.core): require CUDA 12.2 for node updates

bfb97aa

Report unsupported driver or binding versions before preparing mutation attachments or calling the generic node setter.

feat(cuda.core): add memset node updates

9add55a

Allow partial memset parameter replacement while preserving graph-owned destination lifetimes and previously instantiated graph behavior.

feat(cuda.core): add memcpy node updates

cd165c8

Support partial copy parameter replacement while preserving independent source and destination ownership across graph instantiations.

Andy-Jost added 10 commits July 28, 2026 13:27

feat(cuda.core): add kernel node updates

4f0638f

Support independent launch configuration and argument replacement while requiring explicit arguments when changing kernels.

feat(cuda.core): add child graph node updates

a21942c

Replace embedded child hierarchies while preserving attachment metadata, invalidating stale views, and keeping existing executables independent.

docs(cuda.core): document graph node updates

e407503

Describe supported mutation methods, CUDA 12.2 requirements, and executable graph behavior in the API and release notes.

fix(cuda.core): avoid cross-extension deleter symbol

ccab117

Use type-erased shared ownership for prepared child updates so extension loading does not depend on a hidden C++ deleter symbol.

fix(cuda.core): align prepared child update stub

d6885b5

Reflect shared ownership for the opaque child update transaction in the generated stub.

api(cuda.core): make partial node updates keyword-only

cc54c9b

Make memcpy and memset mutation calls explicit and unambiguous before the public API freezes.

fix(cuda.core): harden graph node updates

951d088

Preserve memory-node contexts, reject unsupported node forms, and fail clearly when child graph metadata cannot be updated.

fix(cuda.core): support older bindings in node updates

61ed8fc

Resolve the CUDA 13.2 graph parameter getter dynamically so CUDA 12 binding builds remain compilable.

fix(cuda.core): harden memory node updates

822636d

Clarify parameter handling and cover host/device memory transitions while exposing context-sensitive test teardown for follow-up.

test(cuda.core): reject updates to destroyed nodes

1b0fdbd

Cover the public invalid-node state to ensure parameter updates fail cleanly without restoring graph membership.

Andy-Jost force-pushed the graph-definition-node-updates branch from fe48941 to 1b0fdbd Compare July 28, 2026 20:29


	assert case.original != case.replacement
	assert old_graph != new_graph

	invalid_argument_update=None,
	invalid_argument_update=lambda: None,

	called = []
	invalid_callback_type = ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_void_p)
	called = []

+    @invalid_callback_type
+    def invalid_fn(data):
+        return 0

	node.update(lambda: None, user_data=b"not valid for a Python callback")
	node.update(invalid_fn, user_data=b"not valid for a Python callback")

Uh oh!

Conversation

Andy-Jost commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Review guide

Related Work

Uh oh!

copy-pr-bot Bot commented Jul 20, 2026

Uh oh!

Andy-Jost commented Jul 20, 2026

Uh oh!

github-actions Bot commented Jul 20, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

mdboom left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Andy-Jost commented Jul 23, 2026

Uh oh!

Andy-Jost commented Jul 24, 2026

Uh oh!

Andy-Jost commented Jul 24, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andy-Jost Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

juenglin Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juenglin Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andy-Jost commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Andy-Jost commented Jul 20, 2026 •

edited

Loading

Andy-Jost Jul 28, 2026 •

edited

Loading

juenglin Jul 27, 2026 •

edited

Loading

juenglin Jul 28, 2026 •

edited

Loading