Skip to content

notebook: invalid variable names for %%cell_to_module --execute #1184

Description

@zilto

If you're using %%cell_to_module my_module --execute it will try to push nodes back into the user namespace. This fails if the node includes a . in its name (internal nodes of @pipe_input, @pipe_output, @mutate, @load_from, etc.)

The following allows to only execute and push the specified nodes, and avoid this issue:

## cell 1
final_vars = ["foo", "bar"]

## cell 2
%%cell_to_module my_dataflow --execute final_vars
from hamilton.function_modifiers import pipe_input, step

def foo() -> dict:
  return {...}

def _invert_mapping(d: dict) -> dict:
   return {v: k for k, v in d.items()}

@pipe_input(step(_invert_mapping))
def bar(foo: dict) -> dict:
  return foo

However, this will when calling %%cell_to_module --execute without an argument because it tries to execute and push all nodes (including the generated bar.with_invert_mapping).

Proposed solution

Before pushing back results to the user namespace, check if the name includes a . and replace by __. The double underscore is a good option because Hamilton already ignores __ suffix when parsing node names. Using single underscore is likely to cause naming conflicts with other user-created nodes

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions