Is your feature request related to a problem? Please describe.
People often want data quality checks to modify the output, but that's really not what they're supposed to do.
@pipe applies a function, but it is run beforehand. So if you just want to change the output of a node, you have to do it in two steps.
Describe the solution you'd like
New decorator that functions like pipe, but runs afterwards:
@mutate(
step(_keep_range, range=(0,100)),
step(_dropna)
)
def data() -> pd.Series:
return ...
This would form the DAG: data.raw -> data.with_drop_between -> data
Describe alternatives you've considered
Making it more central to data quality or just integating with the node.
Additional context
Related to an OS question.
Is your feature request related to a problem? Please describe.
People often want data quality checks to modify the output, but that's really not what they're supposed to do.
@pipeapplies a function, but it is run beforehand. So if you just want to change the output of a node, you have to do it in two steps.Describe the solution you'd like
New decorator that functions like
pipe, but runs afterwards:This would form the DAG:
data.raw -> data.with_drop_between -> dataDescribe alternatives you've considered
Making it more central to data quality or just integating with the node.
Additional context
Related to an OS question.