I am attempting to write an @transform
that uses Input and Output and ctx. What I am trying to do, and there is an example of this in the documentation is to take 1 Input and have 2 Outputs for this transform.
from transforms.api import transform, Input, Output
@transform(
i1=Input('/examples/i1'),
o1=Output('/examples/o1'),
o2=Output('examples/o2'),
)
def split_data(ctx, i1, o1, o2):
transformed_df_1 = ...
transformed_df_2 = ...
ctx.write_dataframe(o1, transformed_df_1)
ctx.write_dataframe(o2, transformed_df_2)
What is happening is that I get 4 required positional arguments are missing. It doesn't seem to recognize that this split_data is a transform, it is like being treated as a function.
What is going on here?
I expect 2 output datasets at the end called o1 and o2.
The @transform
syntax is only valid in Code Repository, when writing Python transforms.
https://www.palantir.com/docs/foundry/code-repositories/create-transforms
You can read/write datasets in Code workspace (Jupyter CodeNotebook flavor) with another syntax See https://www.palantir.com/docs/foundry/code-workspaces/data/#tabular-datasets
from foundry.transforms import Dataset
kittens = Dataset.get("kittens")\
.read_table(format="arrow")\
.to_pandas()