I am trying to create a time series as below but running into ValueError: TimeSeriesOutput does not support setting output_format or options. I could not find any details or hints in the available documentation.
from transforms.api import transform, incremental, Input, Output
from transforms.timeseries import TimeSeriesOutput
@incremental(semantic_version=1)
@transform(
output_dataset= TimeSeriesOutput("/path/to/output/dataset"),
input_dataset=Input("/path/to/input/dataset"),
)
def my_compute_function(output_dataset, input_dataset):
output_dataset.write_dataframe(
input_dataset.dataframe('added').repartitionByRange('seriesId').sortWithinPartitions('seriesId', 'timestamp'),
output_format='soho', options={'noho': 'true'})
TimeSeriesOutput is part of the a transforms-addons library which provide a collection of utility methods to help with a variety of pipeline tasks. If you control click on the TimeSeriesOutput you should be able to see the logic that backs it. Which does include the code that generates error you are hitting here.
if output_format != 'soho' or options is not None:
raise ValueError('TimeSeriesOutput does not support setting output_format or options')
options = dict({'Noho': 'true'})
Your exception looks like a bug on the library, since noho is set to true within the same function. So a quick fix for you here is to just to set options=None