I am trying to partition an athena table by a timestamp variable called 'forecast_datetime'
. To use this variable as a partition I have made it 'string'
, with following values:
'2007-12-10-11-00-00', '2007-12-10-12-00-00', '2007-12-10-13-00-00', '2007-12-10-14-00-00', ... etc.
I can't figure this out. Thanks.
Yes you can. But you should consider the most partition key suitable for your case (i mean, Does it really need to be that detailed?, that even seconds is included).
Because by using partition key, you can restrict the amount of data scanned by each query. Thus it can be improving performance and reducing cost.
Source: https://docs.aws.amazon.com/athena/latest/ug/partitions.html