sqljupyter-notebookamazon-athenajupyterhub

How to query a data using DateTime format while the data stored in table is in Unix Timestamp?


on Amazon Athena, I have my table:

name server_time register
Rudy 1718251764 Yes
John 1718294964 Yes
Charles 1718035764 Yes

Instead of have to go unixtimestamp web to get the Unix Timestamp for my desired date, I want to filter the table using YYYY-MM-DD HH:MM in my query. is it actually possible to do so? if it's not, is it possible to do it on jupyter-notebook?

Honestly, I'm not sure which SQL is used on Amazon Athena. I tried :

  1. SQL Server
SELECT *
FROM my_table
where server_time between datediff(second,'1970-01-01','2024-06-12') and datediff(second,'1970-01-01','2024-06-13')

I got error Column 'second' cannot be resolved or requester is not authorized to access requested resources. Also tried datediff_big still not works.

  1. MySQL
SELECT *
FROM my_table
where server_time between unix_timestamp('2024-06-12') and unix_timestamp('2024-06-13')

I got error Function 'unix_timestamp' not registered

  1. SQLite
SELECT *
FROM my_table
where server_time between unixepoch('2024-06-12') and unixepoch('2024-06-13')

I got error Function 'unix_timestamp' not registered


Solution

  • You can use AWS Athena UI to to write your queries. The online editor has almost all the features that a modern tool has. There is no need to install any plugin. Navigate to Athena from AWS console. From the beginning of Athena, the query engine under the hood was Trino and you can go through trino documents to get syntax https://trino.io/docs/current/functions.html

    Coming to question you asked, you can use following query to convert your unix time to datetime in string format and then filter out

    with cte as (
    select 'Rudy' as name, 1718251764 as server_time, 'Yes' as register 
    union all 
    select 'John' as name, 1718294964 as server_time, 'Yes' as register 
    union all 
    select 'Charles' as name, 1718035764 as server_time, 'Yes' as register 
    )
    select *, format_datetime(from_unixtime(server_time), 'Y-MM-dd HH:MM') as date_str from cte
    where format_datetime(from_unixtime(server_time), 'Y-MM-dd HH:MM') <= '2024-06-13 05:05'
    

    yes you can use Jupyterlab to query athena but you need to install boto3 and configure AWS credentials and subsequently create instance of athena client and use appropriate functions to get query results and following link should be of help https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/start_query_execution.html