aws-glueaws-glue-connectionaws-glue3.0

Access runtime parameters in AWS Glue Studio


I have created a Glue job using Glue studio Visual Editor. This job is getting data from a JDBC data source (Clickhouse Database) using a query. Doing some business transformations and then storing the data back to the database table using the same JDBC connection. In the source I want to parametrized the sql query so that I can change the dates with each run. I have defined the job parameters in the job details. Can any one help me how to access these params. As per the business requirement I have to develop the job using visual editor for easy maintenance purpose later on.

Job Params


Solution

  • Unfortunately this is not possible by just using the UI currently.

    If you can edit the code, you can use the getResolvedOptions function.

    import sys
    from awsglue.utils import getResolvedOptions
    
    args = getResolvedOptions(sys.argv, ["enddate"])
    
    print(args["enddate"])