pythonaws-glue

Why does Python Glue Job throw a KeyError for my Job parameter?


I have defined a simple Glue Job of type Python Shell:

import sys
from awsglue.utils import getResolvedOptions

args = getResolvedOptions(sys.argv, [
  'test-parameter'
])

value = args["test-parameter"]
print(f'value = {value}')

I am trying to pass the following Job parameter:

When I run this Glue Job, I get the following error:

KeyError: 'test-parameter'

How can I fix this problem?


Solution

  • It appears that getResolvedOptions translates hyphens to underscores in parameter names. I confirmed this by adding some logging:

    import sys
    from awsglue.utils import getResolvedOptions
    
    print(f'sys.argv = {sys.argv}')
    
    args = getResolvedOptions(sys.argv, [
      'test-parameter'
    ])
    
    print(f'args = {args}')
    
    value = args["test-parameter"]
    print(f'value = {value}')
    

    In the CloudWatch logs, I see:

    To resolve this issue, I changed the Job parameter to use underscores instead of hyphens. (Note you still need the two leading hyphens...so --test_parameter instead of --test-parameter.)