I have defined a simple Glue Job of type Python Shell:
import sys
from awsglue.utils import getResolvedOptions
args = getResolvedOptions(sys.argv, [
'test-parameter'
])
value = args["test-parameter"]
print(f'value = {value}')
I am trying to pass the following Job parameter:
--test-parameter
hello world
When I run this Glue Job, I get the following error:
KeyError: 'test-parameter'
How can I fix this problem?
It appears that getResolvedOptions
translates hyphens to underscores in parameter names. I confirmed this by adding some logging:
import sys
from awsglue.utils import getResolvedOptions
print(f'sys.argv = {sys.argv}')
args = getResolvedOptions(sys.argv, [
'test-parameter'
])
print(f'args = {args}')
value = args["test-parameter"]
print(f'value = {value}')
In the CloudWatch logs, I see:
sys.argv = ['--test-parameter', 'hello world', ...]
args = {'test_parameter': 'hello world'}
To resolve this issue, I changed the Job parameter to use underscores instead of hyphens. (Note you still need the two leading hyphens...so --test_parameter
instead of --test-parameter
.)