I am trying to create a emr-serverless application through the EmrServerlessCreateApplicationOperator but I keep facing the error botocore.exceptions.NoRegionError: You must specify a region.
I am passing the region like below:
create_app = EmrServerlessCreateApplicationOperator(
task_id="create_spark_app",
job_type="SPARK",
release_label="emr-6.6.0",
config={"aws_access_key_id":args["aws_access_key_id"],
"aws_secret_access_key": args["aws_secret_access_key"],
"aws_session_token": args["aws_session_token"],
"region_name":'us-east-1'}
)
Any help is appreciated. Thank you.
I was able to figure it out. I passed them as environment variables.
os.environ["AWS_ACCESS_KEY_ID"]= YOUR VALUE
os.environ["AWS_SECRET_ACCESS_KEY"]= YOUR VALUE
os.environ["AWS_SESSION_TOKEN"]= YOUR VALUE
os.environ["AWS_DEFAULT_REGION"]= YOUR VALUE
This worked.