pythonamazon-web-servicespysparkaws-glue

awsglue.utils.GlueArgumentError: argument --JOB_NAME is required

When I run the Glue job boilerplate in AWS Glue using Python, I get an error

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

glueContext = GlueContext(SparkContext.getOrCreate())
spark = glueContext.spark_session

args = getResolvedOptions(sys.argv, ['JOB_NAME'])
job = Job(glue_context)
job.init(args[‘JOB_NAME’], args)

Here's the error

in getResolvedOptions
File "/usr/lib64/python2.7/argparse.py", line 1733, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "/usr/lib64/python2.7/argparse.py", line 1957, in _parse_known_args
self.error(_('argument %s is required') % name)
awsglue.utils.GlueArgumentError: argument --JOB_NAME is required

This follows my code and then 

job.commit()

But AWS says do not provide JOBNAME. What am I doing wrong? And also what does getResolved do? I tried reading AWS documentation but couldn't get anything.

Solution

  • You cannot run the following command in dev environment. 

    args = getResolvedOptions(sys.argv, ['JOB_NAME'])