pythonamazon-s3aws-gluepy4j

Py4JException error while saving file to S3 with Glue

I'm trying to save default value file to S3 bucket but getting below error:

An error occurred while calling o654.__getstate__. Trace:
py4j.Py4JException: Method __getstate__([]) does not exist
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
    at py4j.Gateway.invoke(Gateway.java:274)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.lang.Thread.run(Thread.java:748)

Code snippet I prepared, it seems to be py4j releated issue. Thanks for suggestions!

    BUCKET = 'my_bucket'
    s3 = boto3.client('s3')
    keyid = 'xxxxx'

    print("Uploading S3 object with SSE-KMS")
    s3.put_object(Bucket=BUCKET,
              Key='encrypt-key',
              Body=b'foobar',
              ServerSideEncryption='aws:kms',
              # Optional: SSEKMSKeyId
              SSEKMSKeyId=keyid)
    print("Saving to S3, Done")
Solution

  • Potential issue was wrong syntax for Key. It should follow subfolder/subfolder/file_to_write.txt.

    / has to be added to each subfolder (aka key) name, otherwise AWS doesn't see it as a path in the bucket.