I'm using the AWS Glue Dev Endpoint. It takes a while for the python virtualenv stuff to be uploaded. Is there a way that I can cache these files to tighten the development loop?
Any help is greatly appreciated.
I think the Dev Endpoint method of development has been superceded by using a Docker environment.
https://aws.amazon.com/blogs/big-data/developing-aws-glue-etl-jobs-locally-using-a-container/