amazon-web-servicesamazon-emramazon-vpcamazon-data-pipeline

Data Pipeline & EMR error: No default VPC found. But I'm not authorized to create default VPC

I need to export a DynamoDB table to an S3 bucket.

I've created a Data Pipeline, but it's stuck in Waiting for runner status so I checked the runsOn value and it says "EmrClusterForBackup". Then I checked EMR and for the cluster df-02286063M3MSXPABCMOX_@EmrClusterForBackup I found an error saying: No default VPC found.

I took a look at my VPCs and I have 1 VPC available. I tried to create a Default VPC but I get an error message saying that I'm not authorized to perform this operation.

Now my question is what can I do instead to make my Data Pipeline work? Is it possible to let the EMR Cluster run on my available VPC or is the default VPC is mandatory?

Solution

  • The AWS documentation says the following:

    To launch a resource into a subnet of a nondefault VPC or a nondefault subnet of a default VPC, you must specify the subnet using the subnetId field when you configure the resource. If you have a default VPC and you don't specify subnetId, we launch the resource into the default subnet of the default VPC.

    Because you are not specifying it is trying to launch the EMR cluster in the default VPC.