amazon-web-servicesbotoboto3emramazon-emr

How to wait for a step completion in AWS EMR cluster using Boto3


Given a step id I want to wait for that AWS EMR step to finish. How can I achieve this? Is there a built-in function?

At the time of writing, the Boto3 Waiters for EMR allow to wait for Cluster Running and Cluster Termination events:

EMR Waiters


Solution

  • There is now a waiter available for step complete events. It was added in a recent boto3 version.

    http://boto3.readthedocs.io/en/latest/reference/services/emr.html#EMR.Waiter.StepComplete

    Example code:

    import boto3
    
    client = boto3.client("emr")
    waiter = client.get_waiter("step_complete")
    waiter.wait(
        ClusterId='the-cluster-id',
        StepId='the-step-id',
        WaiterConfig={
            "Delay": 30,
            "MaxAttempts": 10
        }
    )