Given a step id I want to wait for that AWS EMR step to finish. How can I achieve this? Is there a built-in function?
At the time of writing, the Boto3 Waiters for EMR allow to wait for Cluster Running and Cluster Termination events:
There is now a waiter available for step complete events. It was added in a recent boto3 version.
http://boto3.readthedocs.io/en/latest/reference/services/emr.html#EMR.Waiter.StepComplete
Example code:
import boto3
client = boto3.client("emr")
waiter = client.get_waiter("step_complete")
waiter.wait(
ClusterId='the-cluster-id',
StepId='the-step-id',
WaiterConfig={
"Delay": 30,
"MaxAttempts": 10
}
)