EmrStepSensor
AmazonAsks for the state of the step until it reaches any of the target states. If it fails the sensor errors, failing the task.
Access Instructions
Install the Amazon provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
job_flow_idRequiredjob_flow_id which contains the step check the state of
step_idRequiredstep to check the state of
target_statesthe target states, sensor waits until step reaches any of these states
failed_statesthe failure states, sensor fails when step reaches any of these states
Documentation
Asks for the state of the step until it reaches any of the target states. If it fails the sensor errors, failing the task.
With the default target states, sensor waits step to be completed.
See also
For more information on how to use this sensor, take a look at the guide: Wait on an Amazon EMR step state