EmrServerlessJobSensor
AmazonAsks for the state of the job run until it reaches a failure state or success state. If the job run fails, the task will fail.
Access Instructions
Install the Amazon provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
application_idRequiredapplication_id to check the state of
job_run_idRequiredjob_run_id to check the state of
target_statesa set of states to wait for, defaults to ‘SUCCESS’
aws_conn_idaws connection to use, defaults to ‘aws_default’
Documentation
Asks for the state of the job run until it reaches a failure state or success state. If the job run fails, the task will fail.
See also
For more information on how to use this sensor, take a look at the guide: Wait on an EMR Serverless Job state