GlueJobSensor
AmazonWaits for an AWS Glue Job to reach any of the status below ‘FAILED’, ‘STOPPED’, ‘SUCCEEDED’
Access Instructions
Install the Amazon provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
job_nameRequiredThe AWS Glue Job unique name
run_idRequiredThe AWS Glue current running job identifier
verboseIf True, more Glue Job Run logs show in the Airflow Task Logs. (default: False)
Documentation
Waits for an AWS Glue Job to reach any of the status below ‘FAILED’, ‘STOPPED’, ‘SUCCEEDED’
See also
For more information on how to use this sensor, take a look at the guide: Wait on an AWS Glue job state