EmrServerlessStartJobOperator
AmazonOperator to start EMR Serverless job.
Access Instructions
Install the Amazon provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
application_idID of the EMR Serverless application to start.
execution_role_arnARN of role to perform action.
job_driverDriver that the job runs on.
configuration_overridesConfiguration specifications to override existing configurations.
client_request_tokenThe client idempotency token of the application to create. Its value must be unique for each request.
configOptional dictionary for arbitrary parameters to the boto API start_job_run call.
wait_for_completionIf true, waits for the job to start before returning. Defaults to True. If set to False, waiter_countdown and waiter_check_interval_seconds will only be applied when waiting for the application be to in the STARTED state.
aws_conn_idAWS connection to use.
nameName for the EMR Serverless job. If not provided, a default name will be assigned.
waiter_countdownTotal amount of time, in seconds, the operator will wait for the job finish. Defaults to 25 minutes.
waiter_check_interval_secondsNumber of seconds between polling the state of the job. Defaults to 60 seconds.
Documentation
Operator to start EMR Serverless job.
See also
For more information on how to use this operator, take a look at the guide: Start an EMR Serverless Job