DatabricksTrigger
Astronomer ProvidersWait asynchronously for databricks job to reach the terminal state.
Access Instructions
Install the Astronomer Providers provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
conn_idThe databricks connection id. The default value is databricks_default.
task_idThe task id.
run_idThe databricks job run id.
retry_limitAmount of times retry if the Databricks backend is unreachable. Its value must be greater than or equal to 1.
retry_delayNumber of seconds to wait between retries (it might be a floating point number).
polling_period_secondsControls the rate which we poll for the result of this run. By default, the operator will poll every 30 seconds.
job_idThe databricks job id.
run_page_urlThe databricks run page url.
Documentation
Wait asynchronously for databricks job to reach the terminal state.