QubolePartitionSensor
QuboleWait for a Hive partition to show up in QHS (Qubole Hive Service).
Access Instructions
Install the Qubole provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
qubole_conn_idConnection id which consists of qds auth_token
dataa JSON object containing payload, whose presence needs to be checked. Check this example for sample payload structure.
Documentation
Wait for a Hive partition to show up in QHS (Qubole Hive Service).
Check for Hive partition presence via QDS APIs.
See also
For more information on how to use this sensor, take a look at the guide: Partition sensor
Note
Both data and qubole_conn_id fields support templating. You can also use .txt files for template-driven use cases.