QuboleFileSensor
QuboleWait for a file or folder to be present in cloud storage.
Access Instructions
Install the Qubole provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
qubole_conn_idConnection id which consists of qds auth_token
dataa JSON object containing payload, whose presence needs to be checked Check this example for sample payload structure.
Documentation
Wait for a file or folder to be present in cloud storage.
Check for file or folder presence via QDS APIs.
See also
For more information on how to use this sensor, take a look at the guide: File sensor
Note
Both data and qubole_conn_id fields support templating. You can also use .txt files for template-driven use cases.