NamedHivePartitionSensor

Apache Hive

Waits for a set of partitions to show up in Hive.

View on GitHub

Last Updated: Oct. 23, 2022

Access Instructions

Install the Apache Hive provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

partition_namesRequiredList of fully qualified names of the partitions to wait for. A fully qualified name is of the form schema.table/pk1=pv1/pk2=pv2, for example, default.users/ds=2016-01-01. This is passed as is to the metastore Thrift client get_partitions_by_name method. Note that you cannot use logical or comparison operators as in HivePartitionSensor.
metastore_conn_idReference to the metastore thrift service connection id.

Documentation

Waits for a set of partitions to show up in Hive.

Was this page helpful?