BigQueryIntervalCheckOperatorAsync

Astronomer ProvidersCertified

Checks asynchronously that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.

Access Instructions

Install the Astronomer Providers provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

tablethe table name
days_backnumber of days between ds and the ds we want to check against. Defaults to 7 days
metrics_thresholdsa dictionary of ratios indexed by metrics, for example ‘COUNT(*)’: 1.5 would require a 50 percent or less difference between the current day, and the prior days_back.
use_legacy_sqlWhether to use legacy SQL (true) or standard SQL (false).
gcp_conn_id(Optional) The connection ID used to connect to Google Cloud.
locationThe geographic location of the job. See details at: https://cloud.google.com/bigquery/docs/locations#specifying_your_location
impersonation_chainOptional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
labelsa dictionary containing labels for the table, passed to BigQuery

Documentation

Checks asynchronously that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.

param table

the table name

param days_back

number of days between ds and the ds we want to check against. Defaults to 7 days

param metrics_thresholds

a dictionary of ratios indexed by metrics, for example ‘COUNT(*)’: 1.5 would require a 50 percent or less difference between the current day, and the prior days_back.

param use_legacy_sql

Whether to use legacy SQL (true) or standard SQL (false).

param gcp_conn_id

(Optional) The connection ID used to connect to Google Cloud.

param location

The geographic location of the job. See details at: https://cloud.google.com/bigquery/docs/locations#specifying_your_location

param impersonation_chain

Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

param labels

a dictionary containing labels for the table, passed to BigQuery

Was this page helpful?