BigQueryIntervalCheckOperator
GoogleChecks that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.
Access Instructions
Install the Google provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
tableRequiredthe table name
days_backnumber of days between ds and the ds we want to check against. Defaults to 7 days
metrics_thresholdsRequireda dictionary of ratios indexed by metrics, for example ‘COUNT(*)’: 1.5 would require a 50 percent or less difference between the current day, and the prior days_back.
use_legacy_sqlWhether to use legacy SQL (true) or standard SQL (false).
gcp_conn_id(Optional) The connection ID used to connect to Google Cloud.
locationThe geographic location of the job. See details at: https://cloud.google.com/bigquery/docs/locations#specifying_your_location
impersonation_chainOptional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).
labelsa dictionary containing labels for the table, passed to BigQuery
deferrableRun operator in the deferrable mode
Documentation
Checks that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.
This method constructs a query like so
SELECT {metrics_threshold_dict_key} FROM {table}WHERE {date_filter_column}=<date>
See also
For more information on how to use this operator, take a look at the guide: Compare metrics over time