SQLIntervalCheckOperator
Common SQLChecks that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.
Access Instructions
Install the Common SQL provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
tableRequiredthe table name
conn_idthe connection ID used to connect to the database.
databasename of database which will overwrite the defined one in connection
days_backnumber of days between ds and the ds we want to check against. Defaults to 7 days
date_filter_columnThe column name for the dates to filter on. Defaults to ‘ds’
ratio_formulawhich formula to use to compute the ratio between the two metrics. Assuming cur is the metric of today and ref is the metric to today - days_back. max_over_min: computes max(cur, ref) / min(cur, ref) relative_diff: computes abs(cur-ref) / ref Default: ‘max_over_min’
ignore_zerowhether we should ignore zero metrics
metrics_thresholdsRequireda dictionary of ratios indexed by metrics
Documentation
Checks that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.