DummyOperator
Apache AirflowOperator that does literally nothing. It can be used to group tasks in a DAG.
Access Instructions
Install the Apache Airflow provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Documentation
Operator that does literally nothing. It can be used to group tasks in a DAG.
The task is evaluated by the scheduler but never processed by the executor.
Example DAGs
A DAG that runs multiple, parameterized dynamic queries against a Greenplum database based on a list of provided states using the PostgresOperator, and then sends an email upon successful completion u…
An example DAG manipulating a Pandas dataframe using the Ray backend.
A DAG using a basic Jupyter notebook model that pulls the HIGGS dataset, splits training and testing data, and creates/validates a model using the XGBoost on Ray.
Using the Ray backend and XGBoost to tune a simpled Jupyter notebook model that pulls the HIGGS dataset.
Example DAG from the Astronomer Azure Operators tutorial to ingest sample COVID-19 tracking data into Azure Blob Storage and setting up email alerts after workflow completion.
Example DAG demonstrating the usage of the BashOperator.
Example DAG demonstrating the usage of the BranchPythonOperator.
Example DAG demonstrating the usage of BranchPythonOperator with depends_on_past=True, where tasks may be run or skipped on alternating runs.
Example DAG demonstrating the DummyOperator and a custom DummySkipOperator which skips by default.
Example DAG demonstrating the usage of the ShortCircuitOperator.
Example DAG demonstrating the usage of the SubDagOperator.
Example DAG demonstrating the usage of the TaskGroup.
Example DAG demonstrating a workflow with nested branching. The join tasks are created with none_failed_or_skipped trigger rule such that they are skipped whenever their corresponding BranchPythonOper…
Example LatestOnlyOperator and TriggerRule interactions
Example of the LatestOnlyOperator
Example DAG demonstrating setting up inter-DAG dependencies using ExternalTaskSensor and ExternalTaskMarker
build dataframe from breast cancer dataset
#### build random dataframe task
Build a dataframe task.
An example of a dbt pipeline which generates tasks dynamically from a ``manifest.json`` file.
Example DAG demonstrating the usage of BranchDayOfWeekOperator.
Example DAG demonstrating the usage of DateTimeBranchOperator with datetime as well as time objects as targets.
Example DAG demonstrating the usage of labels with different branches.
### Sample DAG
An example pipeline demonstrating how to perform data quality checks in a database using SQL Check Operators.
An example pipeline demonstrating how to perform data quality checks in Redshift using SQL Check Operators.
#### build random dataframe task
This DAG demonstrates orchestrating multiple Azure Data Factory (ADF) pipelines using Airflow to perform classic ELT operations.
This reverse-ETL pipeline syncs customer data from a Snowflake data warehouse to a marketing platform using Census.
An ELT pipeline which extracts Salesforce customer data to Snowflake for transformation and enrichment, and synced back to Salesforce via Hightouch.
A modern ELT pipeline from extracting Salesforce data to loading and transforming in Snowflake.
Upload the following Zendesk objects to S3: Tickets, Organizations, Users. From S3, loads into Snowflake. Loads can be daily or full-extracts.
Example DAG demonstrating how to implement Microsoft Teams alerting and notifications.
Example DAG demonstrating how to implement alerting and notifications for multiple Microsoft Teams channels.
Example DAG demonstrating how to implement alerting and notifications in Slack.
Example DAG demonstrating how to implement alerting and notifications for multiple Slack channels
The google_analytics_to_snowflake DAG transfers data from Google Analytics (GA) to Google Cloud Storage (GCS), and then loads the data to Snowflake staging tables. From staging, data is loaded to Snow…
Example DAG showcasing a write-audit-publish data quality pattern with Snowflake and Great Expectations.
A simple DAG demonstrating an extract-and-load pipeline with data quality checks in BigQuery.
Example DAG showcasing loading and data quality checking with BigQuery and Great Expectations.
Example DAG showcasing loading and data quality checking with Redshift and Great Expectations.
Example DAG showcasing loading and data quality checking with Snowflake and Great Expectations.
This is a very simple DAG showing a minimal EL data pipeline with a data integrity check. A single file is uploaded to S3, then its ETag is verified against the MD5 hash of the local file.
This is the second in a series of DAGs showing an EL pipeline with data integrity checking of data in S3 as well as Redshift.
This is the third in a series of DAGs showing an EL pipeline with data integrity and data quality checking for data in S3 and Redshift using ETag verification and row-based data quality checks where t…
An example DAG showing the use of SnowflakeCheckOperators.
This example showcases an example of adding an operational check to ensure the dbt Cloud job is not running prior to triggering.
This example showcases a modern ELT pipeline extracting with Fivetran, performing data transformations via a dbt Cloud job, and reverse-ETL using Census.
