Pipeline Alerts and Notifications for Multiple Slack Channels

Example DAG demonstrating how to implement alerting and notifications for multiple Slack channels

Alerts/Notifications

Providers:

Apache Airflow Slack

Modules:

Last Updated: Nov. 15, 2021

Run this DAG

1. Install the Astronomer CLI:Skip if you already have our CLI

2. Download the repository:

3. Navigate to where the repository was cloned and start the DAG:

Readme Code Graph Dockerfile Requirements View on GitHub

Airflow Callbacks

Showcasing several ways to implement Airflow callbacks and notifications via Slack.

Description

Monitoring tasks and DAGs at scale can be cumbersome. Sometimes you'd like to be notified of certain events, and not others. These DAGs cover several methods of implementing custom Slack notifications, so you can be confident you aren't missing critical events that may require immediate attention.

Slack Callback Examples

Example Callbacks

Getting Started

Dependencies

To implement notifications via Slack, add the following to your requirements.txt:


apache-airflow-providers-slack
apache-airflow-providers-http

Installing

In order to run these demos on your localhost, be sure to install:

Executing demos

Clone this repository, then navigate to the cs-tutorial-slack-callbacks directory and start your local Airflow instance:


astro dev start

In your browser, navigate to http://localhost:8080/

Username: admin
Password: admin

Setting up Slack Connections in Airflow

In order to receive callback notifications, you must also create your webhooks and set up your connections in the Airflow UI. Follow the instructions found in the Appendix section of the accompanying slide deck to create these.