Apache Airflow
While not an explicit provider package, Apache Airflow contains a set of native modules that are closely-coupled to the project codebase. Many providers extend these core modules to add additional, more specialized functionality.
Quick Install
Available Modules
Modules are Python callables available from this provider package.
Types:
SensorsSecretsOperatorsHooksSensor operators are derived from this class and inherit these attributes.
Waits until the specified datetime.
Executes a bash command/script.
Waits for a different DAG, task group, or task to complete for a specific logical date.
Abstract base class to retrieve Connection object given a conn_id or Variable given a key
This class is deprecated. Please use airflow.utils.task_group.TaskGroup.
Waits until the first specified day of the week.
Triggers a DAG run for a specified dag_id
Allows a pipeline to continue based on the result of a python_callable.
This class stores a sensor work with decoded context value. It is only used inside of smart sensor. Create a sensor work based on sensor instance reco…
Waits for a Python callable to return True.
Waits for a file or folder to land in a filesystem.
Allows one to run a function in a virtualenv that is created and destroyed automatically (with certain caveats).
Executes a Python callable
Sends an email.
This is a base class for creating operators with branching functionality, similarly to BranchPythonOperator.
Execute a Bash script, command or set of commands.
Allows for interaction with an file server.
Operator that does literally nothing. It can be used to group tasks in a DAG.
Allows a workflow to “branch” or follow a path following the execution of this task.
Allows a workflow to skip tasks that are not running during the most recent schedule interval.
Retrieves Connection objects and Variables from local files
Retrieves Connection object and Variable from airflow metastore database.
Interface that providers can implement to be discovered by ProvidersManager.
Abstract base class for hooks.
Retrieves Connection object and Variable from environment variable.
Waits for a timedelta after the run’s data interval.
Waits until the specified time of the day.
Hook for running processes with the subprocess module.
Branches into one of two lists of tasks depending on the current day. For more information on how to use this operator, take a look at the guide: Bran…
Branches into one of two lists of tasks depending on the current datetime. For more information on how to use this operator, take a look at the guide:…
Operator link for TriggerDagRunOperator. It allows users to access DAG triggered by task using TriggerDagRunOperator.
A deferrable drop-in replacement for TimeDeltaSensor.
Waits until the specified datetime occurs.
Waits until the specified time of the day.
Operator that does literally nothing. It can be used to group tasks in a DAG.
Allows one to run a function in a virtualenv that is not re-created but used as is without the overhead of creating the virtualenv (with certain cavea…
Operator that does literally nothing but it logs YouTube link to Sade song “Smooth Operator”.