PythonOperator

Apache Airflow Certified

Executes a Python callable

View on GitHub

Last Updated: Apr. 8, 2023

Access Instructions

Install the Apache Airflow provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

python_callableRequiredA reference to an object that is callable
op_kwargsa dictionary of keyword arguments that will get unpacked in your function
op_argsa list of positional arguments that will get unpacked when calling your callable
templates_dicta dictionary where the values are templates that will get templated by the Airflow engine sometime between __init__ and execute takes place and are made available in your callable’s context after the template has been applied. (templated)
templates_extsa list of file extensions to resolve while processing templated fields, for examples ['.sql', '.hql']
show_return_value_in_logsa bool value whether to show return_value logs. Defaults to True, which allows return value log output. It can be set to False to prevent log output of return value when you return huge data such as transmission a large amount of XCom to TaskAPI.

Documentation

Executes a Python callable

See also

For more information on how to use this operator, take a look at the guide: PythonOperator

When running your callable, Airflow will pass a set of keyword arguments that can be used in your function. This set of kwargs correspond exactly to what you can use in your jinja templates. For this to work, you need to define **kwargs in your function header, or you can add directly the keyword arguments you would like to get - for example with the below code your callable will get the values of ti and next_ds context variables.

With explicit arguments:

def my_python_callable(ti, next_ds):
pass

With kwargs:

def my_python_callable(**kwargs):
ti = kwargs["ti"]
next_ds = kwargs["next_ds"]

Example DAGs

complex

Example Airflow DAG that shows the complex DAG structure.

Apache Airflow
Airflow Fundamentals
python_operator

Example DAG demonstrating the usage of the PythonOperator.

Apache Airflow
Airflow Fundamentals
xcom

Example DAG demonstrating the usage of XComs.

Apache Airflow
Airflow Fundamentals
trigger_target_dag

Example usage of the TriggerDagRunOperator. This example holds 2 DAGs: 1. 1st DAG (example_trigger_controller_dag) holds a TriggerDagRunOperator, which will trigger the 2nd DAG 2. 2nd DAG (example_tri…

Apache Airflow
Airflow Fundamentals
passing_params_via_test_command

Example DAG demonstrating the usage of the params arguments in templated arguments.

Apache Airflow
Airflow Fundamentals
xcomargs

Example DAG demonstrating the usage of the XComArgs.

Apache Airflow
Airflow Fundamentals
kubernetes_executor

This is an example dag for using the Kubernetes Executor.

Apache Airflow
Containers
Pipeline Alerts and Notifications using Microsoft Teams

Example DAG demonstrating how to implement Microsoft Teams alerting and notifications.

Apache Airflow HTTP
Alerts/Notifications
Pipeline Alerts and Notifications for Multiple Microsoft Teams Channels

Example DAG demonstrating how to implement alerting and notifications for multiple Microsoft Teams channels.

Apache Airflow HTTP
Alerts/Notifications
Pipeline Alerts and Notifications using Slack

Example DAG demonstrating how to implement alerting and notifications in Slack.

Apache Airflow Slack
Alerts/Notifications
Pipeline Alerts and Notifications for Multiple Slack Channels

Example DAG demonstrating how to implement alerting and notifications for multiple Slack channels

Apache Airflow SlackHTTP
Alerts/Notifications
Example Reporting DAG

Example DAG that queries the Airflow Metadata Database and moves data through GCS to Postgres.

PostgresGoogle
ETL/ELTDatabases
lakefs-dag

### Example lakeFS DAG

LakeFS
Data Management & Governance

Was this page helpful?