Microsoft Azure

Start a container on Azure Container Instances

View on GitHub

Last Updated: Oct. 23, 2022

Access Instructions

Install the Microsoft Azure provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.


ci_conn_idRequiredconnection id of a service principal which will be used to start the container instance
registry_conn_idRequiredconnection id of a user which can login to a private docker registry. For Azure use Azure connection id
resource_groupRequiredname of the resource group wherein this container instance should be started
nameRequiredname of this container instance. Please note this name has to be unique in order to run containers in parallel.
imageRequiredthe docker image to be used
regionRequiredthe region wherein this container instance should be started
environment_variableskey,value pairs containing environment variables which will be passed to the running container
secured_variablesnames of environmental variables that should not be exposed outside the container (typically passwords).
volumeslist of Volume tuples to be mounted to the container. Currently only Azure Fileshares are supported.
memory_in_gbthe amount of memory to allocate to this container
cputhe number of cpus to allocate to this container
gpuGPU Resource for the container.
commandthe command to run inside the container
container_timeoutmax time allowed for the execution of the container instance.
tagsazure tags as dict of str:str
os_typeThe operating system type required by the containers in the container group. Possible values include: ‘Windows’, ‘Linux’
restart_policyRestart policy for all containers within the container group. Possible values include: ‘Always’, ‘OnFailure’, ‘Never’
ip_addressThe IP address type of the container group.
network_profileThe network profile information for a container group.


Start a container on Azure Container Instances


ci_conn_id = "azure_service_principal",
registry_conn_id = "azure_registry_user",
resource_group = "my-resource-group",
name = "my-container-name-{{ ds }}",
image = "",
region = "westeurope",
environment_variables = {"MODEL_PATH": "my_value",
"POSTGRES_LOGIN": "{{ macros.connection('postgres_default').login }}",
"POSTGRES_PASSWORD": "{{ macros.connection('postgres_default').password }}",
"JOB_GUID": "{{ ti.xcom_pull(task_ids='task1', key='guid') }}" },
secured_variables = ['POSTGRES_PASSWORD'],
volumes = [("azure_container_instance_conn_id",
gpu=GpuResource(count=1, sku='K80'),
command=["/bin/echo", "world"],

Was this page helpful?