AzureContainerInstancesOperator

Microsoft Azure

Start a container on Azure Container Instances

View on GitHub

Last Updated: Oct. 23, 2022

Access Instructions

Install the Microsoft Azure provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

Documentation

Connections

Parameters

ci_conn_idRequiredconnection id of a service principal which will be used to start the container instance

registry_conn_idRequiredconnection id of a user which can login to a private docker registry. For Azure use Azure connection id

resource_groupRequiredname of the resource group wherein this container instance should be started

nameRequiredname of this container instance. Please note this name has to be unique in order to run containers in parallel.

imageRequiredthe docker image to be used

regionRequiredthe region wherein this container instance should be started

environment_variableskey,value pairs containing environment variables which will be passed to the running container

secured_variablesnames of environmental variables that should not be exposed outside the container (typically passwords).

volumeslist of Volume tuples to be mounted to the container. Currently only Azure Fileshares are supported.

memory_in_gbthe amount of memory to allocate to this container

cputhe number of cpus to allocate to this container

gpuGPU Resource for the container.

commandthe command to run inside the container

container_timeoutmax time allowed for the execution of the container instance.

tagsazure tags as dict of str:str

os_typeThe operating system type required by the containers in the container group. Possible values include: ‘Windows’, ‘Linux’

restart_policyRestart policy for all containers within the container group. Possible values include: ‘Always’, ‘OnFailure’, ‘Never’

ip_addressThe IP address type of the container group.

network_profileThe network profile information for a container group.

Documentation

Start a container on Azure Container Instances

Example:

AzureContainerInstancesOperator(
    ci_conn_id = "azure_service_principal",
    registry_conn_id = "azure_registry_user",
    resource_group = "my-resource-group",
    name = "my-container-name-{{ ds }}",
    image = "myprivateregistry.azurecr.io/my_container:latest",
    region = "westeurope",
    environment_variables = {"MODEL_PATH":  "my_value",
     "POSTGRES_LOGIN": "{{ macros.connection('postgres_default').login }}",
     "POSTGRES_PASSWORD": "{{ macros.connection('postgres_default').password }}",
     "JOB_GUID": "{{ ti.xcom_pull(task_ids='task1', key='guid') }}" },
    secured_variables = ['POSTGRES_PASSWORD'],
    volumes = [("azure_container_instance_conn_id",
            "my_storage_container",
            "my_fileshare",
            "/input-data",
        True),],
    memory_in_gb=14.0,
    cpu=4.0,
    gpu=GpuResource(count=1, sku='K80'),
    command=["/bin/echo", "world"],
    task_id="start_container"
)