SystemsManagerParameterStoreBackend

Amazon

Retrieves Connection or Variables from AWS SSM Parameter Store

View on GitHub

Last Updated: Mar. 1, 2023

Access Instructions

Install the Amazon provider package into your Airflow environment.

Update your environment config per the instructions in the docs below.

Parameters

Documentation

Connections

Parameters

connections_prefixSpecifies the prefix of the secret to read to get Connections. If set to None (null), requests for connections will not be sent to AWS SSM Parameter Store.

connections_lookup_patternSpecifies a pattern the connection ID needs to match to be looked up in AWS Parameter Store. Applies only if connections_prefix is not None. If set to None (null value in the configuration), all connections will be looked up first in AWS Parameter Store.

variables_prefixSpecifies the prefix of the secret to read to get Variables. If set to None (null), requests for variables will not be sent to AWS SSM Parameter Store.

variables_lookup_patternSpecifies a pattern the variable key needs to match to be looked up in AWS Parameter Store. Applies only if variables_prefix is not None. If set to None (null value in the configuration), all variables will be looked up first in AWS Parameter Store.

config_prefixSpecifies the prefix of the secret to read to get Variables. If set to None (null), requests for configurations will not be sent to AWS SSM Parameter Store.

config_lookup_patternSpecifies a pattern the config key needs to match to be looked up in AWS Parameter Store. Applies only if config_prefix is not None. If set to None (null value in the configuration), all config keys will be looked up first in AWS Parameter Store.

Documentation

Retrieves Connection or Variables from AWS SSM Parameter Store

Configurable via airflow.cfg like so:

[secrets]
backend = airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend
backend_kwargs = {"connections_prefix": "/airflow/connections", "profile_name": null}

For example, if ssm path is /airflow/connections/smtp_default, this would be accessible if you provide {"connections_prefix": "/airflow/connections"} and request conn_id smtp_default. And if ssm path is /airflow/variables/hello, this would be accessible if you provide {"variables_prefix": "/airflow/variables"} and variable key hello.

You can also pass additional keyword arguments listed in AWS Connection Extra config to this class, and they would be used for establish connection and passed on to Boto3 client.

[secrets]
backend = airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend
backend_kwargs = {"connections_prefix": "airflow/connections", "region_name": "eu-west-1"}