CloudSecretManagerBackend

Google

Retrieves Connection object from Google Cloud Secrets Manager

View on GitHub

Last Updated: Oct. 23, 2022

Access Instructions

Install the Google provider package into your Airflow environment.

Update your environment config per the instructions in the docs below.

Parameters

Documentation

Connections

Parameters

connections_prefixSpecifies the prefix of the secret to read to get Connections. If set to None (null), requests for connections will not be sent to GCP Secrets Manager

variables_prefixSpecifies the prefix of the secret to read to get Variables. If set to None (null), requests for variables will not be sent to GCP Secrets Manager

config_prefixSpecifies the prefix of the secret to read to get Airflow Configurations containing secrets. If set to None (null), requests for configurations will not be sent to GCP Secrets Manager

gcp_key_pathPath to Google Cloud Service Account key file (JSON). Mutually exclusive with gcp_keyfile_dict. use default credentials in the current environment if not provided.

gcp_keyfile_dictDictionary of keyfile parameters. Mutually exclusive with gcp_key_path.

gcp_scopesComma-separated string containing OAuth2 scopes

project_idProject ID to read the secrets from. If not passed, the project ID from credentials will be used.

sepSeparator used to concatenate connections_prefix and conn_id. Default: “-”

Documentation

Retrieves Connection object from Google Cloud Secrets Manager

Configurable via airflow.cfg as follows:

[secrets]
backend = airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend
backend_kwargs = {"connections_prefix": "airflow-connections", "sep": "-"}

For example, if the Secrets Manager secret id is airflow-connections-smtp_default, this would be accessible if you provide {"connections_prefix": "airflow-connections", "sep": "-"} and request conn_id smtp_default.

If the Secrets Manager secret id is airflow-variables-hello, this would be accessible if you provide {"variables_prefix": "airflow-variables", "sep": "-"} and request Variable Key hello.

The full secret id should follow the pattern “[a-zA-Z0-9-_]”.