GCSTaskHandler
GoogleGCSTaskHandler is a python log handler that handles and reads task instance logs. It extends airflow FileTaskHandler and uploads to and reads from GCS remote storage. Upon log reading failure, it reads from host machine’s local disk.
Access Instructions
Install the Google provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
base_log_folderRequiredBase log folder to place logs.
gcs_log_folderRequiredPath to a remote location where logs will be saved. It must have the prefix gs://. For example: gs://bucket/remote/log/location
filename_templatetemplate filename string
gcp_key_pathPath to Google Cloud Service Account file (JSON). Mutually exclusive with gcp_keyfile_dict. If omitted, authorization based on the Application Default Credentials will be used.
gcp_keyfile_dictDictionary of keyfile parameters. Mutually exclusive with gcp_key_path.
gcp_scopesComma-separated string containing OAuth2 scopes
project_idProject ID to read the secrets from. If not passed, the project ID from credentials will be used.
Documentation
GCSTaskHandler is a python log handler that handles and reads task instance logs. It extends airflow FileTaskHandler and uploads to and reads from GCS remote storage. Upon log reading failure, it reads from host machine’s local disk.