Executes an COPY command to load files from s3 to Redshift

View on GitHub

Last Updated: Feb. 20, 2023

Access Instructions

Install the Amazon provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.


schemaRequiredreference to a specific schema in redshift database
tableRequiredreference to a specific table in redshift database
s3_bucketRequiredreference to a specific S3 bucket
s3_keyRequiredkey prefix that selects single or multiple objects from S3
redshift_conn_idreference to a specific redshift database OR a redshift data-api connection
aws_conn_idreference to a specific S3 connection If the AWS connection contains ‘aws_iam_role’ in extras the operator will use AWS STS credentials with a token https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-authorization.html#copy-credentials
verifyWhether or not to verify SSL certificates for S3 connection. By default SSL certificates are verified. You can provide the following values: False: do not validate SSL certificates. SSL will still be used(unless use_ssl is False), but SSL certificates will not be verified. path/to/cert/bundle.pem: A filename of the CA cert bundle to uses.You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.
column_listlist of column names to load
copy_optionsreference to a list of COPY options
methodAction to be performed on execution. Available APPEND, UPSERT and REPLACE.
upsert_keysList of fields to use as key on upsert action
redshift_data_api_kwargsIf using the Redshift Data API instead of the SQL-based connection, dict of arguments for the hook’s execute_query method. Cannot include any of these kwargs: {'sql', 'parameters'}


Executes an COPY command to load files from s3 to Redshift

See also

For more information on how to use this operator, take a look at the guide: Amazon S3 To Amazon Redshift transfer operator

Was this page helpful?