RedshiftToS3Operator
AmazonExecute an UNLOAD command to s3 as a CSV with headers.
Access Instructions
Install the Amazon provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
s3_bucketRequiredreference to a specific S3 bucket
s3_keyRequiredreference to a specific S3 key. If table_as_file_name is set to False, this param must include the desired file name
schemareference to a specific schema in redshift database Applicable when table param provided.
tablereference to a specific table in redshift database Used when select_query param not provided.
select_querycustom select query to fetch data from redshift database
redshift_conn_idreference to a specific redshift database
aws_conn_idreference to a specific S3 connection If the AWS connection contains ‘aws_iam_role’ in extras the operator will use AWS STS credentials with a token https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-authorization.html#copy-credentials
verifyWhether or not to verify SSL certificates for S3 connection. By default SSL certificates are verified. You can provide the following values: False: do not validate SSL certificates. SSL will still be used(unless use_ssl is False), but SSL certificates will not be verified. path/to/cert/bundle.pem: A filename of the CA cert bundle to uses.You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.
unload_optionsreference to a list of UNLOAD options
autocommitIf set to True it will automatically commit the UNLOAD statement. Otherwise it will be committed right before the redshift connection gets closed.
include_headerIf set to True the s3 file contains the header columns.
parameters(optional) the parameters to render the SQL query with.
table_as_file_nameIf set to True, the s3 file will be named as the table. Applicable when table param provided.
redshift_data_api_kwargsIf using the Redshift Data API instead of the SQL-based connection, dict of arguments for the hook’s execute_query method. Cannot include any of these kwargs: {'sql', 'parameters'}
Documentation
Execute an UNLOAD command to s3 as a CSV with headers.
See also
For more information on how to use this operator, take a look at the guide: Amazon Redshift To Amazon S3 transfer operator