S3DeleteObjectsOperator
AmazonTo enable users to delete single object or multiple objects from a bucket using a single HTTP request.
Access Instructions
Install the Amazon provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
bucketRequiredName of the bucket in which you are going to delete object(s). (templated)
keysThe key(s) to delete from S3 bucket. (templated) When keys is a string, it’s supposed to be the key name of the single object to delete. When keys is a list, it’s supposed to be the list of the keys to delete.
prefixPrefix of objects to delete. (templated) All objects matching this prefix in the bucket will be deleted.
aws_conn_idConnection id of the S3 connection to use
verifyWhether or not to verify SSL certificates for S3 connection. By default SSL certificates are verified. You can provide the following values: False: do not validate SSL certificates. SSL will still be used,but SSL certificates will not be verified. path/to/cert/bundle.pem: A filename of the CA cert bundle to uses.You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.
Documentation
To enable users to delete single object or multiple objects from a bucket using a single HTTP request.
See also
For more information on how to use this operator, take a look at the guide: Delete Amazon S3 objects
Example DAGs
Move Files in S3 with Dynamic Task Mapping
This DAG shows an example implementation of sorting files in an S3 bucket into two different buckets based on logic involving the content of the files using dynamic task mapping with the expand_kwargs…
Airflow FundamentalsETL/ELT
Crate Data Quality Checks DAG
Imports local files to S3, then to CrateDB and checks several data quality properties
ETL/ELTData QualityDatabases