SFTPToWasbOperator
Microsoft AzureTransfer files to Azure Blob Storage from SFTP server.
Access Instructions
Install the Microsoft Azure provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
sftp_source_pathRequiredThe sftp remote path. This is the specified file path for downloading the single file or multiple files from the SFTP server. You can use only one wildcard within your path. The wildcard can appear inside the path or at the end of the path.
container_nameRequiredName of the container.
blob_prefixPrefix to name a blob.
sftp_conn_idThe sftp connection id. The name or identifier for establishing a connection to the SFTP server.
wasb_conn_idReference to the wasb connection.
load_optionsOptional keyword arguments that WasbHook.load_file() takes.
move_objectWhen move object is True, the object is moved instead of copied to the new location. This is the equivalent of a mv command as opposed to a cp command.
wasb_overwrite_objectWhether the blob to be uploaded should overwrite the current data. When wasb_overwrite_object is True, it will overwrite the existing data. If set to False, the operation might fail with ResourceExistsError in case a blob object already exists.
create_containerAttempt to create the target container prior to uploading the blob. This is useful if the target container may not exist yet. Defaults to False.
Documentation
Transfer files to Azure Blob Storage from SFTP server.
See also
For more information on how to use this operator, take a look at the guide: Transfer Data from SFTP Source Path to Blob Storage