LocalToAzureDataLakeStorageOperator
Microsoft AzureThis class is deprecated. Please use airflow.providers.microsoft.azure.transfers.local_to_adls.LocalFilesystemToADLSOperator.
Access Instructions
Install the Microsoft Azure provider package into your Airflow environment.
Import the module into your DAG file and instantiate it with your desired params.
Parameters
local_pathlocal path. Can be single file, directory (in which case, upload recursively) or glob pattern. Recursive glob patterns using ** are not supported
remote_pathRemote path to upload to; if multiple files, this is the directory root to write within
nthreadsNumber of threads to use. If None, uses the number of cores.
overwriteWhether to forcibly overwrite existing files/directories. If False and remote path is a directory, will quit regardless if any files would be overwritten or not. If True, only matching filenames are actually overwritten
buffersizeint [2**22] Number of bytes for internal buffer. This block cannot be bigger than a chunk and cannot be smaller than a block
blocksizeint [2**22] Number of bytes for a block. Within each chunk, we write a smaller block for each API call. This block cannot be bigger than a chunk
extra_upload_optionsExtra upload options to add to the hook upload method
azure_data_lake_conn_idReference to the Azure Data Lake connection
Documentation
This class is deprecated. Please use airflow.providers.microsoft.azure.transfers.local_to_adls.LocalFilesystemToADLSOperator.