adp.core.adls.ADLSClient

class adp.core.adls.ADLSClient(account_url: str, file_system_name: str, credential: str | Dict[str, str] | AzureNamedKeyCredential | AzureSasCredential | TokenCredential | None = None, **kwargs: Any)

Client for interfacing with Azure Datalake Gen2

__init__(account_url: str, file_system_name: str, credential: str | Dict[str, str] | AzureNamedKeyCredential | AzureSasCredential | TokenCredential | None = None, **kwargs: Any) → None

Methods

__init__(account_url, file_system_name[, ...])
acquire_lease([lease_duration, lease_id])	Requests a new lease.
build_from_appsettings()	build_from_appsettings returns a ADLSClient from the specified adp.core.settings
close()	This method is to close the sockets opened by the client.
create_directory(directory[, metadata])	Create directory
create_file(file, **kwargs)	Create file
create_file_system([metadata, public_access])	Creates a new file system under the specified account.
delete_directory(directory, **kwargs)	Marks the specified path for deletion.
delete_file(file, **kwargs)	Marks the specified file for deletion.
delete_file_system(**kwargs)	Marks the specified file system for deletion.
download_file(path)	download_file downloads a file from the ADLS
exists(**kwargs)	Returns True if a file system exists and returns False otherwise.
from_connection_string(conn_str, ...[, ...])	Create FileSystemClient from a Connection String.
get_all_directories_with_extension(path[, ...])	get_all_delta_directories returns all the directories ending with .delta in the given path
get_directory_client(directory)	Get a client to interact with the specified directory.
get_file_client(file_path)	Get a client to interact with the specified file.
get_file_system_access_policy(**kwargs)	Gets the permissions for the specified file system.
get_file_system_properties(**kwargs)	Returns all user-defined metadata and system properties for the specified file system.
get_paths([path, recursive, max_results])	Returns a generator to list the paths(could be files or directories) under the specified file system.
is_dir(path)	is_dir checks whether a path is a directory
list_deleted_paths(**kwargs)	Returns a generator to list the deleted (file or directory) paths under the specified file system.
list_dir(path)	List the contents of a directory path.
set_file_system_access_policy(signed_identifiers)	Sets the permissions for the specified file system or stored access policies that may be used with Shared Access Signatures.
set_file_system_metadata(metadata, **kwargs)	Sets one or more user-defined name-value pairs for the specified file system.
test_path(path)	test_path check if path exists

Attributes

api_version	The version of the Storage API used for requests.
location_mode	The location mode that the client is currently using.
primary_endpoint	The full primary endpoint URL.
primary_hostname	The hostname of the primary endpoint.
secondary_endpoint	The full secondary endpoint URL if configured.
secondary_hostname	The hostname of the secondary endpoint.
url	The full endpoint URL to this entity, including SAS token if used.

classmethod build_from_appsettings()

build_from_appsettings returns a ADLSClient from the specified adp.core.settings

download_file(path: str) → bytes

download_file downloads a file from the ADLS

Parameters:

path (str) – Path to the file to download

Returns:

The contents of the file

Return type:

bytes

get_all_directories_with_extension(path: str, extension: str = 'delta') → List[str]

get_all_delta_directories returns all the directories ending with .delta in the given path

Parameters:

path (str) – Path on the ADLS

Returns:

List of directories ending with .delta

Return type:

list

is_dir(path: str)

is_dir checks whether a path is a directory

Checks whether the given path is a directory.

Parameters:

path (str) – Path on the ADLS

Returns:

True when dir, False when file

Return type:

bool

list_dir(path: str)

List the contents of a directory path.

Parameters:

path (str) – Path to the directory to list

Returns:

List of items (can be files or subdirectories) in the directory

Return type:

List

test_path(path: str)

test_path check if path exists

User can either insert a Databricks formatted path (starting with dbfs:) or a ADLS path. Databricks formatted paths are converted to ADLS paths.

Parameters:

path (str) – Path to test if exists

Returns:

True if file exists, False otherwise

Return type:

bool