IpcDst#

class polars_cloud.IpcDst( uri: str | PartitioningScheme, *, compression: IpcCompression | None = 'zstd', compat_level: CompatLevel | None = None, maintain_order: bool = True, storage_options: dict[str, Any] | None = None, credential_provider: CredentialProviderFunction | Literal['auto'] | None = 'auto', )

Ipc destination arguments.

Parameters:

uri

Path to which the output should be written. Must be a URI to an accessible object store location.

It is recommended to write to a directory path for example "my-location/", instead of as single file as a single file can only be written from a single node.

If set to "local", the query is executed locally.

compression{‘uncompressed’, ‘lz4’, ‘zstd’}

Choose “zstd” for good compression performance. Choose “lz4” for fast compression/decompression.

compat_level

Use a specific compatibility level when exporting Polars’ internal data structures.

maintain_order

Maintain the order in which data is processed. Setting this to False can be much faster.

Warning

This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.

storage_options

Options that indicate how to connect to a cloud provider.

The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here:

aws
gcp
azure
Hugging Face (hf://): Accepts an API key under the token parameter: {'token': '...'}, or by setting the HF_TOKEN environment variable.

If storage_options is not provided, Polars will try to infer the information from environment variables.

credential_provider

Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time.

Warning

This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.