polars_cloud.LazyFrameExt.distributed#
- LazyFrameExt.distributed(
- *,
- shuffle_compression: ShuffleCompression = 'auto',
- sort_partitioned: bool = True,
- pre_aggregation: bool = True,
Whether the query should run in a distributed fashion.
- Parameters:
- shuffle_compression{‘auto’, ‘lz4’, ‘zstd’, ‘uncompressed’}
Compress files before shuffling them. Compression reduces disk and network IO, but disables memory mapping. Choose “zstd” for good compression performance. Choose “lz4” for fast compression/decompression. Choose “uncompressed” for memory mapped access at the expense of file size.
- sort_partitioned
Whether group-by and selected aggregations are pre-aggregated on worker nodes.
- pre_aggregation
Whether group-by and selected aggregations are pre-aggregated on worker nodes if possible.