polars_cloud.ExecuteRemote.sink_csv#
- ExecuteRemote.sink_csv(
- uri: str,
- *,
- include_bom: bool = False,
- include_header: bool = True,
- separator: str = ',
- ',
- line_terminator: str = '\n',
- quote_char: str = '"',
- batch_size: int = 1024,
- datetime_format: str | None = None,
- date_format: str | None = None,
- time_format: str | None = None,
- float_scientific: bool | None = None,
- float_precision: int | None = None,
- decimal_comma: bool = False,
- null_value: str | None = None,
- quote_style: CsvQuoteStyle | None = None,
- maintain_order: bool = True,
- storage_options: dict[str,
- Any] | None = None,
- credential_provider: CredentialProviderFunction | Literal['auto'] | None = 'auto',
- sink_to_single_file: bool | None = None,
- optimizations: QueryOptFlags = <polars.lazyframe.opt_flags.QueryOptFlags object>,
Start executing the query and write the result to csv.
- Parameters:
- uri
Path to which the output should be written. Must be a URI to an accessible object store location.
It is recommended to write to a directory path for example
"my-location/"
, instead of as single file as a single file can only be written from a single node.If set to
"local"
, the query is executed locally.- include_bom
Whether to include UTF-8 BOM in the CSV output.
- include_header
Whether to include header in the CSV output.
- separator
Separate CSV fields with this symbol.
- line_terminator
String used to end each row.
- quote_char
Byte to use as quoting character.
- batch_size
Number of rows that will be processed per thread.
- datetime_format
A format string, with the specifiers defined by the chrono Rust crate. If no format specified, the default fractional-second precision is inferred from the maximum timeunit found in the frame’s Datetime cols (if any).
- date_format
A format string, with the specifiers defined by the chrono Rust crate.
- time_format
A format string, with the specifiers defined by the chrono Rust crate.
- float_scientific
Whether to use scientific form always (true), never (false), or automatically (None) for
Float32
andFloat64
datatypes.- float_precision
Number of decimal places to write, applied to both
Float32
andFloat64
datatypes.- decimal_comma
Use a comma as the decimal separator instead of a point. Floats will be encapsulated in quotes if necessary; set the field separator to override.
- null_value
A string representing null values (defaulting to the empty string).
- quote_style{‘necessary’, ‘always’, ‘non_numeric’, ‘never’}
Determines the quoting strategy used.
necessary (default): This puts quotes around fields only when necessary. They are necessary when fields contain a quote, delimiter or record terminator. Quotes are also necessary when writing an empty record (which is indistinguishable from a record with one empty field). This is the default.
always: This puts quotes around every field. Always.
never: This never puts quotes around fields, even if that results in invalid CSV data (e.g.: by not quoting strings containing the separator).
non_numeric: This puts quotes around all fields that are non-numeric. Namely, when writing a field that does not parse as a valid float or integer, then quotes will be used even if they aren`t strictly necessary.
- maintain_order
Maintain the order in which data is processed. Setting this to
False
can be much faster.Warning
This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.
- storage_options
Options that indicate how to connect to a cloud provider.
The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here:
Hugging Face (
hf://
): Accepts an API key under thetoken
parameter:{'token': '...'}
, or by setting theHF_TOKEN
environment variable.
If
storage_options
is not provided, Polars will try to infer the information from environment variables.- credential_provider
Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time.
Warning
This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.
- sink_to_single_file
Perform the sink into a single file.
Setting this to
True
can reduce the amount of work that can be done in a distributed manner and therefore be more memory intensive and slower.- optimizations
The optimization passes done during query optimization.
Warning
This functionality is considered unstable. It may be changed at any point without it being considered a breaking change.
Examples
>>> query.remote(ctx).sink_csv("s3://your-bucket/folder/file.csv") <polars_cloud.query.query.ProxyQuery at 0x107e68fb0>