ClusterContext#

class polars_cloud.ClusterContext( uri: str, *, domain_name: str | None = None, extra_headers: Mapping[str, str] | None = None, tls_options: TLSOptions | None = None, observatory: ClientOptions | None = None, )

Cluster context in which queries are executed.

The cluster context is an abstraction of some remote cluster that is routable from the client. This will bypass any calls to the Polars Cloud control plane and talk to the cluster’s scheduler directly. Will attempt to establish an gRPC connection on __init__.

Parameters:

uri: URI of the scheduler reachable from the client, e.g. https://[HOSTNAME|IP]:5051. If no port is specified, defaults to 5051.
domain_name: Used as authority, host header (inc port) and for TLS verification. Make sure to use an [IP] in the uri when using this option. https://developer.mozilla.org/en-US/docs/Glossary/Domain_name
extra_headers: Additional HTTP headers to include with each request to the cluster. The following headers are forbidden and will raise an error if provided: host, content-length, transfer-encoding.
tls_options: TLS options for the connection, including custom CA certificates and insecure mode. If not set, uses system defaults.
observatory: Custom client options for the observatory. If not set, observatory is automatically configured from uri using the default port 3001. When specified, none of the other options will be copied by default.

Examples

>>> import polars as pl
>>> import polars_cloud as pc

>>> lf = pl.LazyFrame({"x": list(range(100))})

>>> # Simple config, observatory will be automatically configured.
>>> ctx = pc.ClusterContext(uri="https://[HOSTNAME|IP]")
>>> lf.remote(ctx).execute()

>>> # Config with custom CA, will be used for both scheduler and observatory.
>>> with open("certificate.pem", "rb") as cert:
...     custom_ca = cert.read()
>>> ctx = pc.ClusterContext(
...     uri="https://[HOSTNAME|IP]",
...     tls_options=TLSOptions(
...         ca_cert=custom_ca,  # Will also be used for the observatory.
...     ),
... )
>>> lf.remote(ctx).execute()

>>> # Custom config for observatory. Observatory will use default tls_options.
>>> ctx = pc.ClusterContext(
...     uri="https://[IP]:5051",
...     domain_name="pola.rs",
...     tls_options=TLSOptions(
...         insecure=True,  # Will NOT be used for the observatory.
...     ),
...     observatory=ClientOptions(uri="https://[IP]:3001", authority="obs.pola.rs"),
... )
>>> lf.remote(ctx).execute()