Autoscaling
Polars can scale its worker pool while the cluster is running, instead of holding a fixed number of workers. Polars does not create or destroy machines itself. Instead, it delegates that to a scaling service that you implement and run, using either REST or gRPC. This lets you drive scaling with the same infrastructure you already use to provision machines, such as a cloud autoscaling group, a VM API or a batch scheduler.
See Autoscaling for how queries ask for workers on an autoscaled cluster.
Configuration
Both transports carry the same two operations. REST is the simpler choice if you already run an HTTP service.
Configure exactly one transport on the node running the scheduler, and set
scheduler.default_workers_per_query to the number of workers a query uses when it does not request
a count of its own. The scheduler refuses to start without it.
[scaling.rest]
uri = "http://my-scaler.internal:8080"
[scaling.grpc]
uri = "http://my-scaler.internal:8080"
The default worker count is set on the scheduler, whichever transport you use.
[scheduler]
enabled = true
default_workers_per_query = 4
See the configuration reference for the remaining keys.
Scaling behavior
The scheduler's leader node tracks how many workers the running queries want, and asks your service to match that number. Demand is the sum of the worker counts requested by all running queries, so two concurrent queries that each want four workers produce a demand of eight.
The scheduler asks for more workers as soon as demand rises, and asks for fewer once the pool has been idle for 60 seconds. It never scales below the number of workers that currently hold a query.
The scheduler clamps every request to the minimum and maximum your service reports. A query asking
for more workers than your maximum is capped to it and runs at that size. Queries give up on workers
that take longer to arrive than the scale-up timeout, so that timeout is the window your service has
to provision them. It defaults to 5 minutes, and is set per transport with
scaling.rest.scale_up_timeout or scaling.grpc.scale_up_timeout.
Scaling to a size
The scheduler asks you to move the pool to a target size.
| Field | Meaning |
|---|---|
amount |
The total number of workers wanted, not a delta. |
workers_to_keep |
Workers that must not be removed. |
workers_to_delete |
Workers that should be removed first, if any need removing. |
The workers listed in workers_to_keep are running a query, and removing one of them fails that
query. If your service cannot choose which specific workers go away, for instance because it only
sets a replica count, it should refuse to scale down while that list is non-empty instead of
removing an arbitrary worker.
Worker names in both lists are the identifiers the workers registered with the scheduler (i.e.
instance_id). The lists are plain arrays of names over REST, and ValueList messages wrapping a
values field over gRPC.
Answer as soon as you have accepted the target, rather than waiting for machines to boot. Report the
new target as desired straight away, and let available catch up as the workers register with the
scheduler (see below).
Failing the request is safe. The scheduler logs a warning, keeps the size it had, and asks again at its next scaling decision, so an error is a reasonable way to refuse a change you cannot apply yet. Queries waiting for those workers still fail once the scale-up timeout passes.
Reporting configuration and state
The scheduler asks for your current state and limits.
| Field | Meaning |
|---|---|
desired |
The number of workers you are currently targeting. |
available |
The number of workers that are up and ready now. |
min |
The fewest workers the scheduler may scale down to. |
max |
The most workers the scheduler may scale up to. |
min must not exceed max. You can change these values whenever your own limits change, for
example when an operator raises a quota, and the scheduler will use the new range without a restart.
Note
If this operation fails five times in a row, the scheduler stops asking for configuration and continues with the last limits it saw until it is restarted.
REST
Your service exposes two endpoints, based at the uri you configured.
GET /scale_config returns the current configuration and state.
{
"desired": 4,
"available": 4,
"min": 0,
"max": 16
}
POST /scale_to receives the target size, and should answer 204 No Content.
{
"amount": 6,
"workers_to_keep": ["worker-a", "worker-b"],
"workers_to_delete": ["worker-c"]
}
The workers_to_keep and workers_to_delete fields are optional, and may be omitted or null.
The scheduler allows 5 seconds per request. It retries a request that times out, cannot connect, or
is answered with 429 Too Many Requests, up to four times, waiting a second longer before each
attempt. Any other error response is not retried. It re-reads /scale_config every 5 minutes, so a
change to your limits takes up to that long to be noticed.
OpenAPI specification
{
"openapi": "3.1.0",
"info": {
"title": "Polars Cloud Scaling",
"description": "REST API specification for the Polars Cloud auto-scaler.",
"version": "1.0.0"
},
"paths": {
"/scale_config": {
"get": {
"tags": [
"Scaling"
],
"description": "Get the current auto-scaling config.",
"responses": {
"200": {
"description": "",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ScaleConfigResponse"
}
}
}
}
}
}
},
"/scale_to": {
"post": {
"tags": [
"Scaling"
],
"description": "Scale to a target amount of workers.",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ScaleToBody"
}
}
},
"required": true
},
"responses": {
"204": {
"description": "no content"
}
}
}
}
},
"components": {
"schemas": {
"ScaleConfigResponse": {
"type": "object",
"properties": {
"available": {
"description": "The currently available amount of workers.",
"type": "integer",
"format": "uint32",
"minimum": 0
},
"desired": {
"description": "The desired amount of workers to auto-scale to.",
"type": "integer",
"format": "uint32",
"minimum": 0
},
"max": {
"description": "The upper-bound of workers to auto-scale (up) to.",
"type": "integer",
"format": "uint32",
"minimum": 0
},
"min": {
"description": "The lower-bound of workers to auto-scale (down) to.",
"type": "integer",
"format": "uint32",
"minimum": 0
}
},
"required": [
"desired",
"available",
"min",
"max"
]
},
"ScaleToBody": {
"type": "object",
"properties": {
"amount": {
"description": "Target amount of workers.",
"type": "integer",
"format": "uint32",
"minimum": 0
},
"workers_to_delete": {
"description": "Workers that must be deleted when scaling down.",
"type": [
"array",
"null"
],
"items": {
"type": "string"
}
},
"workers_to_keep": {
"description": "Workers that must be kept when scaling down.",
"type": [
"array",
"null"
],
"items": {
"type": "string"
}
}
},
"required": [
"amount"
]
}
}
}
}
gRPC
Your service implements the ScalingService service. ScaleConfig is a server-streaming RPC:
answer immediately with your current configuration, then send a new message every time it changes. A
ScaleTo call answered with UNAVAILABLE is retried; other statuses are not.
syntax = "proto3";
import "compute_scaling/client/v1/scale_to.proto";
import "compute_scaling/client/v1/scale_config.proto";
package compute_scaling.client.v1;
/**
* ScalingService is a service which provides scaling up/down of the cluster.
*/
service ScalingService {
// Scale the cluster to the requested size. This is called when the cluster wants to scale up or down.
rpc ScaleTo(ScaleToRequest) returns (ScaleToResponse);
// Sync the autoscaler's configuration and state. The autoscaler should respond with its current configuration and state as quickly as possible and then send additional responses whenever its configuration changes
rpc ScaleConfig(ScaleConfigRequest) returns (stream ScaleConfigResponse);
}
syntax = "proto3";
package compute_scaling.client.v1;
// This request is sent when we want to get the latest config information.
message ScaleConfigRequest {}
message ScaleConfigResponse {
// Amount which currently is requested from the AutoScaler.
uint32 desired = 1;
// Amount of workers currently scaled up.
uint32 available = 2;
// Lower bound of allowed scaling.
uint32 min = 3;
// Upper bound of allowed scaling.
uint32 max = 4;
}
syntax = "proto3";
package compute_scaling.client.v1;
message ScaleToRequest {
message ValueList { repeated string values = 1; }
// Number of workers to scale to.
uint32 amount = 1;
// Workers which should NOT be removed (when scaling down).
optional ValueList workers_to_keep = 2;
// Workers which should be removed first (and replaced if amount requires it).
optional ValueList workers_to_delete = 3;
}
message ScaleToResponse {}