Scaling

A sandbox runs as many instances as its traffic needs, between a floor and a ceiling you set — and by default the floor is zero. You are charged for an instance only while it runs, so a sandbox nobody is calling costs nothing.

The settings

The first three belong to the sandbox and change in place. max_concurrency belongs to the revision, so changing it deploys.

Sandboxes → api → ⋯ → Scaling → Always running, At most, Idle after → Save

Starting from zero

When a request arrives and nothing is running, it is held, an instance starts, and the request is sent to it as soon as it is accepting connections. To the caller it is a slow response — typically a few seconds — rather than an error. Requests that arrive during the start wait with it rather than each starting an instance of their own.

The wait counts against the revision's request timeout (5m unless you changed it), because that is what the caller experiences. If no instance can be started in time, the caller gets 503 Service Unavailable with a Retry-After header.

An instance is ready when your container accepts a connection on its port. A container that takes longer than the revision's startup timeout (4m by default) to do that is treated as failed to start.

Scaling out

Plany watches how many requests each instance is handling and starts more when the running instances are at their max_concurrency. That number is a promise rather than a hint: an instance is sent at most that many requests at once, and Plany aims to fill each one to it rather than running extra instances "to be safe".

Set it to suit your code:

• 1 for code that cannot safely handle two requests at once. Plany starts an instance per concurrent request instead.
• High (the default 80, up to 1000) for code that spends its time waiting on a database, another API or a game server — which is most web code.

Scaling stops at max_instances. When that many instances are all at their max_concurrency, a further request waits for one of them to free a slot, up to the request timeout.

Scaling in

An instance with nothing in flight for idle_timeout is released, down to min_instances. Idle is measured from the last request finishing, and it is decided from the requests Plany sent the instance — never by asking your container whether it is busy.

An instance being released stops receiving new requests and finishes the ones it has before it goes; nothing in flight is cut off. Plany also never removes an instance it cannot tell is idle: if it has no reading for one, the instance is kept.

Keeping instances running

min_instances above zero keeps that many instances running whatever the traffic, so nobody ever waits for a start — at the cost of paying for them continuously, as for any always-on service. It is how you run a container permanently: a Discord bot, a worker holding a connection open, a service that must answer instantly.

The ceiling is a limit and a budget

max_instances is the most a sandbox will ever run, and therefore the most it can ever cost per hour. It matters most for a sandbox fed by triggers, where the traffic comes from your players rather than from anything you control.

It is also what the project's Sandbox instances quota counts: the sum of every sandbox's max_instances in the project and region, not how many happen to be running. Creating a sandbox whose ceiling would take the project past its quota is refused there and then — not later, during the spike.

Watching it

The sandbox's page in the console shows how many instances are running now, and its size, scaling range and connected network.

plany sandbox sandboxes get api

instance_count on the API says the same. When Plany has no current reading it says so rather than reporting zero — "no instances" and "not measured" are different answers, and only the first is normal.