What Is Scale to Zero and How Does It Work?

It is shutting a service down completely when no one is using it, then restarting it whenever a request comes in. You incur zero cost during the idle time, at the cost of a short startup delay before the first request is answered.

It is the delay when a service that has scaled to zero must start an instance before it can handle a request. Because starting from scratch takes longer than using a running instance, the first request after idle is slower.

Autoscaling increases or decreases the number of instances based on load but normally keeps at least one active. Scale to zero goes all the way to no instances when idle, so it pays nothing while idle but incurs a cold start on the next request.

Because GPUs are expensive, an idle GPU endpoint wastes a lot of money. Scaling it to zero means it costs nothing between bursts of use, which is where the savings are largest, so bursty GPU workloads benefit most.

For steady, high-traffic services, or ones that cannot tolerate any startup delay. There, always-on or autoscaling with a minimum instance is better, since the latency of cold starts would outweigh the savings on idle capacity.