Cloud Run cold starts: a field guide

Measure which phase of a Cloud Run cold start is slow, match it to the lever that shortens it, and pay for warm instances only where the latency earns it.

Published

Updated

Reading time

15 min

A cold start is not one delay. It is a request waiting while Cloud Run finds capacity, starts a container and lets your code initialize, and each of those phases has a different owner and a different fix. Treating the whole thing as "Cloud Run is slow at first" leads to the expensive remedy (keep instances warm) before the cheap ones (stop doing work at import time) have been tried. This guide goes in the order I would debug a slow first request: find the cold requests, work out which phase is slow, apply the lever that matches, and only then decide whether warmth is worth a standing charge. It comes from running 32 Cloud Run services for a large-scale social platform I'm building, nearly all with minimum instances at zero. It ends with the other half of the scale-to-zero bargain, the fixed costs that stay on the bill even when every Cloud Run service is idle.

Isolate the cold requests#

A cold start happens whenever Cloud Run routes a request to an instance that does not exist yet: after scale-to-zero, when a burst outgrows the running instances, after a new revision is deployed, or when the platform replaces an instance. The request waits for four phases: scheduling, image fetch, container start and application init.

The request waits for scheduling, image fetch, container start and your app's init. Only the last one is fully under your control.

Every request produces a request log entry with httpRequest.latency. When a request caused a new instance to start, Cloud Run adds a note to that entry: "This request caused a new container instance to be started and may thus take longer and use more CPU than a typical request." That makes cold requests easy to pull out in Logs Explorer:

Logs Explorertext
resource.type="cloud_run_revision"
resource.labels.service_name="api"
logName:"run.googleapis.com%2Frequests"
textPayload:"caused a new container instance to be started"

Compare the latency of those entries with warm requests over the same window and you have the user-facing price of a cold start. In my services the structured logs also tag every request with the age of the instance that served it, so "first request on a fresh instance" is a filter rather than a guess. I watch the p95 of that filtered set, not the average: the average is flattered by mild cases, and the p95 is what someone hits when they open the app after a quiet spell. Count them over a week as well: the number of cold requests, not their duration, is what decides how much any fix is worth.

Cloud Run also exports run.googleapis.com/container/startup_latencies, a distribution of how long new instances take to become ready. Chart p50 and p99 per revision in Metrics Explorer and keep it open across deploys: a dependency bump that adds 800 ms of init shows up the same day.

Measure the phases you own#

Platform metrics say how long. Your own timestamps say where. Log phase markers relative to process start, as JSON so Cloud Logging parses them into fields:

src/startup-timing.tsts
const t0 = performance.timeOrigin;
 
export function mark(phase: string): void {
  // One JSON line per phase; Cloud Logging turns it into jsonPayload fields.
  process.stdout.write(
    JSON.stringify({
      severity: "INFO",
      message: `startup:${phase}`,
      phase,
      msSinceProcessStart: Math.round(performance.now()),
      processStartedAt: new Date(t0).toISOString(),
    }) + "\n",
  );
}

Call mark("modules-loaded") after the imports, mark("config-ready") after configuration is parsed and mark("listening") once the server is up. If startup_latencies is 2.4 s and listening is logged at 1.9 s, the platform phases are about half a second and the rest is yours. If the two numbers are nearly equal, nothing you change in the application will help much.

One more thing to check: by default Cloud Run uses a TCP startup probe, so an instance counts as ready as soon as the port accepts connections. A framework that opens the port first and warms caches afterwards will receive traffic early and make the first requests slow. An HTTP startup probe makes readiness honest:

service.yaml (excerpt)yaml
spec:
  template:
    spec:
      containers:
        - image: europe-west1-docker.pkg.dev/my-project/apps/api:latest
          startupProbe:
            httpGet:
              path: /healthz
            initialDelaySeconds: 0
            periodSeconds: 1
            timeoutSeconds: 1
            failureThreshold: 30

Keep periodSeconds short: with a 10-second period, a service that is ready at 1.1 s may not be marked ready until the next probe fires. Keep failureThreshold times periodSeconds comfortably above your p99 startup time, or a slow but healthy start turns into a crash loop.

Symptom, cause, lever#

What you seeLikely causeLeverCost
modules-loaded is most of the startup timeLarge import graph, many small filesLazy imports, bundling, narrower importsNone
Startup is slow and CPU-bound, but the markers are spread outModule compilation and JIT warm-up on a small CPU allocationStartup CPU boostExtra CPU billed during startup
listening is early but first requests are slowClients and pools created at module top level, or the TCP probe passing too soonLazy client getters, HTTP startup probeNone
startup_latencies is much larger than listeningScheduling, image start or sandbox costTry the other execution environmentNone
Cold requests appear in bursts, not only after idleConcurrency too low, so bursts need many new instancesRaise concurrency after a load testUsually lowers cost
Cold requests are rare but land on a login or checkout pathScale-to-zero on a path that cannot absorb itMinimum instances on that service onlyPaid while idle
Every deploy produces a wave of cold requestsNew revisions start fresh instancesDeploy with --no-traffic, then shift trafficNone
Table 1— What the measurements point to, and what to change

The rest of the guide works through the levers in that order, free ones first.

Lazy clients and a lighter import graph#

The classic mistake is creating every client at module top level. A Firestore client, a Pub/Sub client, a database pool and a secrets fetch each load a large dependency tree, and some open connections, before the server listens and for requests that may never need them. The Cloud Run general tips say the same about dynamic languages: module load time and global variable initialization both add to startup latency.

The fix is a memoized async getter. The module loads on first use through dynamic import(), the client is built once, and concurrent first callers share one promise:

src/clients/lazy.tsts
export function lazy<T>(factory: () => Promise<T>): () => Promise<T> {
  let instance: Promise<T> | undefined;
  return () => {
    if (!instance) {
      instance = factory().catch((err) => {
        instance = undefined; // allow a retry on the next call
        throw err;
      });
    }
    return instance;
  };
}
src/clients/index.tsts
import { lazy } from "./lazy.js";
import { env } from "../config/env.js";
 
export const getFirestore = lazy(async () => {
  const { Firestore } = await import("@google-cloud/firestore");
  return new Firestore({ projectId: env.GCP_PROJECT });
});
 
export const getPubSub = lazy(async () => {
  const { PubSub } = await import("@google-cloud/pubsub");
  return new PubSub({ projectId: env.GCP_PROJECT });
});

In my services every external client (document database, graph database, pub/sub, storage) lives in a module that exports a getter rather than an instance, and the rule I settled on is that startup code may validate but may not connect. Checking configuration costs microseconds; a TLS handshake to a database does not. Handlers call const db = await getFirestore(). The first request that touches Firestore pays for loading and connecting, which is still better than every cold start paying for every client, including the ones the health check and most routes never use. If one client is needed on nearly every request, start it right after listen() without awaiting:

src/server.tsts
await app.listen({ host: "0.0.0.0", port: Number(env.PORT) });
mark("listening");
void getFirestore(); // warm in the background; errors surface on first real use

Under request-based billing, CPU is only allocated while requests are in flight (and during startup), so that background warm-up may be throttled until the first request arrives. It still starts early; just do not count on it finishing.

Lazy initialization moves failures later: a wrong database URL used to crash the container and fail the deploy, and now fails the first real request. Validate environment variables eagerly at startup (that costs nothing) and leave only the connections lazy.

Beyond clients, the import graph itself matters:

  • Import narrowly. Pull the submodule you need rather than a whole SDK barrel when the library supports it.
  • Keep optional features out of the startup path. PDF generation, image processing and admin-only routes belong behind import() inside the handler.
  • Bundle for production. A bundler such as esbuild collapses thousands of files in node_modules into a few, which cuts filesystem lookups at boot. Mark packages with native bindings as external.
  • Profile it. node --cpu-prof dist/server.js locally, then open the profile in Chrome DevTools, shows which imports dominate.

Node 22 also has an on-disk module compile cache (module.enableCompileCache() or NODE_COMPILE_CACHE). Cloud Run's writable filesystem is in-memory and per instance, so a cache written at runtime is gone at the next cold start. It can only help if you generate it at build time and ship it in the image, which makes it an experiment to measure, not a default.

The image: smaller is hygiene, not speed#

It is tempting to shrink the image in the hope of a faster start. Cloud Run streams image content and the documentation states that image size does not affect container startup time. What does count is what your process reads and executes once it starts, so a slimmer image helps indirectly (fewer files to resolve, fewer vulnerabilities) rather than through download time. A multi-stage build with dev dependencies pruned is still worth having:

Dockerfiledockerfile
# ---- build ----
FROM node:22-slim AS build
WORKDIR /app
RUN corepack enable
COPY package.json pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile
COPY tsconfig.json ./
COPY src ./src
RUN pnpm run build \
 && pnpm prune --prod
 
# ---- runtime ----
FROM gcr.io/distroless/nodejs22-debian12:nonroot
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY --from=build /app/package.json ./
# distroless nodejs images use node as the entrypoint
CMD ["dist/server.js"]

Distroless has no shell, which shrinks the attack surface and removes docker exec as a debugging tool; keep a debug variant of the image or rely on logs and Cloud Trace. Alpine is small but uses musl, and prebuilt native Node modules sometimes lack musl binaries, which you discover at runtime. -slim Debian is the pragmatic middle if you need a shell. Container image hygiene covers the rest.

Platform settings that move the numbers#

tune-service.shbash
gcloud run services update api \
  --region=europe-west1 \
  --cpu-boost \
  --execution-environment=gen1 \
  --concurrency=80 \
  --max-instances=20

Startup CPU boost gives the instance extra CPU while it starts, then drops back to the configured amount. It helps the CPU-bound part of init: module compilation, JIT warm-up, TLS setup. It does nothing for time spent waiting on a network call to a secrets store. See the CPU documentation.

Execution environment. Cloud Run has two. Gen1 is documented as having faster cold starts; gen2 runs on a microVM with full Linux compatibility, generally faster sustained CPU and network performance, and is required for features such as network file system mounts. For a small latency-sensitive HTTP API gen1 is worth trying, and for CPU- or I/O-heavy work gen2 usually wins once running. Compare both on the startup_latencies chart instead of trusting either description.

Concurrency. Every request served by an existing instance is a cold start avoided. The default is 80 requests per instance and the maximum is 1000. Node services doing I/O-bound work often handle many requests at once, so concurrency set to 1 (a common copy-paste from single-threaded examples) multiplies instance count and cold starts during bursts. Going the other way, a CPU-heavy handler at 250 makes every request slow. Load-test with a realistic mix and pick the highest value that keeps p99 acceptable.

Billing mode. Instance-based billing (--no-cpu-throttling) keeps CPU available between requests, so post-listen warm-up and background work run reliably. It does not remove cold starts by itself. Check the billing settings page for the current naming.

Minimum instances: paying for warmth where it earns its keep#

Minimum instances keep instances warm:

terminalbash
gcloud run services update api --region=europe-west1 --min-instances=1

That removes cold starts for traffic that fits inside the warm instances. It does nothing for a spike that exceeds them, and it does not cover the first requests to a new revision unless you deploy with --no-traffic, wait for the new instances to come up, then move traffic with gcloud run services update-traffic.

An idle minimum instance bills continuously. Under request-based billing the idle rate is lower than the active rate, but it runs around the clock. Multiply one instance's vCPU and memory by roughly 730 hours and the idle rate on the Cloud Run pricing page, then by every service and every environment you set it on. One small instance is cheap; --min-instances=2 pasted across thirty services and two environments is not.

So make warmth a variable, set per service and per environment, and default it to zero:

infra/cloud_run.tfhcl
variable "min_instances" {
  description = "Warm instances for this service in this environment. 0 unless latency is worth the standing charge."
  type        = number
  default     = 0
}
 
resource "google_cloud_run_v2_service" "api" {
  name     = "api"
  location = "europe-west1"
 
  template {
    containers {
      image = var.api_image
      resources {
        limits            = { cpu = "1", memory = "512Mi" }
        cpu_idle          = true # request-based billing
        startup_cpu_boost = true
      }
    }
    scaling {
      min_instance_count = var.min_instances
      max_instance_count = 10
    }
    max_instance_request_concurrency = 80
  }
}

Production sets min_instances = 1 for the handful of services on the critical path of an interactive request, such as authentication and the main read API. Staging and preview environments stay at zero unless you are measuring latency there. Four questions decide each case:

  1. How many requests hit a cold instance? Count them from the request logs. For a service with steady traffic it may be a fraction of a percent; for one that idles between calls it may be most of them.
  2. Who experiences them? A cold start on a background webhook consumer costs nothing. One on a sign-in request has a conversion cost.
  3. What does removing them cost? The monthly arithmetic above.
  4. What is left after the free fixes? If lazy init took a cold start from three seconds to 600 ms, the case for paying drops sharply.

Take a hypothetical service with a 1.5 s cold start that is hit cold on 2 percent of requests, all internal: it does not justify a standing charge. The same service in front of a sign-in flow, where the first request of many sessions is cold, probably does, and one warm instance is likely the cheapest line in that decision. Write the numbers down instead of setting minimums everywhere "for performance". I wrote my own rule down: a service gets a warm floor when the monthly cost of the warm instance, divided by the sessions its cold starts affect, falls below what I would pay for a good first impression. One or two user-facing services have crossed that line and keep a warm instance in production, and the rest keep sleeping. I have also resisted scheduling fake traffic to keep instances warm, since that is just buying idle capacity with extra steps and worse observability.

Instances are also not kept warm forever by default. Cloud Run may keep idle instances for a while after traffic stops, but that is not a documented window, so do not design around "it stays warm for 15 minutes".

What scale-to-zero does not remove#

Scale-to-zero is a budget choice as much as a performance one. A system with a few hundred active users is idle most of the day, and idle capacity is billed per hour. Cloud Run services, Cloud Run functions and jobs, Firestore, Pub/Sub and Cloud Storage bill by use, so an idle system costs little more than stored data. The bill that surprises people comes from components beside them that do not scale to zero:

ComponentWhat keeps billingCheaper path where it fits
Cloud SQL, especially with high availabilityvCPU, memory and storage around the clock; HA adds a standbySmallest shared-core tier outside production; stop non-production instances; or Firestore if the data model allows
MemorystoreProvisioned capacityIn-process cache, or no cache until a measurement asks for one
External Application Load BalancerForwarding rules, hourly, before any trafficCloud Run's own URL or domain mapping until you need Cloud Armor, a CDN or multi-region routing
Cloud NATHourly gateway charge plus per-GiB processingOnly for an integration that truly needs a fixed egress IP
Serverless VPC Access connectorConnector VMs that run continuouslyDirect VPC egress, which needs no connector VMs
Minimum instancesIdle rate, every hourThe min_instances variable above, set only where measured
Table 2— Components that bill by the hour with no traffic

Multiplying by environments is what hurts. Dev, staging and production each carry their own copy of whatever has a floor, so a database, a load balancer and a NAT gateway are paid for three times. The practical rules: keep non-production environments free of always-on databases and load balancers, put max-instances on every service so a retry loop cannot scale without limit, and write a two-column cost model (idle floor, cost per extra thousand users) before building. In my own setup the managed graph database is an always-on instance and the largest fixed line, about half of the monthly bill, with stateful stores, storage buckets and a couple of scheduled jobs making up the rest of a small floor. Budget alerts are set at thresholds that would be boring in a normal month, so a misconfigured retry loop shows up in the invoice early, and every service carries a label that lets me group the billing console by component. Rebuild it from the pricing pages for your region; the shape of it is what matters, because the floor column does not move with usage and the unit column does.

Scale-to-zero stops being the right default in two places. A path with a tight latency budget, such as a synchronous callback with a short provider timeout, belongs on a warm instance. And a service that stays busy around the clock is usually cheaper on instance-based billing or committed capacity than on per-request pricing. In both cases move that one service off zero and leave the rest alone.

Share
All articles →

Batch work behaves badly behind an HTTP endpoint. Compare Cloud Run jobs and services on retries, parallelism and exit codes, and shard work across tasks.

12 min

Turn the Cloud Billing export into five queries that explain a Google Cloud invoice: what is fixed, what scales, what is a credit, and who owns each line.

9 min