Cold Starts, Warmups, and Prewarming Strategies

A cold start is not one mysterious pause before useful work. It is a chain of state transitions: capacity is assigned, a process or isolate begins, code and dependencies load, runtime profiles and caches are empty, connections are established, and application state is prepared. Which transitions occur depends on the platform and on what is already resident. Calling all of them “startup” hides both the owner and the remedy.

The central thesis is that cold-start engineering should model explicit states and optimize the user-visible critical path, not chase an undifferentiated startup number. Sometimes the correct answer is a smaller dependency graph. Sometimes it is lazy initialization, a snapshot, a minimum pool of warm instances, or accepting a slower first request because prewarming costs more than the latency is worth. The decision must connect a measured arrival pattern and service-level objective to the lifecycle of actual instances.

This article uses a scale-to-zero document-rendering function as a worked example, but the model also applies to command-line tools, autoscaled containers, language workers, desktop applications, and machine-learning services.

Model Startup as a Critical Path #

Begin at the boundary the caller experiences. For an on-demand service, response time can be decomposed as

T_queue is delay waiting for capacity. T_provision covers the platform’s allocation and sandbox or container setup. T_runtime includes executable loading and runtime initialization. T_application covers module evaluation, configuration, clients, and application caches. T_request is the operation that would still occur on a fully ready instance. Some stages overlap, and a platform may conceal parts of them, but the decomposition produces testable questions.

Draw the dependency graph rather than adding every timer. Loading configuration may run alongside importing an independent template bundle. Opening a database connection may begin only after credentials resolve. The user-visible cost is the longest dependent path, not the sum of all work performed. Parallelizing two stages helps only if they were on that path and do not create contention for the same CPU, disk, or network.

Define at least three states:

There may also be “restored” instances created from a snapshot and “stale warm” instances whose credentials, DNS answers, or caches need refresh. Do not collapse restored into warm until measurement shows equivalent behavior.

Measure distributions for each state. A median cold start does not reveal image-fetch stalls or dependency timeouts in the tail. Record cold frequency as well as cold duration: a severe cold path that affects one request per month has a different priority from a modest delay imposed after every burst.

Trace Ownership Across the Startup Lifecycle #

Instrument boundaries with a monotonic clock and a trace identifier that survives from the platform entry point to the response. Emit events for module-load completion, configuration readiness, client construction, first connection, cache fill, handler entry, and first byte. Where platform logs expose allocation or restore timings, correlate them rather than guessing the invisible interval.

Startup instrumentation must itself be lightweight. Importing a large tracing SDK before recording the first event can become part of the problem. A tiny bootstrap marker followed by full telemetry after initialization preserves the early timeline. Keep clocks on one host when possible; wall-clock subtraction across machines can introduce skew.

At fleet level, classify every request by instance state and reuse count. Useful dimensions include deployment version, region, memory or CPU size, artifact size, trigger type, tenant, and concurrent work. An instance identifier lets you reconstruct a lifecycle: created, initialized, served requests, refreshed state, failed, and retired. Without that sequence, repeated cold starts caused by crashes can look like healthy autoscaling.

Separate application-controlled time from provider-controlled time. You may not be able to reduce sandbox allocation directly, but artifact size, memory tier, regional placement, and minimum capacity can influence it. Module evaluation and eager network calls are usually application-owned. A trace that ends with “init: 900 ms” gives neither team a useful contract.

Watch resource profiles during initialization. CPU-starved startup can make parallel imports slower. Decompression can trade artifact transfer for CPU. Memory spikes can cause an instance to fail before readiness. Network calls can serialize on DNS or connection limits. The same stage name can therefore have different bottlenecks under different capacity settings.

Work a Cold-Start Budget End to End #

Imagine an on-demand function that renders an invoice PDF. It receives a request, loads templates and fonts, fetches tenant branding, creates a renderer, and stores the result. The product requires ordinary warm responses to finish within 600 ms and allows cold responses up to 1,500 ms at p95. The following numbers are illustrative budget inputs, not observed benchmark results:

Naively summing gives 1,410 ms. If the branding fetch starts after credentials load and overlaps 80 ms of template preparation, the modeled critical path becomes 1,330 ms. That leaves 170 ms of p95 headroom. It does not guarantee the SLO because percentiles of stages cannot generally be added to obtain a percentile of the total; the table is a design budget that must be verified end to end.

Now examine frequency. Suppose requests arrive in short daytime bursts with long overnight gaps. Keeping one instance warm all day may remove the first-request penalty, but much of the reserved time is idle. Instead, the team could schedule minimum capacity shortly before the known burst, use a snapshot after deterministic initialization, or accept the first cold response while caching its generated invoice if repeat access is common.

The worked decision should compare complete alternatives:

1. Trim eager dependencies: remove an unused image codec and defer a rarely used font pack. This reduces every new instance’s load and memory.
2. Snapshot deterministic state: restore parsed templates and initialized renderer tables, while reopening external connections after restore.
3. Scheduled prewarming: create capacity before the predictable business window, then scale it down after arrivals cease.
4. No prewarming: preserve scale-to-zero economics and route the first request through a UI that honestly represents document preparation.

The best option can be a composition. Dependency trimming has low recurring cost and benefits every path. A snapshot removes repeatable CPU work. Scheduled prewarming handles the known burst without paying for the entire day. The remaining cold path stays visible in monitoring rather than being declared solved.

Reduce Initialization and Dependency Work #

Profile import and initialization time by module or phase. Large packages are not automatically slow; cost comes from bytes transferred and decompressed, files opened, code parsed or compiled, module side effects, and objects allocated. A small module that performs synchronous filesystem discovery or a network call during import can dominate a large inert data package.

Keep the bootstrap graph narrow. Move command-specific or route-specific dependencies behind explicit dynamic imports. Split optional format handlers so a CSV request does not initialize PDF support. Replace broad convenience imports with supported focused entry points when the package provides them. Remove duplicate dependency versions and generated assets that the runtime never reads.

Lazy initialization is useful when the deferred feature is uncommon and the first user of that feature can afford its cost. It is harmful when every request immediately touches the deferred object, because it merely moves startup into the handler and may make concurrent requests race to initialize it. Protect shared lazy state with a single in-flight promise, clear failed state so retries are possible, and expose the delay as a named span.

type Renderer = { render(input: Invoice): Promise<Uint8Array> };

let rendererPromise: Promise<Renderer> | undefined;

async function getRenderer(): Promise<Renderer> {
  rendererPromise ??= createRenderer().catch((error) => {
    rendererPromise = undefined;
    throw error;
  });
  return rendererPromise;
}

export async function handle(invoice: Invoice): Promise<Response> {
  const [renderer, branding] = await Promise.all([
    getRenderer(),
    loadBranding(invoice.tenantId),
  ]);
  const bytes = await renderer.render({ ...invoice, branding });
  return new Response(bytes, { headers: { 'content-type': 'application/pdf' } });
}

The pattern deduplicates concurrent initialization and overlaps independent work. It still needs deadlines and cancellation around external calls, and it must not cache tenant-specific authorization in process-global state. Readiness should remain false until components required by normal traffic are usable; otherwise the load balancer can turn initialization into customer-visible failures.

Use Caches and Snapshots Without Freezing Bad State #

Startup caches exist at several layers: artifact caches, operating-system page caches, runtime code caches, parsed configuration, DNS and connection state, application data, and JIT profiles. Name the layer when claiming an instance is warm. Running a no-op request may load code without populating the template or connection path that real traffic needs.

Snapshots can capture memory after deterministic initialization and restore it for new instances. They are strongest when initialization is expensive, repeatable, and independent of machine identity or current credentials. They can reduce repeated parsing and object construction, but restore is not free: snapshot bytes must be stored, transferred, mapped, and sometimes fixed up.

Classify state before snapshotting:

Snapshot creation must be tied to the exact code, runtime, architecture, and configuration schema it represents. Treat the snapshot as a build artifact with compatibility checks and a fallback to ordinary initialization. Test restore after credentials rotate and after long storage intervals. A fast restore that serves stale policy or duplicates randomness is a correctness defect, not a performance success.

External caches also shift tradeoffs. A shared cache can survive instance churn but adds a network round trip and a consistency policy. A local cache is faster but starts empty and multiplies memory across instances. Cache only data with explicit keys, freshness, size limits, and invalidation behavior. Preloading an unbounded tenant catalog makes startup proportional to fleet history and turns each autoscaling event into a load spike on the source system.

Choose Prewarming by Arrival Pattern and Cost #

Prewarming creates or exercises capacity before demand needs it. The main strategies are minimum resident instances, scheduled capacity, event-driven prediction, and synthetic warm-up requests. Each buys lower cold probability by spending compute, complexity, or both.

Let C_idle be the cost per instance-time, N_warm(t) the warm capacity held at time t, and V_cold the business cost assigned to a cold response. A simplified decision compares

with

Neither side is purely an infrastructure invoice. V_cold can reflect SLO penalties, abandonment, delayed jobs, or no meaningful harm at all. Estimates should be ranges because arrival forecasts and business impact are uncertain.

Minimum capacity is simple and protects unpredicted arrivals, but it pays continuously and may still be too small for a burst. Scheduled prewarming suits known traffic windows, but calendars drift and deployments can invalidate warmed instances. Predictive warming can follow richer patterns but needs forecast evaluation, conservative bounds, and a fallback when the prediction misses. Synthetic requests must exercise the relevant path without generating side effects, corrupting analytics, or bypassing normal authorization.

Prewarming also moves load earlier. Starting hundreds of instances simultaneously can overwhelm configuration, secrets, databases, or shared caches. Ramp capacity with jitter and bounded concurrency. A warm instance that has never opened its constrained downstream connection may simply transfer the cold spike to the database at the first real request.

Anticipate Failure Modes at Fleet Scale #

An optimization that improves one-instance startup can worsen fleet behavior. Eagerly opening connection pools may reduce first-request latency but multiply idle database connections during scale-out. Preloading data can create a synchronized read storm. Increasing memory may grant more CPU on one platform and improve boot time, yet raise cost and reduce packing density. Measure the entire dependency system while changing instance policy.

Deployment waves are a common cold-start amplifier. Replacing every instance at once discards warm code, connections, and caches. Roll gradually, preserve old capacity until new instances pass readiness, and limit simultaneous initialization. A bad release may crash during startup, causing a retry loop that looks like aggressive autoscaling. Alert on initialization failures and instance churn, not only request latency.

Other recurring failure modes include warming the wrong route, allowing credentials captured at initialization to expire silently, treating background initialization as complete before it is safe, and measuring a local process that bypasses the provider’s allocation path. Warm-up traffic can also pollute business metrics or trigger billable third-party calls. Give it an explicit identity and side-effect-free contract.

Cold-start mitigation can hide architectural coupling. If every new worker must synchronously contact six control services, keeping workers alive reduces symptoms but leaves recovery fragile. Define degraded startup: which cached configuration can be used, how old it may be, and which features remain unavailable when a dependency is down. Faster initialization is valuable; bounded, understandable initialization is more valuable.

Test, Roll Out, and Operate the Cold Path #

Create a repeatable cold-path test that starts from an absent process and, when possible, an absent platform instance. Run it many times because provisioning and artifact fetches vary. Test warm reuse separately, then test transitions: first request after deployment, first request after idle eviction, snapshot restore, credential rotation, dependency failure, and concurrent arrivals during initialization.

Verify correctness alongside timing. Assert that one-time initialization occurs once under concurrency, snapshots do not reuse unsafe state, caches obey tenant boundaries, and failed initialization can recover. Inject delays and errors into configuration, DNS, secrets, and downstream connections. Confirm that readiness, deadlines, and telemetry describe the partial state accurately.

Roll changes to a small region or traffic cohort. Watch cold frequency, cold and warm latency distributions, instance reuse, initialization failures, memory, CPU, downstream connection counts, cache load, and cost per completed request. A lower cold p95 accompanied by more warm failures or a database connection surge is not a clean improvement. Compare equivalent arrival windows because cold frequency is workload-dependent.

Keep a startup manifest with artifact identity, phase spans, snapshot version, warm-up action, minimum-capacity policy, and rollback. Revisit it when the runtime, dependency graph, traffic shape, or platform changes. Cold starts are not a one-time tax; they are behavior emerging from a lifecycle.

The practical goal is not to make every instance permanently warm. It is to decide which state each user request may encounter and make the transition explicit, bounded, and economically justified. Trace the critical path, remove needless eager work, snapshot only safe state, warm capacity according to arrivals, and test the fleet effects. Then cold starts become an engineering budget rather than an unpredictable pause.