JIT Compilers: Speculation, Optimization, and Deoptimization

A dynamic program gives its runtime information that source code alone cannot provide. A call site may receive the same function millions of times. A property read may repeatedly see objects with one layout. A numeric loop may use only small integers even though the language permits strings, large numbers, accessors, proxies, and exceptions. A just-in-time compiler can turn those observations into fast machine code, but observations are not promises.

The governing idea is: a JIT converts likely runtime facts into guarded assumptions, then preserves correctness by returning to generic execution whenever an assumption stops holding. Optimization and deoptimization are therefore one design. Inlining without an exit, or unboxing without a reconstruction map, would make the runtime fast only for programs that never surprise it.

This article begins after the language already has executable semantics, typically an interpreter or quickly generated code. It does not revisit lexing, type checking, whole-program intermediate representations, object files, linking, or the ahead-of-time compiler pipeline. Its subject is the adaptive loop that observes one running program, specializes hot paths, and safely changes its mind.

Gather Evidence in the Lower Tiers #

Most tiered runtimes start execution with an interpreter or a baseline compiler. The first tier must become ready quickly and handle every legal value. While doing that generic work, it records compact feedback that can justify later specialization.

Useful feedback includes function and loop invocation counts, value categories, object layouts, call targets, branch frequencies, allocation behavior, and which exception paths execute. A property-access site might record that its receiver has shape S1 and that property price lives at a particular offset. An addition site might record small integers on both sides. A call site might record one target function.

Feedback belongs to a program location, not merely to a variable name. Two reads of item.price can observe different populations because they occur in different callers or branches. Runtimes commonly associate feedback slots with bytecode offsets or baseline-code sites. The exact representation varies, but the contract is the same: later compilation can ask what this operation has actually encountered.

Collection has costs. Counters add instructions, feedback tables consume memory, and updating them can create contention or barriers. Sampling reduces overhead but provides less exact site information. Runtimes use thresholds and compact states because compiling every function eagerly would spend time and memory on initialization code, error paths, and features that execute once.

Hotness is a heuristic, not value. A frequently called trivial function may not repay an expensive optimization. A moderately hot loop with costly dynamic checks may. Tiering policy therefore combines counts with code size, observed types, current optimization queue, and estimated benefit. Some runtimes also tier down or discard code under memory pressure.

Build Inline Caches at Dynamic Operations #

An inline cache accelerates an operation by remembering how it succeeded for recent receiver or target classes. Despite the name, it is not a general data cache. It is a small dispatch mechanism attached to a dynamic site such as a property read, method call, global lookup, or arithmetic operation.

Consider order.total. A fully generic lookup may need to inspect the receiver, account for proxies or accessors, walk prototype links, resolve the property, and choose the correct semantics. If prior receivers all share a stable layout and total is a plain field at offset 24, the site can check that layout and load directly from offset 24. On a miss, it calls a generic stub that performs full lookup and updates cache state.

Cache states are often described as:

These labels describe a common design, not fixed universal limits. A polymorphic site is not inherently bad: a small stable union of layouts can remain efficient. Megamorphism becomes costly when no compact specialization covers the population, and it can contaminate callers if aggressive inlining duplicates the dispatch.

Object shape is also an implementation concept, not source identity. Construction order, conditional fields, prototype changes, framework wrappers, and deserialization paths can produce distinct layouts for objects that appear similar in source. Keeping construction consistent can help a runtime, but application code should not depend on undocumented shape rules for correctness.

Inline caches serve two roles. They speed lower-tier execution immediately, and their accumulated states become evidence for the optimizing tier. A monomorphic call cache can justify inlining one target. A stable property cache can justify folding a lookup into a guarded memory load. This is how observations at individual sites become larger cross-operation optimizations.

Turn Observations into Guards and Optimized Code #

An optimizing compiler starts with the function’s executable representation plus feedback. It creates specialized paths under assumptions such as “this call target is calculateTax,” “these operands are small integers,” or “this receiver has shape S1.” Every assumption that is not guaranteed by language semantics needs a guard or another invalidation mechanism.

For a property read, pseudocode might look like:

if receiver.shape != S1:
    deoptimize(reason = wrong_shape)
value = load(receiver + price_offset)

The load is cheap because the expensive semantic alternatives moved behind the guard. Guards also enable downstream reasoning. Once a value is guarded as a number, arithmetic can use an unboxed representation. Once an array’s storage kind and bounds are guarded, repeated element operations can avoid generic dispatch. If a call target is guarded and inlined, constants and ranges can flow across the former call boundary.

Speculation composes. One optimized loop may rely on the receiver shape, element representation, integer range, absence of certain side effects, and a stable call target. That can produce excellent code for the common path, but every additional dependency creates a reason to exit or invalidate the code. The compiler must weigh saved work against guards, code size, compilation time, and expected stability.

Some assumptions are checked locally on every execution. Others are protected by dependencies registered with runtime metadata. If a prototype or global binding changes, the runtime can invalidate optimized code that assumed the old state before it executes again. This avoids repeatedly checking facts that change rarely, at the cost of maintaining precise dependency lists.

Inlining is especially powerful because it removes dynamic calls and exposes context, but it has limits. Duplicating a large callee increases code-cache pressure and compile time. Inlining a call site with many targets needs several guarded versions or a generic fallback. Recursive and deeply nested inlining can make deoptimization metadata large. A mature JIT uses budgets rather than treating “more inlining” as an unconditional improvement.

Work Through a Speculative Price Loop #

Consider a JavaScript-like function that sums item prices after applying a numeric multiplier:

function totalPrice(items, multiplier) {
  let total = 0;
  for (let index = 0; index < items.length; index += 1) {
    total += items[index].price * multiplier;
  }
  return total;
}

During early executions, the generic tier handles all legal behavior. Suppose feedback then shows a packed array, item objects with one shape, numeric price fields, a numeric multiplier, integer indices, and totals that remain in the runtime’s efficient integer range. The loop becomes hot enough to optimize.

The optimized version can conceptually perform these steps:

guard items is the expected packed-array kind
guard multiplier is a number
total = unboxed_integer(0)
index = 0
loop:
    guard index < current_length
    item = direct_element_load(items, index)
    guard item.shape == ItemShape
    price = direct_field_load(item, price_offset)
    guard price is a number
    product = multiply_with_overflow_check(price, multiplier)
    total = add_with_overflow_check(total, product)
    index = index + 1
    jump loop
return box_number(total)

The exact checks depend on language rules. Array access might invoke unusual behavior in some cases; numbers might require floating-point representation; multiplication can produce a value outside a tagged-integer range. The key is that the fast loop is valid only while every relevant assumption holds.

Now pass an item whose price is exposed through an accessor, or whose shape differs because a field was added in another order. The shape guard fails before the direct field load. The runtime exits optimized code and resumes a generic tier at the same logical operation. Generic lookup invokes the accessor if required, preserving semantics. Later feedback may lead to a polymorphic specialization for both common shapes, or the runtime may decide the site is too unstable to optimize further.

A different surprise occurs when arithmetic exceeds the specialized integer range. The objects can retain the expected shape while an overflow guard fails. Deoptimization must carry the mathematical value into a representation that generic execution understands, then continue without repeating visible side effects. This distinction is why deopt reasons are more useful than a single “optimization failed” counter.

The example is not advice to rewrite objects until a particular runtime keeps this loop optimized. It demonstrates the proof structure: feedback suggests a narrow fast path, guards verify its preconditions, and the generic semantics remain available for every other case.

Enter and Leave Optimized Code Safely #

Tiering need not wait for a function to return. A long-running loop might become hot during its first invocation. On-stack replacement, or OSR, lets the runtime transfer execution from a lower-tier loop position into optimized code. The compiler creates an entry mapping from current locals and operand-stack values into the optimized frame’s expected representation.

Deoptimization performs the reverse, but it can be harder because optimization has changed program structure. Functions may have been inlined, values kept unboxed, objects represented only as scalar fields, and source-level variables proven redundant. To resume generic execution, the runtime needs metadata at legal exit points describing how to reconstruct one or more logical frames.

A deoptimization record can specify:

• The bytecode or generic-code position where execution should resume.
• How machine registers and stack slots map to logical local variables.
• How to box specialized integers or floating-point values.
• Which inlined call frames must reappear and their arguments.
• How to materialize an object that optimization had kept only as fields.
• Which values are constants or can be recomputed without visible effects.

These records are usually tied to safepoints or statepoints where runtime state is sufficiently described. Garbage collection needs related maps to identify live references while optimized code runs. Exception handling also needs a route from machine positions through inlined logical frames to the correct handler.

Deoptimization can be eager: a guard fails and transfers immediately. It can also be lazy or invalidation-driven: a dependency changes, optimized code is marked unusable, and execution exits at an appropriate boundary. Terms differ among runtimes, but correctness requires invalid code to stop relying on the obsolete assumption before it can observe a forbidden state.

Side effects make resume positions delicate. If a getter, write, or external call already occurred, generic execution must not repeat it. Compilers place exits before effects, after effects with complete state, or use continuation metadata that identifies the exact logical point. Testing rare exits around exceptions and mutation is consequently central to JIT correctness.

Balance Tiering, Code Memory, and Runtime Work #

Optimized code is not free performance. Compilation consumes CPU and memory, often on background threads. Generated code occupies executable memory and instruction caches. Feedback tables, dependency records, and deoptimization maps consume additional space. A program can finish before optimization repays its cost, or generate so many specialized versions that code-cache pressure slows the system.

Tiering tries to spend compilation effort where expected future savings exceed that cost. Baseline code favors quick generation and compactness. Intermediate tiers may apply inexpensive local optimizations. The highest tier uses richer analysis only for sufficiently hot, stable code. Thresholds can adapt when the compilation queue grows or the code cache approaches a limit.

Reoptimization needs restraint. One wrong-shape exit may reveal a second stable case worth including. Recompiling after every new shape can create a deoptimization loop: optimize narrowly, encounter another case, exit, compile again, and repeat. Runtimes track exit reasons, widen representations, use polymorphic dispatch, raise thresholds, or temporarily blacklist unstable sites.

Code eviction introduces another lifecycle. Optimized code for inactive functions may be discarded while bytecode or baseline forms remain. Later calls warm again. This is normal resource management, but it complicates latency: a service with a vast route surface may continually compile and evict cold handlers. Code-cache occupancy and compilation time can matter as much as the speed of one optimized function.

Garbage collection interacts with speculation too. Optimized frames must expose live references exactly, and write barriers cannot disappear unless analysis proves them unnecessary. Allocation elimination may keep an object virtual, but deoptimization or identity-observing behavior can force materialization. A fast path that violates collector invariants would corrupt memory, so collector barriers and safepoints are part of the optimization contract.

Recognize Performance Cliffs and Misread Signals #

Adaptive optimization creates nonlinear behavior. A stable monomorphic site may be fast until one new receiver shape appears. A loop may switch tiers mid-run. An integer accumulator may deopt only on large data. These cliffs are legitimate consequences of specialization, but explanations need evidence from the runtime rather than folklore about source syntax.

Common instability patterns include megamorphic property or call sites, alternating numeric and nonnumeric values, prototype mutations that invalidate dependencies, repeated OSR and deopt in one loop, enormous generated functions, and high compilation queues. Framework-generated wrappers can make a call site diverse even when business inputs look uniform. Conversely, a deoptimization count is not automatically a problem; a rare exit may be cheap relative to the optimized work it enables.

Optimization status is transient. Debuggers, profilers, diagnostic flags, coverage instrumentation, and runtime versions can change tiering or generated code. A trace that says a function optimized once does not prove requests are currently spending time there. Correlate tier and deopt events with CPU samples, latency, input class, and code-cache state.

Avoid source-level cargo cults such as deleting try blocks, forcing every object into one construction pattern, or replacing clear built-ins based on advice for an old runtime. Implementation heuristics change. First choose sound algorithms and stable data contracts; investigate JIT details only when representative profiles show that adaptive behavior is material.

The boundary with JIT benchmarking is important. Runtime traces can explain warm-up and deoptimization, but they do not by themselves establish a speedup. Timing experiments still need isolated processes, representative inputs, correctness checks, and statistical treatment. Internals explain a mechanism; a benchmark establishes its effect.

Verify Semantics and Diagnose Adaptive Behavior #

JIT implementers test optimization as a semantic equivalence problem. Run programs in the interpreter or lowest tier and in optimized tiers, then compare results, exceptions, and visible side effects. Force tier-up and deoptimization at many legal points. Fuzz value types, object shapes, prototype changes, overflows, accessors, exceptions, and garbage collections. A wrong answer after a rare exit is more serious than a missed optimization.

IR and machine-code verifiers can assert representation, control-flow, safepoint, and reference-map invariants before code executes. Differential testing against another engine can find discrepancies, though language ambiguities and shared bugs require careful reduction. Stress modes that compile early, deopt frequently, move objects, or disable selected tiers expose assumptions hidden by ordinary thresholds.

Application engineers should use a narrower operational loop:

1. Reproduce a user-visible latency or CPU symptom with representative input classes.
2. Capture a sampling profile to establish that the suspected function matters.
3. Use the exact runtime version’s diagnostics to inspect tiering, inline caches, compilation, and deopt reasons in a controlled run.
4. Map frequent exits back to data or mutation patterns without changing semantics merely to satisfy the optimizer.
5. Test the smallest justified change across cold, warm, common, and uncommon inputs.
6. Recheck end-to-end latency, CPU, memory, compilation time, and correctness after rollout.

Keep diagnostics scoped because verbose tracing can expose source names or data and can perturb timing. Preserve runtime version, flags, workload seed, and event logs with the investigation. If a runtime upgrade removes the issue, verify the upgrade against the full application rather than carrying an obsolete workaround forward.

A JIT is best understood as a reversible argument. Runtime evidence says a narrower implementation is likely to work; guards check that claim; optimized code exploits it; metadata explains how to return when it is false. Inline caches connect observed dynamic operations to that argument, while tiering limits where compilation effort is spent. Once deoptimization is seen as a normal correctness path rather than a failure, adaptive compilation becomes less mysterious: it is fast code built with an explicit, testable way to be wrong about the future without ever being wrong about the program.