JIT compilation

MatchBox includes a multi-tier JIT compiler built on Cranelift. It profiles hot code at runtime and compiles it to native machine code, eliminating interpreter dispatch overhead for the loops and functions that matter most.

Where JIT is Available

JIT is available in native builds with the jit feature, including the default CLI and native runners built with it. Browser/WASI and ESP32 builds use the interpreter. A browser's WebAssembly engine may optimize the module independently.

The Four Tiers

The JIT uses a tiered strategy: cheap profiling runs first, and compilation only fires once a site has proven itself hot.

Tier 1: Empty loop elimination

Trigger: A for loop whose body is empty, after ≥ 10 000 iterations.

The entire loop is replaced by a single select instruction: the counter jumps straight to its final value in one native call. No iteration actually occurs.

Tier 2: Numeric loop body

Trigger: A for loop with a numeric-only body, after ≥ 10 000 cumulative iterations (counter persists across fiber scheduler quanta).

The loop body is translated to Cranelift SSA IR. Local variables live in registers throughout; only the final values are written back to the VM's locals array on exit. Inline-cached struct member reads (monomorphic or bi-morphic) are also supported.

Tier 3: Array iteration

Trigger: A for-in loop over a float-only array, after ≥ 5 000 cumulative iterations.

The array data pointer and length are passed directly to the compiled loop, bypassing the iterator protocol entirely.

Tier 4: Hot function compilation

Trigger: A function body that passes a translatability check, after ≥ 100 calls.

The entire function body is compiled to a native fn(locals_ptr, heap_ptr, out_val_ptr) -> status function. Supported operations:

If any unsupported opcode is present, the function is silently skipped and keeps running in the interpreter with no correctness impact. If a type guard fails at runtime (for example a variable expected to be numeric turns out to be a heap pointer), the compiled frame deopts: execution transfers back to the interpreter at the current bytecode position.

Deopts are counted per function, and a function that keeps deopting is recompiled in a more tolerant mode (crates/matchbox-vm/src/vm/jit.rs, inc_fn_deopt):

Each upgrade drops the compiled code and sets the call counter to 10 below the threshold, so the function is recompiled after 10 more calls. Mode 0 is the first compile. Each step prints a [JIT] Tier-4 function ... deopted N times. Upgrading to Mode ... line to stderr.

What Is Not Supported in Tier 4

The following categories cause a function to remain interpreted:

• Global variables: GET_GLOBAL, SET_GLOBAL, DEFINE_GLOBAL, GET_PRIVATE, SET_PRIVATE
• I/O: PRINT, PRINTLN
• Object / collection creation: NEW, ARRAY, STRUCT
• Property and index access: MEMBER, SET_MEMBER, INC_MEMBER, INDEX, SET_INDEX
• Dynamic calls: INVOKE, INVOKE_NAMED, CALL_NAMED
• Loops inside functions: FOR_LOOP_STEP, LOOP, ITER_NEXT
• Exception handling: PUSH_HANDLER, POP_HANDLER, THROW
• Multi-word compare-jumps: LOCAL_COMPARE_JUMP, GLOBAL_COMPARE_JUMP, COMPARE_JUMP, LOCAL_JUMP_IF_NE_CONST
• Miscellaneous: INC, DEC, INC_LOCAL, INC_GLOBAL, SWAP, OVER, JUMP_IF_NULL

In practice, Tier 4 is most effective for pure computational functions: numeric kernels, string-building loops, recursive math, and hot helper functions that operate only on their arguments and local variables.

Diagnostics

The JIT reports through eprintln!, so its [JIT] ... lines (compiled hot fn, compiled empty loop, compiled generic loop, compiled iter loop, deopt upgrades and compile failures) go to stderr, not stdout. They are not controlled by --verbose.

Opting Out

To build the CLI without the JIT, leave out jit and also server. The matchbox_server crate enables jit on the VM itself, so with server on the JIT cannot be switched off (cargo tree -e features shows cranelift-jit for any feature set that includes server).

cargo build --release --no-default-features \
  --features "bif-io,bif-jni,bif-crypto,bif-cli,bif-http"

This drops --serve and --target wasi-http along with the JIT. The dependency tree for this command contains no cranelift-jit and no axum; it was inspected, not built.

Future Directions

JIT in native runner stubs

The build.rs feature-forwarding block (the features list at lines 92 to 108) explicitly lists which features are passed to matchbox-runner stub builds. Adding "jit" there would give deployed native runner stubs the same JIT benefits as the CLI's local interpreter. The trade-off is binary size: Cranelift would be linked into every stub, which multiplies across targets in cross-compile mode. When that trade-off becomes acceptable, the change is a one-liner in build.rs:

if env::var("CARGO_FEATURE_JIT").is_ok() { features.push("jit"); }

Wider Tier-4 opcode coverage

The most impactful missing opcodes for real-world functions are globals and property access. Both are achievable with runtime helpers following the same pattern as jit_concat and jit_ic_member_fallback. The main constraint is the deopt story: any operation that can throw or have side effects needs a clean deopt path back to the interpreter.

String constant support in Tier-4

Currently, CONSTANT only accepts numeric literals in compiled functions. String constants require the GC heap at JIT-compile time, which isn't available. One path forward: a jit_intern_const helper that lazily allocates the string object on first call and caches its GC id, making string constants safe to use from compiled frames.

Tier-4 for embedded loops

Functions containing FOR_LOOP_STEP are currently rejected by fn_is_translatable. Supporting them would require the Tier-2 loop compiler to operate within Tier-4's SSA frame, essentially merging the two tiers. The deopt model is the main complexity: a loop inside a compiled function that deopts mid-iteration needs to restore both the function's locals and the loop counter.