Development documentation · Some APIs may be unreleased Published release →

Guides

General execution benchmarks

Preview 19 — 31 October 2026

Five-round release measurements pass on macOS ARM64 and Windows x64 using the qualified runtime payloads. The macOS host was under development load; these are bounded workload measurements, not a controlled speedup comparison. Exact reports and host conditions. The notice-only archive refresh preserves the measured executable bytes.

Allocation-pattern samples compare low-allocation work, object churn, bursts and retained windows under sync and concurrent GC. The standard procedure supports separate benchmark and profiling runs.

Development: lazy JIT — 26 October 2026

Post-Preview-18 source builds can opt into lazy method lowering and compilation. Fresh production-default-codegen measurements on Apple M1/macOS 27.0.1 separate process readiness, first use and steady execution. Each standalone row is the median of three launches; all 24 invocations pass.

Sample / GC Eager median ready s Lazy median ready s
Compute / sync 2.489 2.465
Compute / concurrent 2.548 2.585
Allocation / sync 2.494 2.350
Allocation / concurrent 2.605 2.515

Small-program startup is mixed. These samples leave 23 selected methods uncompiled; metadata preparation still consumes much of startup. The HTTP tables show larger readiness gains, but also substantial first-request latency. Lazy JIT now uses bounded arena placement on macOS ARM64; matched placement controls improve HTTP throughput by 22–30% in paired medians. Eager and other-host policies are unchanged.

The application-load report compares complex handlers, downstream I/O, cold traffic and multiple apps. Lazy concurrent GC remains the intended direction where it improves broad workload coverage; current first-call blocking, checking overhead and CPU costs direct the next optimizations. Shared metadata and execution bookkeeping improvements can help all modes.

The earlier five-round lazy baseline and initial factorial profiles use a private Cranelift speed override. Their results remain separate historical development evidence. Current raw results, ranges and profiling record the production-default codegen series. Eager JIT and synchronous GC remain defaults; published release tables below are unchanged.

Preview 18 — 26 October 2026

Both packages pass all five measurement stages: 55 CPU/JSON processes and 1,152 HTTP responses per platform. SDK/editor 0.1.18-neoclr.20261025.1, Raven ed1d08def, synchronous GC. Five measured rounds retain ranges; startup is separate. Native CPU uses longer batches, and HTTP uses sequential loopback requests with connection close. These results do not measure peak concurrency, allocation rates or collector comparisons.

macOS ARM64

Apple M1/macOS 27.0.1; the final archive includes PostgreSQL. Git stayed clean and no unrelated builds overlapped timing. Ordinary desktop activity remained present.

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 4.94 2.59–5.06 3.266
cpu.sum jit 39392.19 38837.47–41515.34 3.576
cpu.sum aot 40466.71 34296.61–41589.11 0.006
json.serialize interpreter 12.94 11.85–13.37 3.366
json.serialize jit 973.35 956.30–1021.27 3.360
json.serialize aot 1035.81 959.68–1066.23 0.005
json.deserialize interpreter 13.12 12.96–14.14 3.548
json.deserialize jit 654.97 619.80–681.49 3.122
json.deserialize aot 670.02 602.18–712.59 0.006

Final macOS reports. The earlier base-only candidate is retained separately.

Windows x64

Windows Server 2025 on AMD Family 25 Model 1. The prior series used Model 17; CPU-control and HTTP results are lower in this hosted run. The host difference prevents attributing the change solely to the runtime and does not rule out a runtime regression. Only generated Python caches changed during measurement.

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 5.42 4.99–5.56 5.607
cpu.sum jit 43724.45 43270.52–43834.43 5.355
cpu.sum aot 43544.85 42850.26–43785.18 0.011
json.serialize interpreter 10.23 10.05–10.30 5.605
json.serialize jit 347.94 342.47–349.52 5.387
json.serialize aot 355.84 343.95–359.20 0.011
json.deserialize interpreter 10.11 10.04–10.13 5.627
json.deserialize jit 244.62 242.83–246.85 5.356
json.deserialize aot 251.45 250.76–252.58 0.011

Windows reports and conditions. No controlled before/after or cross-platform speedup is claimed.

Preview 17 — 22–23 October 2026

The matching macOS ARM64 and Windows x64 packages each pass all five stages: 55 CPU/JSON processes and 1,152 HTTP responses. SDK/editor version 0.1.17-neoclr.20261022.1, Raven 301e668a5. Five measured rounds retain their ranges; startup is separate. HTTP uses sequential loopback requests with connection close. Native CPU rows use longer batches than interpreter/JSON rows. Desktop and hosted-runner conditions prevent controlled speedup or cross-platform claims.

macOS ARM64

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 5.05 4.98–5.08 4.426
cpu.sum jit 41917.63 41337.75–42001.13 3.626
cpu.sum aot 41831.46 41122.51–41852.21 0.007
json.serialize interpreter 14.40 14.38–14.45 4.433
json.serialize jit 725.64 721.61–729.68 3.574
json.serialize aot 699.13 693.42–766.44 0.007
json.deserialize interpreter 11.86 11.26–11.88 4.439
json.deserialize jit 523.88 518.46–524.37 3.573
json.deserialize aot 516.92 511.69–549.59 0.007

Raw macOS reports and conditions.

Windows x64

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 6.37 6.18–6.62 6.050
cpu.sum jit 54422.27 51174.01–54535.53 5.165
cpu.sum aot 54349.18 46513.73–54590.73 0.010
json.serialize interpreter 12.34 12.08–12.53 5.943
json.serialize jit 366.93 349.91–367.73 5.172
json.serialize aot 367.67 350.12–369.09 0.010
json.deserialize interpreter 9.85 8.44–10.08 5.964
json.deserialize jit 269.52 254.45–271.13 5.176
json.deserialize aot 270.52 250.29–272.16 0.010

Raw Windows reports and conditions. Both runs record only generated Python caches in their initial Git status; no tracked input changes occurred. Earlier series below retain their original provenance and limitations. These results do not measure collector comparisons, peak concurrent throughput or HTTPS performance.

Earlier measurements

CPU and JSON workloads run through the same Raven worker in interpreter, JIT and AOT. These 18 October 2026, Apple M1/macOS 27.0 measurements use the Preview 16 package with Raven c0854cdda. All 45 CPU/JSON processes and ten longer native CPU processes pass correctness and unchanged-input checks. Windows results follow below as a separate hosted-runner series.

Complete SDK development refresh — 22 October 2026

The complete macOS ARM64 development SDK passes the entire sequential benchmark runner: 55 CPU/JSON processes and 1,152 HTTP responses, with unchanged-input checks. Apple M1/macOS 27.0; Raven compiler 301e668a5. This is a development candidate, not a published release or a controlled comparison with the earlier snapshot.

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 5.06 4.69–5.23 4.405
cpu.sum jit 42206.91 42046.77–42822.33 3.565
cpu.sum aot 42007.71 41375.75–42155.22 0.007
json.serialize interpreter 14.63 14.33–14.67 4.391
json.serialize jit 720.73 670.15–727.86 3.553
json.serialize aot 707.77 421.02–776.19 0.007
json.deserialize interpreter 11.95 11.50–11.99 4.377
json.deserialize jit 523.30 519.97–529.01 3.562
json.deserialize aot 514.11 511.07–519.37 0.007

Five measured rounds rotate modes. CPU/JSON uses 16 operations and two warmups; native CPU rows use a separate 10,000-operation, 20-warmup run. Each HTTP process validates 32 responses, discarding the first eight from timing; each mode also has one whole-process warmup. Requests are sequential loopback HTTP/1.1 with connection close, using a Python client on the same machine. Startup is separate and includes JIT compilation. No unrelated local builds or full suites overlapped timed workers; desktop and OS activity remained present. These results do not establish controlled speedups, peak concurrency, allocation rates, GC comparisons or HTTPS throughput.

Candidate identity and raw reports. The earlier development measurements are preserved. Windows qualification and the matching performance run have passed; results follow below.

Windows development refresh — 22 October 2026

Windows x64 hosted runner, Windows Server build 26100, AMD64 Family 25 Model 1. All five aggregate stages pass: 55 CPU/JSON processes and 1,152 HTTP responses. This series uses the package qualified by run 39841298887 (original package run 39835531991), with benchmark source 69441f2fd. It is separate from macOS and published Preview 16 measurements; host differences prevent platform rankings.

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 5.08 4.48–5.34 7.527
cpu.sum jit 42245.41 41560.15–42727.94 6.219
cpu.sum aot 43193.06 43100.68–43290.72 0.012
json.serialize interpreter 9.60 8.22–9.98 7.634
json.serialize jit 281.55 198.75–287.40 6.376
json.serialize aot 251.35 195.55–285.68 0.013
json.deserialize interpreter 7.47 6.74–7.77 7.685
json.deserialize jit 205.44 136.87–213.24 6.765
json.deserialize aot 199.89 184.79–216.20 0.014

Five rounds rotate modes. CPU/JSON uses 16 operations and two warmups; native CPU uses a separate 10,000-operation, 20-warmup series. HTTP validates 32 responses per process, discards the first eight from timing and uses one whole-process warmup per mode. Requests are sequential loopback HTTP/1.1 with connection close. Startup is separate from throughput and includes JIT compilation. These results do not measure peak concurrency, allocation/GC rates, HTTPS throughput or production load.

The aggregate reports a dirty checkout. This is retained, not relabeled clean: an audit of recorded source hashes matches 85 tracked inputs at the recorded revision (exact bytes or Windows CRLF checkout). Two generated native archives are hashed separately and have no tracked-source equivalent. The raw report does not retain the dirty path list, so its cause is not established. Every measurement's input-integrity and correctness checks pass; no controlled speedup is claimed.

Candidate identity, source audit and raw reports.

Published Preview 16 series

Workloads and results

One CPU operation sums integers 0 through 9,999 and checks 49,995,000. JSON operations serialize a prepared node tree or parse a fixed nested document containing Unicode, objects, arrays, numbers, Boolean and null. Serialization checks the complete output; parsing checks root shape/count and name. Setup verifies a full round trip.

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 6.26 6.11–6.43 3.316
cpu.sum jit 39423.95 39416.56–39559.80 2.770
cpu.sum aot 39560.05 38478.95–39694.88 0.008
json.serialize interpreter 3.81 3.78–3.85 3.363
json.serialize jit 438.32 423.51–441.45 2.723
json.serialize aot 462.62 445.82–464.27 0.008
json.deserialize interpreter 4.17 4.10–4.20 3.347
json.deserialize jit 341.29 329.64–345.89 2.770
json.deserialize aot 357.27 346.74–361.41 0.008

Windows x64

The same package passes on Windows Server 2025 (AMD Family 26 Model 2), with 55 CPU/JSON processes and 1,152 HTTP responses validated across the full run. Shared hosted-runner load is uncontrolled; these numbers are a separate series, not a comparison with the macOS machine.

Workload Mode Median operations/s Range Median startup s
cpu.sum interpreter 5.34 5.25–5.48 3.857
cpu.sum jit 71621.49 69438.18–72756.32 3.345
cpu.sum aot 79799.16 75297.18–85303.27 0.010
json.serialize interpreter 4.28 3.73–4.51 3.831
json.serialize jit 379.43 346.25–386.29 3.220
json.serialize aot 376.71 359.75–388.06 0.010
json.deserialize interpreter 4.28 4.12–4.39 3.995
json.deserialize jit 284.26 278.49–289.74 3.328
json.deserialize aot 281.36 267.93–285.82 0.010

Windows raw reports and package identity

Measurement boundaries

Five fresh-process rounds rotate execution modes. Interpreter CPU and JSON use 16 operations and two warmups per process. Native CPU uses 10,000 operations and 20 warmups, avoiding the sub-millisecond batches of the retained 16-operation pilot. Startup includes worker registration, JSON setup, metadata loading and JIT compilation, even for the CPU case; it excludes compilation and warmup batches.

External batch timings include callbacks, correctness checks, console/pipe markers and reader scheduling. They do not isolate serializer CPU time. No builds ran concurrently, but macOS indexing was active. These exploratory workstation results are not a controlled regression threshold, language ranking or platform comparison. Peak memory, allocation rate and deployment size are not measured here.

Raw samples, package identity and reproduction details · Benchmark framework · Earlier measurements

HTTP workloads · All benchmark reports