Benchmark history
Preview 19 — 31 October 2026
Five-round release measurements pass on macOS ARM64 and Windows x64 using the qualified runtime payloads. The macOS host was under development load; these are bounded workload measurements, not a controlled speedup comparison. Exact reports and host conditions. The notice-only archive refresh preserves the measured executable bytes.
Recorded runs remain available as neoCLR evolves. Compare results only when the workload, machine and measurement settings match; a newer date alone does not make two runs comparable. These first Preview 14 measurements are exploratory baselines, not established performance regression gates.
2026-10-26 — Preview 18
Final macOS ARM64 (with PostgreSQL) and Windows x64 packages pass the same five-stage release measurement suite. Each validates 55 CPU/JSON processes and 1,152 HTTP responses. Retain startup, ranges and slower Windows results; its CPU model differs from the prior hosted run, so no controlled speedup is claimed. macOS evidence · Windows evidence. The earlier base-only macOS candidate remains separate historical evidence.
2026-10-13 — Preview 14, macOS ARM64
Apple M1, macOS 27.0, with desktop and indexing activity. Every recorded case passed its output checks. Reports retain all rounds, startup timings and input hashes.
| Workload series | Measurement | Results and evidence |
|---|---|---|
| HTTP plain text v1 | GET, 10-byte UTF-8 response, one client, connection close | Report · Raw run |
| HTTP object JSON v1 | Construct and serialize a Person per request | Report · Raw run |
| HTTP PostgreSQL + JSON v1 | Parameterized SELECT, object mapping and serialization; reused connection | Report · Raw run |
| CPU sum v1 | Sum integers 0 through 9,999 per operation | Report · Interpreter run · Native run |
| JSON nodes serialize/deserialize v1 | Nested Unicode document, separate serialization and parsing cases | Report · Raw run |
These are separate series, not interchangeable measures of runtime speed. The SQL AOT case links the shared native host; other HTTP AOT cases use the standalone HTTP profile. General execution uses the console profile. Host and GC policies matter.
The initial native CPU batches were shorter than one millisecond. They remain in the initial raw run as pilot data; the report uses the subsequent 10,000-operation native batches. Interpreter CPU and JSON cases use 16-operation batches. This measurement-setting difference is explicit and must be retained in later comparisons.
2026-10-13 — development GC comparisons
The GC benchmark report keeps distinct paired series: fixed-fixture macOS JIT/AOT, newer streaming-library macOS AOT, and Windows x64 JIT/AOT. Each links immutable raw runs, including slower candidate samples. Only the root observer changes within each pair. These are development experiments, not new Preview 14 release measurements or cross-platform rankings.
The later Windows GC-versus-wait series compares legacy sleep and readiness waits on the same VM, with separate diagnostics. It retains all rounds and the earlier independent profiling run. Its absolute rates must not be compared against a different hosted VM's root-observation results.
How history will grow
Keep raw runs immutable, including slower results. Corrections and replacement runs will link back to the original evidence. Changes to payloads, queries, operation semantics or timing protocols start a new comparable series. Runtime implementation changes retain the workload identity so their effect can be measured.
Our goal is stable or improving performance with correct behavior. Establishing a regression gate requires repeated measurements on a controlled host and thresholds based on observed noise. A small change in one short run is not proof of an improvement or regression. No automatic statistical acceptance gate is active yet.
See the benchmark history policy for provenance and comparison requirements. Windows x64 GC comparisons are now recorded separately above; Windows ARM64 and sustained cross-runtime measurements remain future work. A local development comparison now retains Kestrel, Go and Node pilot results; publication is on hold pending neoCLR HTTP improvements.
The dense object-graph series is a separate collection-only native runtime benchmark. It includes chain, wide, free-hole and sparse-buffer controls; it must not be combined with HTTP throughput or cross-platform ratios.
The allocation-throughput series compares complete validation plus separate search against a combined walk. Its allocation-only timings exclude final collection; the linked pacing experiment remains separate and is not a shipping or HTTP performance claim.
The host-handle admission series compares separate validation/membership walks with a combined walk. Early/late-base and small-heap controls remain visible. It measures root creation/read/release, not collection time; its separate AOT HTTP comparison shows no material gain.
2026-10-22–23 — Preview 17
Both versioned packages pass 55 CPU/JSON processes and 1,152 HTTP responses. macOS evidence and Windows evidence retain input identities, sample distributions and Python-cache-only Git status. These desktop/hosted runs are separate from previous development and Preview 16 series.