Read one next layer into reserved slots of the admitted expert cache using two 8 MiB staging buffers. Protect current inputs, publish only completed copies, keep unused read-ahead cold, and join before cancellation or cache/model teardown.
Enable read-ahead from 2K tokens; smaller prefills were slower with full-layer reads. Add eviction, truncated-source, buffered-I/O, cancellation and early-exit tests. Validate exact logits and saved state on the Sparks, including Compute Sanitizer.
Add bounded SSD streaming, exact short prefills, accelerated large prefills, and native session batching. Support resident tensor parallel inference across two DGX Sparks over RoCE, with aligned expert shards, split vocabulary projection, decode graphs, and shared-expert overlap.
Include primitive, state, transport, and model regression tests, plus usage documentation and measured QA results.
Salvatore Sanfilippo