Keep the table outside model mappings and Metal residency. Use bounded explicit reads, reject failed lookups before GPU consumption, and remove the runtime sidecar option and paging controls. Preserve Qwen hash and speculative state handling; cover malformed files and exact recovery after prefill, decode, and MTP read failures.
Implement qwen4exp loading, tokenizer and chat formatting, PLE sidecars,
Metal text and vision graphs, built-in MTP, and recurrent KV persistence.
Integrate the CLI, agent, server, download targets, and conversion tools.
Preserve the native Q4_1 encoder, Q4_K accumulation guard, and shared
quantizer build target. Keep exact-MTP rewind and vision-cache row-width
fixes. Persist valid logits and recurrent state at completed prefill chunks,
and reuse Qwen text-answer history when tools clients omit reasoning.
Retain exact reasoning replay and test continued checkpoints, cancellation,
cache eviction, and restart reuse with and without reasoning_content. Include
reproducible charts and measurements comparing throughput before and after
the checkpoint fixes on M3 Ultra.
Add Qwen trunk steering and prompt activation capture, including GDN layers.
Keep the MTP predictor outside the 48-row bank and preserve prompt dumps
through one-token prefill tails and multi-row speculative verification.
Recover Chrome target creation and retain its last page target. Preserve
full agent transcripts in stripped-session format when KV no longer fits,
while retaining the newer automatic compaction and continuation logic.
Add steering/MTP/capture, Chrome CDP recovery, and full-session save/reload
regressions. Adapt the intended changes from fork PRs #13 and #7.
Reconcile with antirez main bd66c40, preserving DeepSeek v4.1 model IDs,
tool syntax, session state, and Metal kernel dispatch. Exclude superseded
engine code and prototypes.
Co-authored-by: Andreas Parusel <aparuael@7170c824-6152-4cf1-985e-9503832ce673.fritz.box>
Co-authored-by: david <contact@davidberard.fr>
Co-authored-by: Michael Kuebbeler <cropduster@users.noreply.github.com>
Co-authored-by: Ogilthorp3 <bertrand@nepveu.name>
Co-authored-by: anemll <realanemll@gmail.com>
Adapt teaching-mode PR #391 to /hints on|off. Append the full
instruction once per context and short state changes at safe
assistant boundaries. New and resumed sessions start off;
compaction reinstates the current policy.
Use ordinary Markdown asides and the existing renderer, without
a learner profile, new tool, or persistent preference.
Tested on M5 Max IT with DeepSeek Flash and GLM Flash coding sessions,
prefix reuse, queued changes, compaction, save/switch/new, frontend
and PTY tests, and agent ASAN/UBSAN checks.
Closes#391
Co-authored-by: Salvatore Sanfilippo <antirez@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Measure the runtime decode map before selecting the global Metal path, account its mapped bytes once, and fall back to per-layer mapping when it cannot fit. Treat explicit cache sizes as performance targets that may be reduced after model, graph, context, and platform reserves are known. Share the final guard-budget calculation between fitting and refusal on Metal and ROCm.
Implement GLM 5.3 Q2 and Q4 inference across Metal and CUDA, including tensor parallelism, RDMA, batching, MTP, snapshots, SSD streaming, and GLM 5.2 shared-path improvements.
Add quantization and validation tools, official quality fixtures, model downloads, regression coverage, release QA, and user documentation.
The new DS4F checkpoint struggles with anchored [upto] edits. Keep exact old/new replacement as the default and make [upto] prompting and automatic insertion an explicit --edit-upto opt-in.
Add ROCm SSD streaming support for PRO/Flash paths, including selected expert caching, overlapped reads, full-layer streaming prefill, resident cache lookups, and ROCm cache reuse handling. Add the corresponding CUDA SSD streaming expert cache backend and shared graph integration, with bounded cache allocation and progress reporting.\n\nThis squashes the rocm-streaming branch work on top of current main.
Add shared help text across the CLI, server, agent, bench, and eval tools. Expand distributed-mode guidance, clean up endpoint naming, and use a TTY-only 256-color layout with clearer section titles, option arguments, separators, examples, and explanatory text.
antirez
Donato Capitella
Donato Capitella
Salvatore Sanfilippo