Keep the table outside model mappings and Metal residency. Use bounded explicit reads, reject failed lookups before GPU consumption, and remove the runtime sidecar option and paging controls. Preserve Qwen hash and speculative state handling; cover malformed files and exact recovery after prefill, decode, and MTP read failures.
Implement qwen4exp loading, tokenizer and chat formatting, PLE sidecars,
Metal text and vision graphs, built-in MTP, and recurrent KV persistence.
Integrate the CLI, agent, server, download targets, and conversion tools.
Preserve the native Q4_1 encoder, Q4_K accumulation guard, and shared
quantizer build target. Keep exact-MTP rewind and vision-cache row-width
fixes. Persist valid logits and recurrent state at completed prefill chunks,
and reuse Qwen text-answer history when tools clients omit reasoning.
Retain exact reasoning replay and test continued checkpoints, cancellation,
cache eviction, and restart reuse with and without reasoning_content. Include
reproducible charts and measurements comparing throughput before and after
the checkpoint fixes on M3 Ultra.
Add Qwen trunk steering and prompt activation capture, including GDN layers.
Keep the MTP predictor outside the 48-row bank and preserve prompt dumps
through one-token prefill tails and multi-row speculative verification.
Recover Chrome target creation and retain its last page target. Preserve
full agent transcripts in stripped-session format when KV no longer fits,
while retaining the newer automatic compaction and continuation logic.
Add steering/MTP/capture, Chrome CDP recovery, and full-session save/reload
regressions. Adapt the intended changes from fork PRs #13 and #7.
Reconcile with antirez main bd66c40, preserving DeepSeek v4.1 model IDs,
tool syntax, session state, and Metal kernel dispatch. Exclude superseded
engine code and prototypes.
Co-authored-by: Andreas Parusel <aparuael@7170c824-6152-4cf1-985e-9503832ce673.fritz.box>
Co-authored-by: david <contact@davidberard.fr>
Co-authored-by: Michael Kuebbeler <cropduster@users.noreply.github.com>
Co-authored-by: Ogilthorp3 <bertrand@nepveu.name>
Co-authored-by: anemll <realanemll@gmail.com>
Implement GLM 5.3 Q2 and Q4 inference across Metal and CUDA, including tensor parallelism, RDMA, batching, MTP, snapshots, SSD streaming, and GLM 5.2 shared-path improvements.
Add quantization and validation tools, official quality fixtures, model downloads, regression coverage, release QA, and user documentation.
Add shared help text across the CLI, server, agent, bench, and eval tools. Expand distributed-mode guidance, clean up endpoint naming, and use a TTY-only 256-color layout with clearer section titles, option arguments, separators, examples, and explanatory text.
Add DeepSeek V4 Pro support on top of main.
- add Flash/Pro shape selection and Pro Metal inference support
- fix shared SwiGLU clamp and indexer QAT behavior for Flash and Pro
- update eval/context tooling and imatrix quantization for Pro
- tag KV caches/sessions with model ids so Flash and Pro state cannot cross-load
- remove misc/PRO.md from tracked files while leaving it as a local scratch log
Use a shared DS4_DEFAULT_MIN_P of 0.05 for sampled generation. The default keeps top_p at 1.0 and uses min_p as the active filter, preventing very low relative-probability tokens from entering the sampling set while preserving plausible alternatives.
Apply the same default to the CLI, server, and ds4-eval, expose --min-p in CLI/eval, and make the server thinking-mode fixed sampler use min_p=0.05 instead of disabling min-p filtering. Explicit non-thinking API min_p values continue to be honored.
Builds checked with make -B ds4 ds4-server ds4-eval. Server unit tests pass; full make test could not run because another DS4 process was already holding /tmp/ds4.lock.
Add routed-MoE imatrix collection to the DS4 graph path, including dataset generation docs and the tracked calibration prompt corpus.
Add a standalone DeepSeek V4 Flash HF-to-GGUF quantizer that reuses GGUF template metadata, emits DS4 Q2/Q4 recipes, supports imatrix-aware routed expert quantization, and internalizes the required Q8_0, Q4_K, Q2_K, and IQ2_XXS quantizers instead of vendoring GGML.
Document quantizer provenance, output formats, and the DS4-specific imatrix mapping used by the generated GGUF files.
antirez
Salvatore Sanfilippo
user
Codex