mtfyun/ds4
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
cd1c5507b660a2e0348520d34501dac1bdb2aafb
ds4/tests
T
History
antirez cd1c5507b6 Preserve resident Qwen prefill speed with speculative decoding 
Prepare predictor history by default for SSD streaming. When weights fit in RAM, initialize historical draft rows to zero and keep target verification enabled. Full prompt history remains available with DS4_QWEN4_MTP_PREFILL=on.

With 7,809 prompt tokens and 64 generated tokens on an M5 Max, the resident default took 6.47 s versus 7.53 s with eager history. Generated text matched previous main in both modes. The same policy recovered prefill throughput on the DGX Spark.
2026-11-03 16:50:59 +01:00
..
test-vectors
Add DeepSeek V4 Flash vision support
2026-09-01 00:11:58 +02:00
vision-fixtures
fix(jpeg): validate SOS spectral parameters in progressive scans
2026-10-30 17:47:37 +01:00
bench_mxfp4_rocm.c
MXFP4 prefill MoE: L2-friendly grid order + wider A-staging (r9)
2026-08-20 18:45:13 -07:00
compare_glm53_vision_embeddings.py
Fix GLM attention masking and reductions across GPU backends
2026-09-05 19:42:01 +02:00
cuda_long_context_smoke.c
cuda: warm up top-k regression timing
2026-06-02 09:07:28 +02:00
deepseek41_dspark_reference.py
Enable V4.1 DSpark on Metal, CUDA and ROCm
2026-10-09 15:07:43 +02:00
ds4_agent_terminal_test.py
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
ds4_agent_test.c
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
ds4_test.c
Merge #1056: Giorgio Oppo’s Qwen3.8 Metal SSD expert streaming
2026-11-03 16:28:10 +01:00
dspark_acceptance_fixture.sh
Record backend QA results and preserve failure diagnostics
2026-09-05 19:42:02 +02:00
generate_long_context_story_prompt.py
Add long-context story recall regression
2026-05-13 19:11:16 +02:00
glm_long_context_smoke.sh
Unify distributed inference CLI and documentation
2026-07-18 18:50:24 +02:00
long_context_security_prompt.txt
DS4 initial release
2026-05-07 17:16:28 +02:00
long_context_story_prompt.txt
Add long-context story recall regression
2026-05-13 19:11:16 +02:00
qwen4_ssd_pread_probe.h
Metal: extend Qwen M1 Max tuning to M2, M3 and M4
2026-10-01 08:04:59 +02:00
qwen4_vision_ref.py
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
run_glm53_vision_quality.py
Add DeepSeek V4 Flash vision support
2026-09-01 00:11:58 +02:00
test_agent_compaction.py
Read Qwen native BF16 n-grams directly from the GGUF
2026-09-14 13:32:48 +02:00
test_agent_vision.py
Read Qwen native BF16 n-grams directly from the GGUF
2026-09-14 13:32:48 +02:00
test_collect_official.py
Add official Qwen quality references and validated API scoring
2026-09-15 15:00:11 +02:00
test_compare_glm53_vision_embeddings.py
Fix GLM attention masking and reductions across GPU backends
2026-09-05 19:42:01 +02:00
test_cuda_attn_side.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_compressor.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_copy_spans.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_dspark_tp_markov.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_dspark_tp_split.c
Optimize Spark attention and support native MXFP4 DSpark experts
2026-10-05 14:17:41 +02:00
test_cuda_dspark.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_embed.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_f16_chunks.cu
Optimize Spark attention and support native MXFP4 DSpark experts
2026-10-05 14:17:41 +02:00
test_cuda_grouped_q8.cu
Use exact Q8 matrix kernels for CUDA TP prefill slices
2026-10-08 11:38:32 +02:00
test_cuda_hc_norm.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_hc_pre.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_indexed_one.cu
Optimize Spark attention and support native MXFP4 DSpark experts
2026-10-05 14:17:41 +02:00
test_cuda_indexer.cu
Fill single-token GLM indexer tensor tiles with heads on Spark
2026-10-31 17:33:15 +01:00
test_cuda_mixed_batch.c
DeepSeek v4.1 Flash support for CUDA
2026-09-13 17:32:48 +02:00
test_cuda_mxfp4_split.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_pool.cu
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_q8_0_gemv.cu
Optimize Spark attention and support native MXFP4 DSpark experts
2026-10-05 14:17:41 +02:00
test_cuda_q8_pair.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_q8_rows.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_q8_scratch.cu
Use aligned Q8 vector kernels for small Spark batches
2026-09-06 00:42:59 +02:00
test_cuda_reductions.cu
DeepSeek v4.1 Flash support for CUDA
2026-09-13 17:32:48 +02:00
test_cuda_router_stream.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_session_batch.c
DeepSeek v4.1 Flash support for CUDA
2026-09-13 17:32:48 +02:00
test_cuda_shared_side.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_shared.cu
Balance V4.1 shared experts across CUDA TP peers
2026-10-08 13:12:38 +02:00
test_cuda_ssd_batch.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_ssd_cache.c
Overlap CUDA SSD expert reads with V4.1 prefill
2026-09-14 00:00:54 +02:00
test_cuda_ssd_repack.cu
DeepSeek v4.1 Flash support for CUDA
2026-09-13 17:32:48 +02:00
test_cuda_tokentile.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_cuda_topk_stream.cu
Optimize Spark attention and support native MXFP4 DSpark experts
2026-10-05 14:17:41 +02:00
test_cuda_tp_link.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_tp_repack.cu
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_tp.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_cuda_v41_fusion.cu
Fuse V4.1 CUDA normalization and HC BF16 stores
2026-10-08 12:43:22 +02:00
test_cuda_verify_head.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_deepseek4_vision_image.c
Resize independent image rows in parallel before vision prefill
2026-10-31 20:28:10 +01:00
test_deepseek41_cli.c
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_conversion.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_dspark_conversion.py
Allow DSpark conversion from only the native draft shards
2026-10-09 13:18:45 +02:00
test_deepseek41_dspark.c
Enable DeepSeek V4.1 DSpark over Metal and CUDA TP
2026-10-10 13:10:32 +02:00
test_deepseek41_gguf.c
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_graph.c
Merge #1073 with local Metal and Spark optimizations
2026-11-02 11:12:11 +01:00
test_deepseek41_http.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_manifest.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_metal.c
Merge #1073 with local Metal and Spark optimizations
2026-11-02 11:12:11 +01:00
test_deepseek41_prefill.c
metal: process medium V4.1 SSD appends in one layer sweep
2026-10-30 17:55:43 +01:00
test_deepseek41_prompt.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_rope.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_vision_conversion.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek41_vision_image.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_deepseek_indexer.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_dspark_confidence_input.py
Optimize Spark attention and support native MXFP4 DSpark experts
2026-10-05 14:17:41 +02:00
test_dspark_eos_contract.py
Avoid full context rebuilds at DSpark response boundaries
2026-10-07 21:43:05 +02:00
test_dspark_rewind.py
Avoid full context rebuilds at DSpark response boundaries
2026-10-07 21:43:05 +02:00
test_dspark_window.py
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_engine_correctness.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_engine_mgpu_placement.c
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
test_engine_mgpu_refusal.c
Make release builds warning-free
2026-07-20 11:09:28 +02:00
test_engine_mgpu_runtime.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_engram.c
Overlap V4.1 disk n-gram reads with CUDA decode
2026-10-08 13:12:15 +02:00
test_glm53_kda.c
Tile single-host GLM 5.3 Flash indexed attention on Metal
2026-10-31 17:32:53 +01:00
test_glm53_vision_engine.c
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_glm53_vision_prompt.c
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_glm_attention.c
Reuse rotated compact keys across legacy GLM prefill queries
2026-10-31 20:31:53 +01:00
test_glm_reference_render.py
Add long GLM FP8 references with reasoning-aware scoring
2026-09-05 20:26:22 +02:00
test_glm_rope_prefill.py
Preserve RoPE in full GLM compact prefill
2026-09-06 12:00:03 +02:00
test_gpu_args_cli.sh
Extend ds4-eval with hard capability suite
2026-09-03 15:28:24 -07:00
test_gpu_args.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_gpu_lookup_cache_strict.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_gpu_model_cache.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_gpu_xdev.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_image_decode.c
fix(jpeg): validate SOS spectral parameters in progressive scans
2026-10-30 17:47:37 +01:00
test_jpeg_decode.py
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
test_layer_pack.c
Add CUDA tensor parallelism and session batching
2026-07-18 18:50:03 +02:00
test_linux_memory.c
Budget ROCm allocations against usable pinned memory
2026-09-05 19:41:28 +02:00
test_metal_command_memory.c
Test CPU-touched Metal buffer reclamation
2026-10-30 17:49:58 +01:00
test_metal_dense_mpp.c
Metal: stabilize Qwen HC arithmetic and extend M5 tuning to M6
2026-09-23 09:36:00 +02:00
test_metal_device_policy.c
Metal: extend Qwen M1 Max tuning to M2, M3 and M4
2026-10-01 08:04:59 +02:00
test_metal_moe_prefill.c
Keep M6 Qwen tuning separate from DeepSeek and GLM kernel selection
2026-11-03 16:41:21 +01:00
test_metal_q8_gemv_reference.m
Merge upstream/main into qwen-kernel-opt
2026-11-02 21:35:23 +01:00
test_metal_q8_reduction.m
Merge upstream/main into qwen-kernel-opt
2026-11-02 21:35:23 +01:00
test_metal_qwen4_hc.m
Metal: tune Qwen hyper-connection dispatches for M1 Max
2026-09-16 08:21:03 +02:00
test_metal_qwen4_moe_down.m
Merge antirez/main at 8db1d1d into qwen-kernel-opt
2026-09-16 19:51:07 +02:00
test_metal_qwen4_moe_half.m
Metal: optimize Qwen Q4 SSD prefill on M1 Max
2026-09-27 17:23:11 +02:00
test_metal_qwen4_moe_mid.m
Merge antirez/main at 8db1d1d into qwen-kernel-opt
2026-09-16 19:51:07 +02:00
test_metal_qwen4_mxfp4_stage.m
Metal: accelerate Qwen MXFP4 prefill and batched down rows
2026-09-27 10:24:37 +02:00
test_metal_session_batch.c
Harden Qwen Metal batch integration and quality checks
2026-09-16 14:14:36 +02:00
test_metal_ssd_experts.c
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_metal_ssd_reuse.m
Metal: plan Qwen SSD expert cache evictions once per batch
2026-09-15 09:19:02 +02:00
test_metal_tp_bulk.c
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_metal_tp_cancel.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_metal_tp_spec.c
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_mmvq.cu
Repair DSpark and ROCm regression tests
2026-10-06 17:48:04 +02:00
test_model_download.py
Add verified V4.1 Flash MXFP4 downloads and resumable assembly
2026-10-19 14:14:09 +02:00
test_mxfp4_cuda.cu
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_mxfp4_dot.c
Add MXFP4 GGUF format and scalar reference
2026-08-01 09:50:29 +02:00
test_mxfp4_metal.c
metal: make exact fast paths automatic and trim experiments
2026-08-09 17:48:01 +02:00
test_mxfp4_rocm.c
MXFP4 prefill MoE occupancy variants (env-gated, bit-exact, hw-verified)
2026-08-20 18:45:13 -07:00
test_mxfp4_staged_cuda.cu
Distribute and accelerate Flash speculative decoding on Spark
2026-10-05 14:17:41 +02:00
test_prompt_prefix.c
Add conversation prefix conditioning
2026-09-03 01:25:25 +02:00
test_q4k_dot.c
Keep Q4K regression test warning-free
2026-08-04 13:57:21 +02:00
test_q8_prefill_variants.c
Metal: optimize Qwen Q4 SSD prefill on M1 Max
2026-09-27 17:23:11 +02:00
test_quality_api.c
Score API top choices independently of alternative order
2026-10-06 18:35:04 +02:00
test_qwen4_checkpoint_replay.py
Read Qwen native BF16 n-grams directly from the GGUF
2026-09-14 13:32:48 +02:00
test_qwen4_cli_vision.py
Read Qwen native BF16 n-grams directly from the GGUF
2026-09-14 13:32:48 +02:00
test_qwen4_conv_parallel.c
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
test_qwen4_generation.c
Use resident Qwen arithmetic when experts are streamed from SSD
2026-11-03 16:37:20 +01:00
test_qwen4_kernels.c
Keep Metal-only Qwen predictor kernel tests out of the CUDA binary
2026-11-03 16:32:40 +01:00
test_qwen4_logit_dump.py
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
test_qwen4_memory.c
Qwen: honor Metal's SSD streaming memory budget
2026-09-26 15:35:21 +02:00
test_qwen4_moe_mm_compact.py
test(qwen): repair compact MoE checks after half-kernel extraction
2026-09-19 21:19:15 +02:00
test_qwen4_moe_mm_specialize.c
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
test_qwen4_mtp_limits.py
Qwen: prepare MTP prefix caches and preserve predictor state
2026-09-14 19:45:36 +02:00
test_qwen4_mtp_policy.c
Preserve resident Qwen prefill speed with speculative decoding
2026-11-03 16:50:59 +01:00
test_qwen4_mtp_prefill.c
Load Qwen version 2 checkpoints and rebuild missing predictor history on demand
2026-11-03 16:38:47 +01:00
test_qwen4_ngram_state.c
Merge antirez/main at 8db1d1d into qwen-kernel-opt
2026-09-16 19:51:07 +02:00
test_qwen4_ngrams.c
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
test_qwen4_ple_prefetch.c
Validate Qwen tokens before prefetched input staging
2026-10-30 18:10:33 +01:00
test_qwen4_prefill.c
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
test_qwen4_resident_arithmetic.c
Use resident Qwen arithmetic when experts are streamed from SSD
2026-11-03 16:37:20 +01:00
test_qwen4_ssd_experts.c
Metal: extend Qwen M1 Max tuning to M2, M3 and M4
2026-10-01 08:04:59 +02:00
test_qwen4_steering.py
Honor steering in standalone greedy Qwen generation
2026-09-14 13:41:49 +02:00
test_qwen4_vision_ref.py
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
test_qwen4_vision.c
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
test_rocm_memory.cu
Budget ROCm allocations against usable pinned memory
2026-09-05 19:41:28 +02:00
test_sampling.c
dspark: support exact stochastic speculative decoding
2026-08-23 15:05:13 +02:00
test_serve_concurrency_bench.py
Reject incomplete concurrency benchmarks and oversized requests
2026-09-16 15:02:09 +02:00
test_server_batching.py
Serve Qwen batched MTP with safe session state and output limits
2026-09-16 15:17:25 +02:00
test_server_speculative_stop.py
Avoid full context rebuilds at DSpark response boundaries
2026-10-07 21:43:05 +02:00
test_server_story.py
Add optimized CUDA inference for Qwen3.8 Flash Next
2026-09-15 15:00:11 +02:00
test_server_vision_agent.py
DeepSeek v4.1 Flash support for Metal
2026-09-12 11:47:54 +02:00
test_server_vision_cache.py
Preserve live vision context across server tool calls
2026-09-08 13:57:09 +02:00
test_server_vision_live_state.py
Validate image positions before token-prefix slot reuse
2026-09-18 23:27:51 +02:00
test_session_state.c
Cross the legacy GLM sparse boundary in one SSD layer sweep
2026-10-31 20:42:03 +01:00
test_ssd_cache.c
Use more of the safe Metal working set for DeepSeek expert caching
2026-09-06 12:19:01 +02:00
test_tp_commands.c
Avoid full context rebuilds at DSpark response boundaries
2026-10-07 21:43:05 +02:00
test_tp_link.c
Keep V4.1 TP sharded when either Mac needs the smaller working set
2026-11-02 11:17:02 +01:00
test_tp_rdma.c
Stop tensor-parallel inference after a transport failure
2026-10-05 14:17:41 +02:00
test_tp_tcp.c
Add two-host CUDA tensor parallelism for DeepSeek V4 Flash
2026-10-05 14:17:41 +02:00
test_v41_indexer_score.c
Share DeepSeek V4 Metal index queries across four keys
2026-11-02 11:36:48 +01:00
test_web_recovery.c
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00
test_web_recovery.py
Add Qwen3.8 Flash Next model support
2026-09-13 14:57:29 +02:00