changshenhan 598e7297eb perf(chunker): make hardSplitPiece O(L) with a single-encode token walk (#19712)

2026-09-16 17:28:01 +08:00

..

bpe_loader_anchors_test.go

fix(tokenizer): load cl100k BPE table from disk instead of failing silently offline (#17712)

2026-08-03 19:03:08 +08:00

bpe_loader_test.go

fix(deepdoc): harden ORT build guard, OCR test contract (#19082)

2026-09-02 12:13:19 +08:00

bpe_loader.go

fix(deepdoc): harden ORT build guard, OCR test contract (#19082)

2026-09-02 12:13:19 +08:00

failfast_test.go

Port agentic RAG PRs to GO (#19527)

2026-09-11 15:58:42 +08:00

message_fit_test.go

refactor(go): merge messagefit into internal/tokenizer (#19225)

2026-09-03 19:33:29 +08:00

message_fit.go

refactor(go): merge messagefit into internal/tokenizer (#19225)

2026-09-03 19:33:29 +08:00

tokenizer_concurrent_test.go

fix(tokenizer): load cl100k BPE table from disk instead of failing silently offline (#17712)

2026-08-03 19:03:08 +08:00

tokenizer_test.go

fix(tokenizer): load cl100k BPE table from disk instead of failing silently offline (#17712)

2026-08-03 19:03:08 +08:00

tokenizer.go

perf(chunker): make hardSplitPiece O(L) with a single-encode token walk (#19712)

2026-09-16 17:28:01 +08:00

usage.go

Fix: num token usage (#19495)

2026-09-10 17:55:17 +08:00