MarMar Labs 2f13f103a0 fix(token_utils): build the tiktoken encoder on first use, not at import (#18986)

2026-09-04 15:46:30 +08:00

..

dsl_examples

Support pipeline DSL modification through dataset configuration (backend) (#16991)

2026-07-17 14:40:09 +08:00

client.py

Refa:replace trio with asyncio (#11831)

2025-12-09 19:23:14 +08:00

test_parser_pdf_positions.py

fix(chunker): carry overlap-head PDF positions into new chunk (#18148) (#18227)

2026-08-13 22:18:33 +08:00

test_title_chunker_position_int.py

feat(title-chunker): configurable token-count cap with sentence-boundary re-split (#18455)

2026-08-18 21:03:51 +08:00

test_title_chunker_token_cap.py

fix(chunker): keep PDF positions on all cap-split sub-chunks so chunks don't lose their preview images (#18772)

2026-08-27 11:26:49 +08:00

test_token_chunker_delimiter.py

fix(token_utils): build the tiktoken encoder on first use, not at import (#18986)

2026-09-04 15:46:30 +08:00

test_token_chunker_tag_overlap.py

fix(chunker): unify TokenChunker merge and strip coord tags in Python JSON path (#18002)

2026-08-07 21:55:07 +08:00

test_token_chunker.py

fix(chunker): carry overlap-head PDF positions into new chunk (#18148) (#18227)

2026-08-13 22:18:33 +08:00