Commit Graph
121 Commits
Author SHA1 Message Date
CCLUCKY 572eb775e2 Fix: attach figure captions by adjacency, not nearest figure centre (#21448) 2026-10-29 19:47:09 +08:00
jay77721 fc766ce3d7 fix(parser): remove duplicate figures for extracted PDF tables (#21673) 2026-10-29 16:37:23 +08:00
CCLUCKY baf314d588 fix(deepdoc): keep body sentences that cite Table N out of table captions (#21453) 2026-10-29 16:36:17 +08:00
Yash Raj Pandey cefc0e59ca fix(deepdoc): add the first page height when a crop spans pages (#21428) 2026-10-29 13:53:09 +08:00
CCLUCKY 0c8af782cb fix(parser): read PDF bookmarks into remote parse results (#21499) 2026-10-28 15:28:30 +08:00
jay77721 1a6a382a2f fix(deepdoc): preserve table rows before cell assignment (#21489) 2026-10-27 11:55:17 +08:00
jay77721 bea5b62faa fix(ingestion): contain process-fatal errors and bound parser work (#21480) 2026-10-26 17:32:47 +08:00
jay77721 914bcb0307 fix(deepdoc): resolve conflicting caption labels by confidence (#21415) 2026-10-26 11:18:27 +08:00
jay77721 1b8ab33c2c fix(deepdoc): keep native PDF text when OCR detection misses it (#21399) 2026-10-26 11:17:28 +08:00
jay77721 651a1c65fc fix(deepdoc): crop clean media previews without context bands (#21412) 2026-10-23 19:09:58 +08:00
jay77721 fd675ebde2 fix(deepdoc): preserve complete PDF figures (#21390) 2026-10-23 17:06:46 +08:00
jay77721 17f173c288 refactor(parser): remove dead output_format readers (#21336) 2026-10-22 17:40:41 +08:00
xugangqiang 7e542f2c13 refactor(deepdoc): make model filenames, threshold, and post-processing overridable at runtime (#21275) 2026-10-21 16:48:08 +08:00
jay77721 f402c001be fix(parser): preserve PaddleOCR PDF highlight positions (#21205) 2026-10-20 15:46:31 +08:00
leilei3167 a87168bf3e fix(deepdoc): sample English pages deterministically (#20689) 2026-10-20 11:30:23 +08:00
xugangqiang 5cb4abb40a Remove RowsToHTML; render PDF tables via canonical RenderTableHTML (#21083) 2026-10-16 22:08:09 +08:00
jay77721 5753d7b27f fix(parser): keep short-line body pages out of PDF TOC removal (#20988) 2026-10-15 19:06:26 +08:00
jay77721 8efbe73da0 fix(chunker): preserve whole lines in PDF text previews (#20890) 2026-10-13 16:54:19 +08:00
lorenzozane e5ea8666ef fix(deepdoc): preserve PDF captions labeled as references (#20767) 2026-10-13 10:43:06 +08:00
xugangqiang 438df385d6 fix(deepdoc): clamp TSR/DLA boxes to image bounds at the analyzer exit (#20836) 2026-10-12 18:50:48 +08:00
jay77721 030d08ac5d fix(deepdoc): keep an attached figure caption on its own line (#20819) 2026-10-12 17:31:45 +08:00
xugangqiang 1788528f8d refactor(deepdoc): move testdata fetch into native and relocate script to tools/scripts (#20822) 2026-10-12 17:08:40 +08:00
Jin Hai 0e5ef65736 refactor (#20783)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-10-12 10:16:30 +08:00
ciel accddd32d3 fix(deepdoc): judge PDF header/footer and TOC removal against the full document instead of the parsed page subset (#20719) 2026-10-10 16:24:22 +08:00
jay77721 829aeb2f29 fix(deepdoc): stop PDF caption merge from dropping text and figures (#20650) 2026-10-10 13:34:48 +08:00
Yash Raj Pandey 6c3217240e fix(parser): honor the PDF page range for the VLM and plain_text methods (#20336) 2026-10-09 16:30:50 +08:00
jay77721 285b2c2903 fix(parser): restore picture OCR independently of VLM enhancement (#20424) 2026-09-30 14:32:48 +08:00
Jin Hai 60016b39c9 refactor: move res into internal (#20390)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-30 10:49:17 +08:00
xugangqiang 25cd847324 refactor(parser): move native PDF/Office extraction into deepdoc facade (#20273) 2026-09-29 17:26:27 +08:00
jay77721 31fd92c2fa feat(parser): add opt-in VLM enhancement for images (#20166) 2026-09-29 15:54:28 +08:00
xugangqiang 18d9750981 feat(config): make ingestor worker count (K) and page concurrency (N) configurable (#20121) 2026-09-29 14:19:24 +08:00
xugangqiang 52a0199c6e refactor(config): nest DeepDoc inference config under ingestor; add cpu-cores budget + fail-fast (#20084) 2026-09-29 14:19:17 +08:00
Jin Hai 72a541dd41 Go: refactor (#20054)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-29 14:17:54 +08:00
xugangqiang 02b668c1de fix(pdf): rejoin line-break hyphens / strip U+FFFD in Go layout (temporary workaround) (#20274) 2026-09-28 13:27:39 +08:00
xugangqiang bceb84bb4b fix(deepdoc): bucket table boxes by page in buildTableHTMLs (#20133) 2026-09-23 22:21:50 +08:00
xugangqiang f97c023050 fix(deepdoc): keep highest-indexed table in MarkNoMergeTables page scoping (#20125) 2026-09-23 20:41:52 +08:00
xugangqiang 483b892bb5 perf(deepdoc): bucket substring-dedup scan by page and column (#20122) 2026-09-23 20:41:00 +08:00
xugangqiang 8a3d26e78a fix(deepdoc): scope table/box overlap to the same page to avoid cross-page expansion (#20089) 2026-09-23 18:41:37 +08:00
xugangqiang 36e1f7cffb fix(parser): drop eager base64 table image from pdf.TableItem (#20051) 2026-09-23 12:55:42 +08:00
xugangqiang 3505e32135 fix(deepdoc): align OCR-rec batching with Python (sort by ratio, <=16, 96*1 pool) (#20041) 2026-09-23 11:22:14 +08:00
xugangqiang 203ae5e901 perf(deepdoc): add single-core PDF throughput + peak-RSS benchmark (#20034) 2026-09-23 10:27:38 +08:00
jay77721 328fadf1f8 fix(parser): replace slog.Debug with common.Debug+zap in toc_header_footer (#20071) 2026-09-23 10:16:00 +08:00
jay77721 dae8016532 fix(parser): catch tight page numbers and site-promo headers/footers (#20019) 2026-09-23 09:45:34 +08:00
xugangqiang cf3986174e feat(deepdoc): make inference concurrency configurable (default 4) (#20044) 2026-09-22 19:41:06 +08:00
jay77721 338703565a refactor(go): unify Go logging on the zap logger (#20026) 2026-09-22 19:40:08 +08:00
jay77721 82b45422e0 fix(ingestion, web): converge cancels fast and stop logging them as errors (#19992) 2026-09-22 14:02:34 +08:00
jay77721 12e6d888db feat(ingestion): report sub-component parsing progress instead of a frozen 20% (#19965) 2026-09-21 17:26:07 +08:00
jay77721 c779b15d9c feat(parser): upgrade PDF header/footer removal with 3D cascaded fusion (#19957) 2026-09-21 14:45:29 +08:00
xugangqiang 11c56bafc4 fix(deepdoc): route PDF parse logs through zap and drive colspan e2e in-process (#19947) 2026-09-21 11:17:58 +08:00
xugangqiang 3aba7b47a5 Fix: parse negative coordinates in @@ position tags (#19945) 2026-09-20 21:31:08 +08:00