Commit Graph
87 Commits
Author SHA1 Message Date
xugangqiang e5a51b680b Provision native libs from versioned user cache (offline CI, drop /opt pre-bake) (#22099) 2026-11-09 18:07:31 +08:00
xugangqiang ed7510638e Revert "Provision native libs from versioned user cache (offline CI, drop /opt pre-bake)" (#22097)
Reverts infiniflow/ragflow#22023
2026-11-06 16:33:19 +08:00
xugangqiang d6dbfd97d4 Provision native libs from versioned user cache (offline CI, drop /opt pre-bake) (#22023) 2026-11-06 14:48:00 +08:00
Wang Qi 543f57ea97 Revert "CI: add check for ASCII, disallow none ASCII code check-in" (#21873)
Reverts infiniflow/ragflow#21784
2026-11-02 20:12:22 +08:00
Wang Qi e13fe445e5 CI: add check for ASCII, disallow none ASCII code check-in (#21784) 2026-11-02 11:30:22 +08:00
buua436 738a239650 fix: harden credential and input handling (#21593) 2026-10-29 14:03:56 +08:00
Wang Qi d717d0c975 Refactor: delete agent browser (#21599) 2026-10-28 15:44:29 +08:00
xugangqiang 0410c4a266 refactor(parser): drop ParseResult.OutputFormat/Markdown, gate on Produced() (#21414) 2026-10-23 19:11:10 +08:00
dependabot[bot] 1f96325523 build(deps): bump certifi from 2026.1.4 to 2026.10.19 in /tools/es-to-oceanbase-migration (#21404) 2026-10-23 15:50:22 +08:00
xugangqiang 6a2ef7ec6b refactor(deps): rename ragflow_deps directory to deps (#21345)
## Summary

Rename the repository dependency-resource directory `ragflow_deps/` to
`deps/`, and update every hardcoded reference to that directory name so
the rename does not break resource loading at build/runtime.

This is not a purely cosmetic rename: the Go runtime hardcodes the
directory name as a resource search path (e.g.
`internal/common/model_assets.go` trimming the `deps/` prefix, and the
tokenizers joining `deps/` into candidate paths), and the Dockerfiles
copy assets into `/ragflow/deps`. Those literals must move together with
the directory, otherwise assets fail to resolve.

## Two intentional exceptions (kept unchanged)

The rename deliberately keeps two names that only *coincidentally*
contained the old string:

- The published image tag `infiniflow/ragflow_deps:latest` (used by
`Dockerfile`, `Dockerfile_base`, docs, and CI `image inspect` checks).
- The CI install path `/opt/ragflow_deps` inside the CI runner image
(`build.sh` `SYSTEM_DEPS_TOKENIZER`).

## Changes

- `git mv ragflow_deps deps`.
- `build.sh`: `CL100K_TABLE_RELPATH`, native asset zip path, and related
references updated to `deps/`.
- `tools/scripts/download_dev_deps.py`: `RAGFLOW_DEPS_DIR` points at
`deps/`.
- Go runtime resource search paths (`internal/common`,
`internal/tokenizer`) updated to the `deps/` layout.
- `Dockerfile` / `Dockerfile_base`: container asset path changed to
`/ragflow/deps`; kept tag `infiniflow/ragflow_deps:latest` unchanged.
- Docs / `AGENTS.md` / CI workflow paths updated to the new directory
name.

## Test plan

- Go unit tests in `internal/common` and `internal/tokenizer` updated to
the new `deps/` layout and pass via `bash build.sh --test`.
- Python download scripts import and build URLs correctly (`py_compile`
clean).
- Added `tools/scripts/test_dep_rename_invariants.py` as a regression
guard asserting the two intentional exceptions (`/opt/ragflow_deps` and
`infiniflow/ragflow_deps`) stay intact, so a future blanket rename
cannot silently regress them.

---

## Follow-up fix (commit 2f6857095d)

The original rename used a blanket `sed s/ragflow_deps/deps/g` and only
selectively reverted `build.sh`. That left two intentionally-kept names
rewritten, which this follow-up restores:

- CI install path `/opt/ragflow_deps` — `sep-tests.yml` had been changed
to read `/opt/deps`, breaking the offline cl100k table fallback (the
cached-image path pointed at a directory that never receives the file).
Restored to `/opt/ragflow_deps`.
- Published image tag `infiniflow/ragflow_deps` — left as bare
`infiniflow/deps` in `sep-tests.yml` comments/echo, `.dockerignore`, and
`Dockerfile` / `Dockerfile_base` error messages. Restored to
`infiniflow/ragflow_deps`.

The regression guard `tools/scripts/test_dep_rename_invariants.py` was
added to prevent recurrence.
2026-10-22 19:20:42 +08:00
jay77721 17f173c288 refactor(parser): remove dead output_format readers (#21336) 2026-10-22 17:40:41 +08:00
jay77721 11ca59d511 refactor(ingestion): retire ck_type in favor of layout and doc_type (#21259) 2026-10-21 17:01:20 +08:00
xugangqiang 9cca46ac1c Fix(deps): use onnxruntime-v1.29.0-patch2 to restore Clip@13 in minimal ORT (#21051) 2026-10-20 13:34:46 +08:00
xugangqiang b44e852e6b Build: add multi-platform native build support (linux/arm64, darwin) (#19247) 2026-10-16 14:26:31 +08:00
xugangqiang eabad0c74c refactor(parser): type ParseResult.JSON as []DocItem boundary adapter (#20998) 2026-10-15 20:15:40 +08:00
xugangqiang c754300366 refactor(deps): split dependency download into two Python scripts (#20861) 2026-10-13 20:44:56 +08:00
lorenzozane aa8185155a fix(es-to-oceanbase): preserve array values during JSON encoding (#20695)
Fixes #20624
2026-10-12 19:20:43 +08:00
xugangqiang 1788528f8d refactor(deepdoc): move testdata fetch into native and relocate script to tools/scripts (#20822) 2026-10-12 17:08:40 +08:00
Jin Hai 9620ccab1b refactor: remove unused scripts (#20813)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-10-12 14:46:45 +08:00
Jin Hai 60016b39c9 refactor: move res into internal (#20390)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-30 10:49:17 +08:00
Wang Qi 7cd4718a0b Refactor: better message - canvas -> agent (#20395) 2026-09-30 09:36:03 +08:00
Wang Qi 1be9590a4a Docs: v0.27.2 -> v1.0.0-rc1 (#20338) 2026-09-29 13:20:25 +08:00
YanZhang fa0f3c29ab docs: update docs with go (#20062) 2026-09-29 10:35:03 +08:00
Ritik Yadav de3944b675 fix(tools): guard temporary table creation during mysql_migration dry run (#19570) (#19623)
Fixes #19570
2026-09-28 10:48:56 +08:00
YanZhang 9e82c9f01b fix: resolve go CLI latest release without GitHub API (#20159)
Resolve the latest release through GitHub’s `releases/latest` redirect
instead of the rate-limited REST API. Explicitly specified versions
continue to work as before.
2026-09-24 10:58:29 +08:00
dependabot[bot] ce0cd5b8b9 chore(deps): bump anyio from 4.12.1 to 4.14.2 in /tools/es-to-oceanbase-migration (#20104) 2026-09-23 15:26:53 +08:00
Zhichang Yu 44b8d366ba feat(go,web): port tokenizer counters, embedding limits and multi-provider web search (#19802) 2026-09-18 19:11:28 +08:00
Sbaaoui Idriss 9ee43ae795 fix(beartype): remove package-wide import hooks (#19531)
### Summary

Removed unnecessary beartype hooks from three package imports. This
prevents unwanted import-time instrumentation and dependency coupling.
based on #15258
2026-09-14 10:04:11 +08:00
Liu An 5bd80b3455 Docs: Update version references to v0.27.2 in READMEs and docs (#19501)
- Update version tags in README files (including translations) from
v0.27.1 to v0.27.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
2026-09-10 18:35:02 +08:00
balibabu 68959037f3 Fix: Install only oxlint and oxfmt to ensure CI checks for web files. (#19197) 2026-09-03 16:33:42 +08:00
MarMar Labs f3ba170764 fix(migrations): parse the --config flag the usage line documents (#18817) 2026-08-31 14:35:42 +08:00
Jin Hai d37486e295 Python && Go: fix parse status (#18841)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-28 12:47:41 +08:00
Liu An e3cc9a1621 Docs: Update version references to v0.27.1 in READMEs and docs (#18849) 2026-08-28 12:04:37 +08:00
05c67ca854 Fix OceanBase chunk order field persistence (#16997)
## Summary

Add chunk_order_int to OceanBase chunk schema and migration. Defaults to 0 on NULL or missing field. Tests added.

Fixes #16967

---------

Co-authored-by: zq <zhouquan1511@163.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-27 14:23:37 +08:00
MarMar Labs 5b161b205c fix(schema-sync): stop reporting BaseModel timestamps as dropped columns (#18824)
Fix false db_schema_sync.py detection reporting timestamp columns as removed. skip_fields only skipped "id" but callers stripped all five timestamps. Now skips all five. Without this, --drop would generate destructive migrations dropping these columns from every table.
2026-08-27 11:24:40 +08:00
Lynn 618c4599b1 Fix: handle tenant_llm without id (#18541) 2026-08-20 14:12:52 +08:00
Liu An ec9c08d809 Docs: Update version references to v0.27.0 in READMEs and docs (#18500)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.26.4 to v0.27.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-08-19 17:06:49 +08:00
Lynn c642e01f4b Fix: set database version to v0.27.0 (#18424)
### Summary

As title.
2026-08-18 14:25:01 +08:00
Jack 8d20cbd0b3 deepdoc(pdf): WarpCrop de-skew + score-based layer-2 rotation (Go OCR parity with Python) (#18299)
Supersedes / folds in #18305. The score-based layer-2 rotation selection from #18305 now lives here, on top of `WarpCrop` (layer 1), applied to **all three** Go OCR paths, together with the Python score plumbing the Go side depends on. #18305 is closed in favor of this PR.
2026-08-17 15:13:56 +08:00
Jack c23d5fc819 fix(parser): support .msg parsing and make email attachments retrievable (#18198)
- Add Go `EmailParser` support for Outlook `.msg` (OLE2/CFB) files via
the `gomsg` library, in addition to the existing `.eml` (RFC 5322)
support. The `.msg` hard-error is gone; emails with `.msg` attachments
are now ingested end-to-end.
- Re-chunk email attachments into retrievable text (user-oriented). Each
attachment is re-parsed by its file extension through the shared parser
registry and folded back into the same document, so attachment content
becomes searchable. This mirrors Python's legacy `rag/app/email.py`.
Binary attachments (images/audio/video/folders) are skipped by design.
- Restore a corrupted `sample.msg` test fixture and add guards so binary
fixtures are never mangled again (`.gitattributes` marks `*.msg` binary;
`check_files.py` skips NUL-byte files). Also made `check_files.py`
ruff-clean.
2026-08-14 10:38:38 +08:00
Jin Hai d7661b676d Go: fix context and warnings (#18097)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-11 16:18:49 +08:00
EthanZhang 33e581a8b3 feat(agent): add Querit search tool (#17548) 2026-07-30 09:36:16 +08:00
euvre a40d5f841b fix: eliminate one-frame flash of title without form content in data source pages (#17214) 2026-07-22 19:06:38 +08:00
Muhammad Zuhaib ZahidandMuhammad Zuhaib Zahid 073fa123b3 docs: correct firecrawl README paths to tools/firecrawl (#17239)
### Summary

`tools/firecrawl/README.md` documents the integration as living in
`intergrations/firecrawl`,

Co-authored-by: Muhammad Zuhaib Zahid <288755265+muhzuhaib@users.noreply.github.com>
2026-07-22 17:06:21 +08:00
Zhichang Yu f69ed74670 fix lefthook (#17023) 2026-07-17 11:18:10 +08:00
rayhan 75d9efa27c fix: remediate urllib3 CVEs (#16907) 2026-07-14 22:57:29 +08:00
euvre 3bfad1f00e fix: correct model type mappings and improve system setting persistence (#16501) 2026-07-13 16:42:55 +08:00
buua436 74bbbba3e0 fix: align model default handling (#16782) 2026-07-10 10:34:19 +08:00
Lynn 1430d0e431 Fix: provider name (#16733) 2026-07-09 10:19:10 +08:00
Lynn 0ae5961e1c Feat: v0.27.0 model provider (#16604) 2026-07-08 09:47:29 +08:00