Commit Graph
413 Commits
Author SHA1 Message Date
xugangqiang a36615c6fa Migrate cache backend from Valkey 8 to apache/kvrocks:2.16.0 (#19374) 2026-09-11 17:39:11 +08:00
Liu An 5bd80b3455 Docs: Update version references to v0.27.2 in READMEs and docs (#19501)
- Update version tags in README files (including translations) from
v0.27.1 to v0.27.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
2026-09-10 18:35:02 +08:00
Lynn c5d9295cbb Fix: run ensure_db_init before start server (#19423) 2026-09-09 20:10:27 +08:00
Wang Qi a13593097b Refactor: Add RERANK_TOKEN_LIMIT_MODE = truncate (default) / passthrough / raise_error (#19404) 2026-09-09 09:55:00 +08:00
Wang Qi b7e577ce48 Refactor: combine 3 retrieval/search API as one (#19303) 2026-09-08 11:41:46 +08:00
Yuanhao Lyu eb24425669 feat(task_executor): add MAX_TASKS_PER_WORKER to recycle long-running workers (#14501) 2026-09-05 10:56:55 +08:00
maoyifeng 0cee507c3e CI: add Dockerfile_go build and api test (#19232)
1. CI add Dockerfile_go build and api test
2. Optimize the Dockerfile_go
2026-09-04 19:33:32 +08:00
buua436 2476e2fbfa feat: configure knowledge compilation runtime settings (#19254) 2026-09-04 17:05:22 +08:00
Wang Qi 4ab91c703c Dev: launch_backend_service.sh to support --debug (#19215) 2026-09-03 16:27:48 +08:00
68cdb795f5 Fix OAuth redirect URI examples (#15683)
Fix OAuth callback URL examples in docs — now use /api/v1/auth/oauth/<channel>/callback (was stale /v1/user/oauth/callback).

Co-authored-by: 谢金城 <xiejincheng@xgd.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-09-03 15:20:42 +08:00
jay77721 de99c7596d feat(ingestor): raise default max_concurrent_workers from 1 to 2 (#19133) 2026-09-02 15:24:04 +08:00
Leon Yin d441aa0331 fix(docker): disable stale macOS nginx mount (#19126)
Fix macOS Docker infinite restart — removed stale bind-mount for missing ragflow.conf.
2026-09-02 13:39:08 +08:00
MarMar Labs dcd1967711 fix(docker): let the backend launcher retry loops actually run (#18926)
Fix launch_backend_service.sh — set -e prevented retry logic from running on non-zero exit. Now uses || EXIT_CODE=$? to capture exit code.
2026-08-31 15:09:29 +08:00
maoyifeng 7690ebd84b Add go build files , reduce image size, update entrypoint-go.sh (#19041)
Add go build files , reduce image size, update entrypoint-go.sh
2026-08-31 14:31:45 +08:00
xugangqiang c28edde895 feat(deepdoc): in-process static-linked ONNX Runtime DeepDoc backend (#18731) 2026-08-31 13:18:12 +08:00
jay77721 de6b12a2b9 fix(ingestor,nats): recover consumption, fail fast on NATS startup and harden reconciliation (P0 blockers) (#18942) 2026-08-28 18:17:07 +08:00
706e87af9f fix(admin): stop silent superuser creation on auth probe and harden bootstrap (#18702)
Fix CWE-798/306 admin security: check_admin() silently created default admin accounts on any anonymous request. Now no longer creates accounts. Bootstrap uses env/random password. Login throttling added. Admin port bound to localhost. 

---------

Co-authored-by: jindou <jindou@local>
Co-authored-by: maoyifeng <myfbee5@163.com>
2026-08-28 17:44:52 +08:00
Liu An e3cc9a1621 Docs: Update version references to v0.27.1 in READMEs and docs (#18849) 2026-08-28 12:04:37 +08:00
Wang Qi edd3caa4cb Refine entrypoint.sh - tidy code (#18838) 2026-08-27 13:04:36 +08:00
MarMar Labs 3175d465cd docs(configuration): name the ragflow serving-port key http_port (#18825)
### Summary

Both configuration guides document the API server's port as `port`:

- `docs/administrator/configurations/configurations.md:152`
- `docker/README.md:190`

The key in `docker/service_conf.yaml.template:5` is `http_port`
2026-08-27 11:06:12 +08:00
陈志谦andjindou fdf937177d fix(security): authenticate sandbox executor API and isolate container network (#18703)
Fix sandbox security gaps: (1) add token auth + rate limiter to POST /run, bind port to localhost; (2) default containers to --network none. Backward compatible. Tests pass, docs updated.

---------

Co-authored-by: jindou <jindou@local>
2026-08-26 19:26:28 +08:00
Sandskbs-eng f6fd4127df fix(docker): prevent all foreground entrypoint.sh loops from exiting on non-zero exit (#18545)
## Summary

Fix entrypoint.sh restart loops where non-zero exits would terminate the subshell instead of retrying (root cause of #18542). Now wraps each binary with set +e / set -e to capture exit status and log it before restarting. Applied to 6 loops. Complementary to #17196.

Co-authored-by: skbs-eng <skbs-eng@users.noreply.github.com>
2026-08-26 17:03:01 +08:00
Paul Y HuiandJin Hai 5bc8f8a967 [Fix] Refined the NLTK data usage directory (#17376)
### Summary

download_deps.py downloads NLTK data into ragflow_deps/nltk_data (see
ragflow_deps/download_deps.py); point NLTK_DATA there directly instead
of a stale top-level ./nltk_data so this matches what's actually
populated.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-25 12:28:28 +08:00
MarMar Labs 57f25ea16e docs(migration): say which volumes the backup actually covers (#18626)
### Summary

Update comments of migration.sh
2026-08-25 11:53:33 +08:00
de029503ee docs: replace root-relative cross links with relative page links (#18664)
configurations.md and backup_and_migration.md cross-linked each other
via /migration#... and /configurations#s3-tigris. These host-absolute
paths match no route on the docs site (real pages live under
/administrator/...) and 404 on GitHub. Point them at the sibling
markdown files so both Docusaurus and GitHub can resolve them; the
existing anchors are preserved.

---------

Co-authored-by: jindou <jindou@local>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-24 11:58:43 +08:00
maoyifeng 024b5680ba add nats and clickhouse expose port in docker compose (#18577)
add nats and clickhouse expose port in docker-compose-base.yml and .env
file,avoid service port name conflict
2026-08-24 10:53:22 +08:00
MarMar Labs f796721ff2 docs(entrypoint): show every option usage() accepts (#18596)
### Summary

Update usage of entrypoint.sh
2026-08-21 20:51:51 +08:00
Rootkit 834dfc1906 Add ulimits for cpu and gpu services in Docker Compose (#17205)
### Summary

fix OSError: [Errno 24] Too many open files
2026-08-20 10:43:22 +08:00
Liu An ec9c08d809 Docs: Update version references to v0.27.0 in READMEs and docs (#18500)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.26.4 to v0.27.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-08-19 17:06:49 +08:00
zws202606 d02723ca76 feat: Gaussdb adaptation (#17703) 2026-08-18 09:53:11 +08:00
Mpider-San c7aca4c97e Replace pgsty/minio image with pgsty/silo (#18231)
### Summary

Closes #18019.

Pigsty renamed its community MinIO fork from `pgsty/minio` to
`pgsty/silo` on 2026-08-06 ([announcement in the repo
README](https://github.com/pgsty/silo#readme)). `pgsty/minio` is frozen
at `RELEASE.2026-08-04T00-00-00Z` — all further releases, including
security fixes, ship as `pgsty/silo`. RAGFlow still pins the old name,
so it is now parked on an image that no longer gets updates.
2026-08-13 18:45:43 +08:00
Haruko386 96f7c39753 feat[Go]: resuming transmission from the point of interruption during data source synchronisation (#18176)
### Summary

As title
2026-08-13 18:38:37 +08:00
qinling0210 0c225b0c39 Bump to infinity v0.7.3 and update go sdk version (#18137)
### Summary

Bump to infinity v0.7.3 and update go sdk version
2026-08-12 11:30:34 +08:00
Haruko386 dd52dee600 feat[Go]: monitoring NATs and refactoring concurrency logic (#18049)
### Summary

As title
2026-08-11 14:36:11 +08:00
YeeandJin Hai 27c8b1a061 fix(nginx): use 127.0.0.1 in python conf to avoid ~1s IPv6 fallback latency (#17909)
## Problem

In the default deployment (`API_PROXY_SCHEME=python`), the nginx config
generated from `docker/nginx/ragflow.conf.python` uses:

```nginx
proxy_pass http://localhost:9381;  # admin
proxy_pass http://localhost:9380;  # api
```

Inside the RAGFlow container, `/etc/hosts` maps `localhost` to both
`127.0.0.1` and `::1`:

```
127.0.0.1   localhost
::1         localhost ip6-localhost ip6-loopback
```

But the backend services (`ragflow_server.py` on 9380, admin server on
9381) only listen on IPv4 (`127.0.0.1`). When nginx resolves `localhost`
to `::1`, the upstream connection fails and nginx falls back to the IPv4
address - each fallback adds ~1 second of latency.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-08-07 11:32:43 +08:00
Zhichang Yu 2e37997ab9 Go knowledge compiler with scheduler-driven dataset compilation (#17913)
Ports dataset knowledge compilation (wiki/graph/tree/mindmap) to the Go
scheduler with a status contract, aligns wiki storage/retrieval with
Python, sizes prompts by content_length, and resolves embedding batch
size from provider capability.
2026-08-06 15:54:00 +08:00
Jin Hai cf13082a1a Revert "feat: Go knowledge compiler with scheduler-driven dataset compilation" (#17897)
Reverts infiniflow/ragflow#17881
2026-08-05 21:50:28 +08:00
Zhichang Yu 14b943a04a feat: Go knowledge compiler with scheduler-driven dataset compilation (#17881)
Ports the dataset knowledge compilation (wiki/graph/tree/mindmap) to the
Go scheduler with a status contract, aligns wiki storage/retrieval with
Python, and sizes prompts by content_length.
2026-08-05 20:00:42 +08:00
deadtricksterandClaude Opus 4.8 197b142cef feat(serenedb): add SereneDB doc-store engine (Go + Python connectors) (#17375)
## What

Adds [**SereneDB**](https://serenedb.com) as a selectable doc-store
engine on **both** RAGFlow paths:
- the **Go** `DocEngine` (`internal/engine/serenedb`), alongside
Elasticsearch and Infinity;
- the **Python** `DocStoreConnection` (`rag/utils/serenedb_conn.py`) +
`DOC_ENGINE=serenedb` registration.

SereneDB is a PostgreSQL-wire engine (DuckDB execution) whose single
inverted index carries **both** a scored text column (`@@`, BM25) and an
IVF vector column (`<#>`, inner product), so hybrid search is one SQL
statement. The Go engine connects with `database/sql` + `lib/pq`
(already a dependency, no new module); the Python connector uses
psycopg2 (already a dependency).

## Storage model

One table per tenant with `kb_id` as a filter column - the
**Elasticsearch / OceanBase** model, not Infinity's per-dataset tables.
This keeps BM25 statistics (IDF, avgdl) computed over the whole tenant
corpus (global IDF). Both connectors use this identical layout, so they
are storage- and retrieval-compatible: `hybrid` proxy routing and
Python↔Go switching are safe. On the Python side the connector is wired
as OceanBase's plain-SQL sibling (chunk_data JSON metadata, inline chunk
vectors, verbatim ES field names); the ES tokenizer path is unchanged.
Metadata stays one table per tenant (`ragflow_doc_meta_<tenant>`).

The query shapes mirror the Python connector, including the five
empirically-found landmines: the scored dictionary needs `frequency +
norm` (else `BM25()` silently returns 0.0), the `@@` query is the
tokenized query, the scored lexical branch matches one column, vectors
use an L2-normalized shadow column with `ip`/`sq8`, and the similarity
threshold goes directly in the ANN scan's `WHERE`. **Minimum engine
version: SereneDB 26.07.4.**

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-04 14:16:39 +08:00
Wang Qi bb1879ba35 REDIS: allkeys-lru -> volatile-lru to avoid got evict if lack of memory (#17720)
Follow on PR: #17707
2026-08-03 15:44:59 +08:00
maoyifeng 3874ef8765 Dockerfile base part separate to base image (#17701)
Unify open-source and closed-source Dockerfile expect nginx
1. per install tools to base image
2. fix copy nginx config
3. fix copy entrypoint*.sh
4. fix go ci error
5. add comment
2026-08-03 14:29:48 +08:00
Jin Hai b2521ebf51 Go: refactor log and config (#17684)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-02 20:55:19 +08:00
Jin Hai 36ae39bc60 Go: refactor config (#17544)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-31 17:18:45 +08:00
Jin Hai 0eb8e0b393 Go: refactor ragflow_server.go (#17494)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-29 11:38:27 +08:00
Wang Qi b4170d0dfd Dev/enhance dev script (#17487) 2026-07-28 17:47:45 +08:00
euvre f45f03a016 Add Go service/handler tests for API contract parity (#16905) 2026-07-20 20:02:41 +08:00
Jin Hai 75a8228d33 Fix missing info (#17107)
### Summary

1. Fix docker/service_conf.yaml.template
2. Remove unused config

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-20 17:32:32 +08:00
qinling0210 6c002ced86 Bump Infinity to v0.7.2 (#17075)
### Summary

Bump to infinity v0.7.2

Infinity image: infiniflow/infinity:v0.7.2-x64-v3
2026-07-20 14:30:24 +08:00
Wang Qi c420de0dc5 Security fix: CVE-2025-21521 (#17046) 2026-07-17 19:35:20 +08:00
Jin Hai f8474a67f5 Go: tracing framework (#17004)
### Summary

Tracing framework

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-16 19:35:43 +08:00