Commit Graph
119 Commits
Author SHA1 Message Date
Liu An 03c1246150 Docs: Update version references to v1.0.0-rc3 in READMEs and docs (#21788) 2026-10-30 19:47:51 +08:00
Wang Qi d717d0c975 Refactor: delete agent browser (#21599) 2026-10-28 15:44:29 +08:00
Sbaaoui Idriss 076d428396 feat(mcp): expose configurable listener transport settings (#21547) 2026-10-27 16:36:57 +08:00
Wang Qi 0b9ce6ddda Combine RAGFLOW_MCP_ALLOW_PRIVATE_HOSTS into ALLOW_ANY_HOST (#21523) 2026-10-27 13:06:05 +08:00
YanZhang a3c3e73a6d docs: update Build Docker Image docs (#21339) 2026-10-26 14:16:33 +08:00
xugangqiang 6a2ef7ec6b refactor(deps): rename ragflow_deps directory to deps (#21345)
## Summary

Rename the repository dependency-resource directory `ragflow_deps/` to
`deps/`, and update every hardcoded reference to that directory name so
the rename does not break resource loading at build/runtime.

This is not a purely cosmetic rename: the Go runtime hardcodes the
directory name as a resource search path (e.g.
`internal/common/model_assets.go` trimming the `deps/` prefix, and the
tokenizers joining `deps/` into candidate paths), and the Dockerfiles
copy assets into `/ragflow/deps`. Those literals must move together with
the directory, otherwise assets fail to resolve.

## Two intentional exceptions (kept unchanged)

The rename deliberately keeps two names that only *coincidentally*
contained the old string:

- The published image tag `infiniflow/ragflow_deps:latest` (used by
`Dockerfile`, `Dockerfile_base`, docs, and CI `image inspect` checks).
- The CI install path `/opt/ragflow_deps` inside the CI runner image
(`build.sh` `SYSTEM_DEPS_TOKENIZER`).

## Changes

- `git mv ragflow_deps deps`.
- `build.sh`: `CL100K_TABLE_RELPATH`, native asset zip path, and related
references updated to `deps/`.
- `tools/scripts/download_dev_deps.py`: `RAGFLOW_DEPS_DIR` points at
`deps/`.
- Go runtime resource search paths (`internal/common`,
`internal/tokenizer`) updated to the `deps/` layout.
- `Dockerfile` / `Dockerfile_base`: container asset path changed to
`/ragflow/deps`; kept tag `infiniflow/ragflow_deps:latest` unchanged.
- Docs / `AGENTS.md` / CI workflow paths updated to the new directory
name.

## Test plan

- Go unit tests in `internal/common` and `internal/tokenizer` updated to
the new `deps/` layout and pass via `bash build.sh --test`.
- Python download scripts import and build URLs correctly (`py_compile`
clean).
- Added `tools/scripts/test_dep_rename_invariants.py` as a regression
guard asserting the two intentional exceptions (`/opt/ragflow_deps` and
`infiniflow/ragflow_deps`) stay intact, so a future blanket rename
cannot silently regress them.

---

## Follow-up fix (commit 2f6857095d)

The original rename used a blanket `sed s/ragflow_deps/deps/g` and only
selectively reverted `build.sh`. That left two intentionally-kept names
rewritten, which this follow-up restores:

- CI install path `/opt/ragflow_deps` — `sep-tests.yml` had been changed
to read `/opt/deps`, breaking the offline cl100k table fallback (the
cached-image path pointed at a directory that never receives the file).
Restored to `/opt/ragflow_deps`.
- Published image tag `infiniflow/ragflow_deps` — left as bare
`infiniflow/deps` in `sep-tests.yml` comments/echo, `.dockerignore`, and
`Dockerfile` / `Dockerfile_base` error messages. Restored to
`infiniflow/ragflow_deps`.

The regression guard `tools/scripts/test_dep_rename_invariants.py` was
added to prevent recurrence.
2026-10-22 19:20:42 +08:00
Sbaaoui Idriss 76b068fba2 fix(mcp): remove unused force_refresh parameter (#21349) 2026-10-22 19:09:10 +08:00
xugangqiang c754300366 refactor(deps): split dependency download into two Python scripts (#20861) 2026-10-13 20:44:56 +08:00
YanZhang b9d814437f docs: Removed the description of API_PROXY_SCHEME and synchronized the README files (#20756) 2026-10-12 11:19:19 +08:00
Wang Qi e0f5fd1900 Docs: fix docker ps/logs (#20671) 2026-10-09 16:39:52 +08:00
Sbaaoui Idriss 9fbf25ea00 fix(mcp): prevent managed token exposure (#20639) 2026-10-09 15:38:36 +08:00
xugangqiang d05e8c317f docs: mark macOS as temporarily unsupported for the Go backend (#20642) 2026-10-09 12:53:04 +08:00
jay77721 4071451ea4 refactor(deps): use Go dependency downloader as sole entrypoint (#20360) 2026-09-30 18:02:03 +08:00
Jin Hai 16e19da82f Doc: fix errors (#20427)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-30 12:04:27 +08:00
Jin Hai 60016b39c9 refactor: move res into internal (#20390)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-30 10:49:17 +08:00
YanZhang 01f6b9c3bc docs:update Docker commands in deployment guides (#20353) 2026-09-29 15:29:42 +08:00
YanZhang dc7656eb92 docs: update docker build in docs (#20345) 2026-09-29 13:17:18 +08:00
YanZhang fa0f3c29ab docs: update docs with go (#20062) 2026-09-29 10:35:03 +08:00
Jin Hai 7c102247c6 refactor: update go verion server port to 9380 and 9381 (#20298)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-28 17:35:16 +08:00
Sbaaoui Idriss 05b167e966 refactor(mcp): centralize configuration and transport wiring (#20011) 2026-09-22 11:18:14 +08:00
JHWu1117 1ca2b9ca24 docs: add macOS notes to the launch-from-source guide (#19970)
Refs #8014
2026-09-21 15:19:11 +08:00
George Pickett 163ab2e032 Fix: preserve MCP connection settings across import and edit (#19518)
Fix MCP settings lost on import/export/edit (headers, substitution variables, auth scheme).
2026-09-20 16:25:32 +08:00
Sbaaoui Idriss d36dc659ec feat(mcp): add SDK-backed native Go MCP runtime (#19673) 2026-09-17 11:44:15 +08:00
YanZhang b2c2afa5e1 docs: update MCP documents (#19730)
Adds an overview of RAGFlow MCP features.  
Adds guides for using RAGFlow as an MCP server and connecting external
MCP servers.
Improves navigation between related MCP documents.
2026-09-16 20:09:32 +08:00
Horace 185f089310 docs: fix Python frontend proxy startup (#19606)
Fix #19600 — update dev guide to use API_PROXY_SCHEME=python npm run dev instead of editing Vite proxy target.
2026-09-14 15:50:22 +08:00
Liu An 5bd80b3455 Docs: Update version references to v0.27.2 in READMEs and docs (#19501)
- Update version tags in README files (including translations) from
v0.27.1 to v0.27.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
2026-09-10 18:35:02 +08:00
Liu An e3cc9a1621 Docs: Update version references to v0.27.1 in READMEs and docs (#18849) 2026-08-28 12:04:37 +08:00
Liu An ec9c08d809 Docs: Update version references to v0.27.0 in READMEs and docs (#18500)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.26.4 to v0.27.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-08-19 17:06:49 +08:00
euvre 0a444343b9 Fix: dataset tag file setting cannot be saved (#17428) 2026-08-05 15:55:34 +08:00
YanZhang 611e0596f6 docs: modify docs display & add images (#17575)
### Summary

modify docs display
add some images
delete enterprise docs
standardize capitalization
2026-07-30 21:24:15 +08:00
Yingfeng 794fcc2517 Fix docs (#16770) 2026-07-09 13:38:29 +08:00
Liu An 4da9429451 Docs: Update version references to v0.26.4 in READMEs and docs (#16716) 2026-07-07 19:36:58 +08:00
Liu An 63a4ed55d8 docs: update Docker build instructions for deps image (#16620)
### Summary

update Docker build instructions for deps image
2026-07-03 19:57:12 +08:00
Liu An 32c5cb16e9 Docs: Update version references to v0.26.3 in READMEs and docs (#16574) 2026-07-02 20:55:15 +08:00
Liu An f86a0e7386 Docs: Update version references to v0.26.2 in READMEs and docs (#16387) 2026-06-29 09:45:16 +08:00
Zhichang YuandClaude 3f805a64f1 feat(agent): align Go agent behavior with Python (except retrieval component) (#16225)
## Summary

Aligns the **Go agent runtime/canvas/components/tools** behavior with
the **Python `agent/` implementation** so the same stored canvas DSL
produces the same execution result on either side. Every component,
tool, and runtime primitive in `internal/agent/` is now driven by the
same semantics as its Python counterpart — variable resolution, template
substitution, control flow, error reporting, retry/cancel, and stream
event shapes.

The **retrieval component is the one explicit exception** in this PR. It
is being reworked in a separate change and is excluded from this
alignment pass; the wrapper slot (`universe_a_wrappers.go →
newRetrievalComponent`) is preserved.

## Scope of alignment

### Components (all aligned with `agent/component/`)
`Begin` · `Message` · `LLM` (incl. ChatTemplateKwargs,
MessageHistoryWindowSize, VisualFiles, Cite, OutputStructure,
JSONOutput, TopP, MaxRetries, DelayAfterError, credentials) · `Agent`
(react + tool artifact capture + `Reset()` interface-assert) · `Switch`
(12/12 operators, Python-equivalent semantics) · `Categorize` · `Invoke`
· `Iteration` · `Loop` (macro-expansion through `workflowx.AddLoopNode`)
· `UserFillUp` (Python-equivalent interrupt/resume via eino
`compose.Interrupt`/`ResumeWithData`) · `FillUp` · `DataOperations` ·
`ListOperations` · `StringTransform` · `VariableAggregator` ·
`VariableAssigner` · `Browser` (full stagehand runtime parity) ·
`DocsGenerator` · `ExcelProcessor`.

### Tools (all aligned with `agent/tools/`)
`Retrieval` (wrapper slot only — logic out of scope) · `MCPToolAdapter`
(streamable-HTTP) · `CodeExec` (sandbox bridge with
`code_exec_contract.go` matching Python contract) · `AkShare` · `ArXiv`
· `Crawler` · `DeepL` · `DuckDuckGo` · `Email` · `ExeSQL` · `GitHub` ·
`Google` · `GoogleScholar` · `Jin10` · `PubMed` · `QWeather` · `SearXNG`
· `Tavily` · `Tushare` · `Wencai` · `Wikipedia` · `YahooFinance` —
uniform `eino tool.InvokableTool` interface, SSRF protection, shared
HTTP client.

### Canvas execution engine (`internal/agent/canvas/`)
Aligned with Python's `agent/canvas.py`:
- **Scheduler** (`scheduler.go`): state pre/post handlers, node lambdas,
per-component timeout resolver (4-level: per-class env → per-class table
→ uniform env → 600s fallback), `legacyNoOpNames`.
- **Loop subgraph** (`loop_subgraph.go`): Python-equivalent
`AddLoopNode` macro expansion + condition translation.
- **Multibranch** (`multibranch.go`): `Switch` / `Categorize` routing
via `compose.NewGraphMultiBranch` — same branch selection semantics as
Python.
- **Parallel subgraph** (`parallel_subgraph.go`): matches Python's
parallel fan-out contract.
- **Interrupt/Resume** (`interrupt_resume.go`): `UserFillUpNodeBody` /
`IsInterruptError` / `ExtractInterruptContexts` — replaces the
deprecated Python sentinel chain with eino's native interrupt API,
preserving the same external behavior.
- **Checkpoint** (`checkpoint_store.go`): `RedisCheckPointStore`
Get/Set/Delete, with business metadata (status / canvas_id /
parent_run_id) on a parallel Redis Hash.
- **RunTracker** (`run_tracker.go`): Start / MarkSucceeded / MarkFailed
/ MarkCancelled / AttachCheckpoint — same lifecycle as the Python run
record.
- **Cancel** (`cancel.go`): Redis pub/sub watch.
- **Stream** (`stream.go`): SSE channel with `messages` / `waiting` /
`errors` / `done` events, same shape as Python's `agent.canvas.RunEvent`
payload.

### DSL bridge (`internal/agent/dsl/`)
- `normalize.go`: v1↔v2 collapsed into a single wire format — Python and
Go consume the same stored JSON.
- `reset.go`: per-run state reset matches Python's `Canvas.reset()`
semantics.
- Testdata mirrors Python's `agent_msg.json` / `all.json` / etc.

### Runtime (`internal/agent/runtime/`)
- `CanvasState` / `NewCanvasState` / `GetVar` / `SetVar` / `ReadVars`:
same `{{cpn_id@param}}` resolution model.
- `ResolveTemplate` (regex fast path + gonja fallback) — Python
Jinja-style semantics.
- `selector.go`, `metrics.go`, `component.go`: shared runtime contracts.

## Out of scope (intentionally)

- **`Retrieval` component logic** — wrapped only; full parity lands in a
follow-up PR.
- **Frontend** — only minor dsl-bridge / canvas UX fixes ride along.
- **CLI / admin / model registry** — orthogonal to agent behavior.

## How alignment is verified

`internal/service/agent_run_e2e_test.go` exercises the **full production
chain** against real Python-shaped DSL fixtures:
```
loadCanvasForUser → versionDAO.GetLatest → decodeCanvasFromDSL →
canvas.Compile → cc.Workflow.Invoke → answer extraction
```
using in-memory SQLite + miniredis (no Docker). Covers:
- `TestRunAgent_RealCanvas_BeginMessage` — happy path, `{{sys.query}}`
resolution
- `TestRunAgent_RealCanvas_WaitForUserResume` — two-run resume cycle
(Python-equivalent)
- `TestRunAgent_RealCanvas_CompileFails` — unknown component name →
sanitized error (Python-equivalent)
- `TestRunAgent_RealCanvas_InvokeFails` — unresolvable template ref
(Python-equivalent)
- `TestRunAgent_RunTracker_AttachCheckpoint_CallSequence` —
Start→AttachCheckpoint→MarkSucceeded lifecycle

`internal/handler/agent_test.go` — SSE streaming parity (`Content-Type:
text/event-stream`, `data: {…}\n\n`, trailing `data: [DONE]\n\n`,
OpenAI-compatible non-stream `choices`).

`internal/agent/canvas/fixture_compile_test.go` + per-component tests
pin the Python-equivalent outputs.

```
go test -count=1 -v -run 'TestRunAgent_RealCanvas|TestRunAgent_RunTracker' ./internal/service/
```

## Design reference

`docs/develop/agent-go-port-design.md` (1329 lines, last cross-checked
2026-06-17) — module layout, per-component / per-tool inventory,
corner-case catalogue, and the actionable backlog (Section 14, including
the retrieval alignment follow-up).

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 11:58:29 +08:00
Liu An 4379269374 Docs: Update version references to v0.26.1 in READMEs and docs (#16158)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.26.0 to v0.26.1
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-06-17 19:35:32 +08:00
Zhichang YuandClaude e45659868a feat(agent): ship the Go agent canvas port — eino interrupt/resume + Redis check-pointing (#16035)
Replaces the Python agent canvas runtime with a Go implementation that
runs inside `cmd/server_main`.

The canvas compiles into an eino Workflow that pauses on wait-for-user
via native Interrupt/Resume (no sentinel flag) and resumes from a
Redis-backed CheckPointStore.

All 21 Python agent components and ~35 tools are ported with functional
parity.

Sandbox providers now read their JSON config from the admin-panel
system_settings table with env fallback.

234 files / +35,413 / -6,111. All Go files are gofmt-clean (CI gate
added); drops the v2 DSL E2E step and the gap-analysis plan (both
redundant after the port ships).

## Type of change

- [x] Refactoring
- [x] New feature
- [x] Bug fix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-17 13:24:03 +08:00
Zhichang Yu 3fa15c0e2f feat(agent): Go port — canvas engine, 22 components, DSL v2, 13 endpoints (#15952)
Ports the agent canvas subsystem from Python to Go.

## What's included

### Canvas Engine (Phase 0/1)
- State engine, scheduler, variable resolver, Redis checkpoint store,
cancel protocol
- **209 tests** across canvas / component / io packages

### 22 Components (P0–P4)
| Tier | Components |
|---|---|
| P0 T1+T2+T3 | LLM, Agent, ExitLoop, Switch, Categorize, Begin,
Message, Invoke |
| P1 T3 | VariableAggregator, VariableAssigner, StringTransform,
ListOperations, DataOperations |
| P2 T3 | Iteration, IterationItem, Loop, LoopItem |
| P3 T3 | UserFillUp, Fillup |
| P4 T5 | Browser, ExcelProcessor, DocsGenerator |

### DSL v2 Schema (Phase 2.5)
- Typed v2 in-memory model with v1-to-v2 auto-detect converter
- v1 legacy field stripping per plan §2.11.7

### HTTP Endpoints & Bug Fixes (Plans PR1–PR3)
- **DELETE SQL bug fix**: gorm v2 `Where("id = ?", id).Delete(...)`
pattern
- **CreateAgent validation**: title/DSL required, duplicate check, 103
envelope
- **13 new endpoints**: templates, prompts, tags, sessions CRUD,
chat/completions (SSE + non-stream stubs), rerun, test_db_connection,
logs, webhook/logs
- **756 Go unit tests** (745 → 756, +18)
- **17 → 0 Python integration test failures** (test_agents.py +
test_session_management/)

### Tools
21 eino tools: HTTPHelper, search tools, financial/data tools, mandatory
stubs

### Infrastructure
OTel observability, NATS message queue, DeepDoc gRPC client, SSRF
guards, IDOR mitigation
2026-06-12 22:58:28 +08:00
Liu An 92c4b7688b Docs: Update version references to v0.26.0 in READMEs and docs (#15941)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.6 to v0.26.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-06-11 18:34:26 +08:00
Jack bea8092007 Update developer doc (#15336)
### What problem does this PR solve?

update developer doc

### Type of change

- [x] Documentation Update
2026-05-28 15:58:09 +08:00
Liu An 0639dba89a Docs: Update version references to v0.25.6 in READMEs and docs (#15248)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.5 to v0.25.6
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-26 19:45:43 +08:00
Jin Hai 775ea55679 Docs: update python version to 3.13 (#15103)
### What problem does this PR solve?

1. update python version to 3.13
2. upgrade ormsgpack to 1.6.0

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 19:09:19 +08:00
Jin Hai 90c76e73d0 Docs: Update version references to v0.25.5 in READMEs and docs (#15059)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-20 20:05:45 +08:00
Liu An f038a34154 Docs: Update version references to v0.25.4 in READMEs and docs (#14912)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.3 to v0.25.4
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-14 11:07:08 +08:00
Liu An 3182fd0789 Docs: Update version references to v0.25.3 in READMEs and docs (#14896)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.2 to v0.25.3
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-13 18:42:42 +08:00
Liu An 57b24be6d6 Docs: Update version references to v0.25.2 in READMEs and docs (#14731)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.1 to v0.25.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-09 19:06:05 +08:00
Liu An ce4c782fd7 Docs: Update version references to v0.25.1 in READMEs and docs (#14488)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.0 to v0.25.1
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-04-30 10:49:26 +08:00
Wang Qi 7fb6a12067 Update API document (#14364)
### What problem does this PR solve?

Update API document

### Type of change

- [ ] Documentation Update
2026-04-24 20:36:47 +08:00
Mukunda Rao Katta 8a2f63e77d docs: fix API key guide typo (#14352)
Fixes a small typo in the RAGFlow API key guide: `This documents
provides` -> `This document provides`.
2026-04-24 16:59:25 +08:00