Commit Graph
122 Commits
Author SHA1 Message Date
Liu An 996df33662 Docs: Update version references to v1.0.0 in READMEs and docs (#22108) 2026-11-06 19:44:48 +08:00
YanZhang 25f167d3fb docs:docs: add use cases for common scenarios (#22054)
Add five use cases covering policy Q&A, research comparison with Agentic
RAG, technical Wiki compilation, Confluence synchronization, and team
collaboration. Each case includes preparation, implementation steps,
examples, validation criteria, and maintenance guidance, with links to
the User Guides for detailed configuration. Place Use Cases between
Quickstart and User Guides and adjust the sidebar order.
2026-11-05 19:32:31 +08:00
YanZhang c841a1c897 docs: Documentation Updates and UI Alignment (#21986) 2026-11-05 10:22:56 +08:00
Liu An 03c1246150 Docs: Update version references to v1.0.0-rc3 in READMEs and docs (#21788) 2026-10-30 19:47:51 +08:00
Wang Qi d717d0c975 Refactor: delete agent browser (#21599) 2026-10-28 15:44:29 +08:00
Sbaaoui Idriss 076d428396 feat(mcp): expose configurable listener transport settings (#21547) 2026-10-27 16:36:57 +08:00
Wang Qi 0b9ce6ddda Combine RAGFLOW_MCP_ALLOW_PRIVATE_HOSTS into ALLOW_ANY_HOST (#21523) 2026-10-27 13:06:05 +08:00
YanZhang a3c3e73a6d docs: update Build Docker Image docs (#21339) 2026-10-26 14:16:33 +08:00
xugangqiang 6a2ef7ec6b refactor(deps): rename ragflow_deps directory to deps (#21345)
## Summary

Rename the repository dependency-resource directory `ragflow_deps/` to
`deps/`, and update every hardcoded reference to that directory name so
the rename does not break resource loading at build/runtime.

This is not a purely cosmetic rename: the Go runtime hardcodes the
directory name as a resource search path (e.g.
`internal/common/model_assets.go` trimming the `deps/` prefix, and the
tokenizers joining `deps/` into candidate paths), and the Dockerfiles
copy assets into `/ragflow/deps`. Those literals must move together with
the directory, otherwise assets fail to resolve.

## Two intentional exceptions (kept unchanged)

The rename deliberately keeps two names that only *coincidentally*
contained the old string:

- The published image tag `infiniflow/ragflow_deps:latest` (used by
`Dockerfile`, `Dockerfile_base`, docs, and CI `image inspect` checks).
- The CI install path `/opt/ragflow_deps` inside the CI runner image
(`build.sh` `SYSTEM_DEPS_TOKENIZER`).

## Changes

- `git mv ragflow_deps deps`.
- `build.sh`: `CL100K_TABLE_RELPATH`, native asset zip path, and related
references updated to `deps/`.
- `tools/scripts/download_dev_deps.py`: `RAGFLOW_DEPS_DIR` points at
`deps/`.
- Go runtime resource search paths (`internal/common`,
`internal/tokenizer`) updated to the `deps/` layout.
- `Dockerfile` / `Dockerfile_base`: container asset path changed to
`/ragflow/deps`; kept tag `infiniflow/ragflow_deps:latest` unchanged.
- Docs / `AGENTS.md` / CI workflow paths updated to the new directory
name.

## Test plan

- Go unit tests in `internal/common` and `internal/tokenizer` updated to
the new `deps/` layout and pass via `bash build.sh --test`.
- Python download scripts import and build URLs correctly (`py_compile`
clean).
- Added `tools/scripts/test_dep_rename_invariants.py` as a regression
guard asserting the two intentional exceptions (`/opt/ragflow_deps` and
`infiniflow/ragflow_deps`) stay intact, so a future blanket rename
cannot silently regress them.

---

## Follow-up fix (commit 2f6857095d)

The original rename used a blanket `sed s/ragflow_deps/deps/g` and only
selectively reverted `build.sh`. That left two intentionally-kept names
rewritten, which this follow-up restores:

- CI install path `/opt/ragflow_deps` — `sep-tests.yml` had been changed
to read `/opt/deps`, breaking the offline cl100k table fallback (the
cached-image path pointed at a directory that never receives the file).
Restored to `/opt/ragflow_deps`.
- Published image tag `infiniflow/ragflow_deps` — left as bare
`infiniflow/deps` in `sep-tests.yml` comments/echo, `.dockerignore`, and
`Dockerfile` / `Dockerfile_base` error messages. Restored to
`infiniflow/ragflow_deps`.

The regression guard `tools/scripts/test_dep_rename_invariants.py` was
added to prevent recurrence.
2026-10-22 19:20:42 +08:00
Sbaaoui Idriss 76b068fba2 fix(mcp): remove unused force_refresh parameter (#21349) 2026-10-22 19:09:10 +08:00
xugangqiang c754300366 refactor(deps): split dependency download into two Python scripts (#20861) 2026-10-13 20:44:56 +08:00
YanZhang b9d814437f docs: Removed the description of API_PROXY_SCHEME and synchronized the README files (#20756) 2026-10-12 11:19:19 +08:00
Wang Qi e0f5fd1900 Docs: fix docker ps/logs (#20671) 2026-10-09 16:39:52 +08:00
Sbaaoui Idriss 9fbf25ea00 fix(mcp): prevent managed token exposure (#20639) 2026-10-09 15:38:36 +08:00
xugangqiang d05e8c317f docs: mark macOS as temporarily unsupported for the Go backend (#20642) 2026-10-09 12:53:04 +08:00
jay77721 4071451ea4 refactor(deps): use Go dependency downloader as sole entrypoint (#20360) 2026-09-30 18:02:03 +08:00
Jin Hai 16e19da82f Doc: fix errors (#20427)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-30 12:04:27 +08:00
Jin Hai 60016b39c9 refactor: move res into internal (#20390)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-30 10:49:17 +08:00
YanZhang 01f6b9c3bc docs:update Docker commands in deployment guides (#20353) 2026-09-29 15:29:42 +08:00
YanZhang dc7656eb92 docs: update docker build in docs (#20345) 2026-09-29 13:17:18 +08:00
YanZhang fa0f3c29ab docs: update docs with go (#20062) 2026-09-29 10:35:03 +08:00
Jin Hai 7c102247c6 refactor: update go verion server port to 9380 and 9381 (#20298)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-09-28 17:35:16 +08:00
Sbaaoui Idriss 05b167e966 refactor(mcp): centralize configuration and transport wiring (#20011) 2026-09-22 11:18:14 +08:00
JHWu1117 1ca2b9ca24 docs: add macOS notes to the launch-from-source guide (#19970)
Refs #8014
2026-09-21 15:19:11 +08:00
George Pickett 163ab2e032 Fix: preserve MCP connection settings across import and edit (#19518)
Fix MCP settings lost on import/export/edit (headers, substitution variables, auth scheme).
2026-09-20 16:25:32 +08:00
Sbaaoui Idriss d36dc659ec feat(mcp): add SDK-backed native Go MCP runtime (#19673) 2026-09-17 11:44:15 +08:00
YanZhang b2c2afa5e1 docs: update MCP documents (#19730)
Adds an overview of RAGFlow MCP features.  
Adds guides for using RAGFlow as an MCP server and connecting external
MCP servers.
Improves navigation between related MCP documents.
2026-09-16 20:09:32 +08:00
Horace 185f089310 docs: fix Python frontend proxy startup (#19606)
Fix #19600 — update dev guide to use API_PROXY_SCHEME=python npm run dev instead of editing Vite proxy target.
2026-09-14 15:50:22 +08:00
Liu An 5bd80b3455 Docs: Update version references to v0.27.2 in READMEs and docs (#19501)
- Update version tags in README files (including translations) from
v0.27.1 to v0.27.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files
2026-09-10 18:35:02 +08:00
Liu An e3cc9a1621 Docs: Update version references to v0.27.1 in READMEs and docs (#18849) 2026-08-28 12:04:37 +08:00
Liu An ec9c08d809 Docs: Update version references to v0.27.0 in READMEs and docs (#18500)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.26.4 to v0.27.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-08-19 17:06:49 +08:00
euvre 0a444343b9 Fix: dataset tag file setting cannot be saved (#17428) 2026-08-05 15:55:34 +08:00
YanZhang 611e0596f6 docs: modify docs display & add images (#17575)
### Summary

modify docs display
add some images
delete enterprise docs
standardize capitalization
2026-07-30 21:24:15 +08:00
Yingfeng 794fcc2517 Fix docs (#16770) 2026-07-09 13:38:29 +08:00
Liu An 4da9429451 Docs: Update version references to v0.26.4 in READMEs and docs (#16716) 2026-07-07 19:36:58 +08:00
Liu An 63a4ed55d8 docs: update Docker build instructions for deps image (#16620)
### Summary

update Docker build instructions for deps image
2026-07-03 19:57:12 +08:00
Liu An 32c5cb16e9 Docs: Update version references to v0.26.3 in READMEs and docs (#16574) 2026-07-02 20:55:15 +08:00
Liu An f86a0e7386 Docs: Update version references to v0.26.2 in READMEs and docs (#16387) 2026-06-29 09:45:16 +08:00
Zhichang YuandClaude 3f805a64f1 feat(agent): align Go agent behavior with Python (except retrieval component) (#16225)
## Summary

Aligns the **Go agent runtime/canvas/components/tools** behavior with
the **Python `agent/` implementation** so the same stored canvas DSL
produces the same execution result on either side. Every component,
tool, and runtime primitive in `internal/agent/` is now driven by the
same semantics as its Python counterpart — variable resolution, template
substitution, control flow, error reporting, retry/cancel, and stream
event shapes.

The **retrieval component is the one explicit exception** in this PR. It
is being reworked in a separate change and is excluded from this
alignment pass; the wrapper slot (`universe_a_wrappers.go →
newRetrievalComponent`) is preserved.

## Scope of alignment

### Components (all aligned with `agent/component/`)
`Begin` · `Message` · `LLM` (incl. ChatTemplateKwargs,
MessageHistoryWindowSize, VisualFiles, Cite, OutputStructure,
JSONOutput, TopP, MaxRetries, DelayAfterError, credentials) · `Agent`
(react + tool artifact capture + `Reset()` interface-assert) · `Switch`
(12/12 operators, Python-equivalent semantics) · `Categorize` · `Invoke`
· `Iteration` · `Loop` (macro-expansion through `workflowx.AddLoopNode`)
· `UserFillUp` (Python-equivalent interrupt/resume via eino
`compose.Interrupt`/`ResumeWithData`) · `FillUp` · `DataOperations` ·
`ListOperations` · `StringTransform` · `VariableAggregator` ·
`VariableAssigner` · `Browser` (full stagehand runtime parity) ·
`DocsGenerator` · `ExcelProcessor`.

### Tools (all aligned with `agent/tools/`)
`Retrieval` (wrapper slot only — logic out of scope) · `MCPToolAdapter`
(streamable-HTTP) · `CodeExec` (sandbox bridge with
`code_exec_contract.go` matching Python contract) · `AkShare` · `ArXiv`
· `Crawler` · `DeepL` · `DuckDuckGo` · `Email` · `ExeSQL` · `GitHub` ·
`Google` · `GoogleScholar` · `Jin10` · `PubMed` · `QWeather` · `SearXNG`
· `Tavily` · `Tushare` · `Wencai` · `Wikipedia` · `YahooFinance` —
uniform `eino tool.InvokableTool` interface, SSRF protection, shared
HTTP client.

### Canvas execution engine (`internal/agent/canvas/`)
Aligned with Python's `agent/canvas.py`:
- **Scheduler** (`scheduler.go`): state pre/post handlers, node lambdas,
per-component timeout resolver (4-level: per-class env → per-class table
→ uniform env → 600s fallback), `legacyNoOpNames`.
- **Loop subgraph** (`loop_subgraph.go`): Python-equivalent
`AddLoopNode` macro expansion + condition translation.
- **Multibranch** (`multibranch.go`): `Switch` / `Categorize` routing
via `compose.NewGraphMultiBranch` — same branch selection semantics as
Python.
- **Parallel subgraph** (`parallel_subgraph.go`): matches Python's
parallel fan-out contract.
- **Interrupt/Resume** (`interrupt_resume.go`): `UserFillUpNodeBody` /
`IsInterruptError` / `ExtractInterruptContexts` — replaces the
deprecated Python sentinel chain with eino's native interrupt API,
preserving the same external behavior.
- **Checkpoint** (`checkpoint_store.go`): `RedisCheckPointStore`
Get/Set/Delete, with business metadata (status / canvas_id /
parent_run_id) on a parallel Redis Hash.
- **RunTracker** (`run_tracker.go`): Start / MarkSucceeded / MarkFailed
/ MarkCancelled / AttachCheckpoint — same lifecycle as the Python run
record.
- **Cancel** (`cancel.go`): Redis pub/sub watch.
- **Stream** (`stream.go`): SSE channel with `messages` / `waiting` /
`errors` / `done` events, same shape as Python's `agent.canvas.RunEvent`
payload.

### DSL bridge (`internal/agent/dsl/`)
- `normalize.go`: v1↔v2 collapsed into a single wire format — Python and
Go consume the same stored JSON.
- `reset.go`: per-run state reset matches Python's `Canvas.reset()`
semantics.
- Testdata mirrors Python's `agent_msg.json` / `all.json` / etc.

### Runtime (`internal/agent/runtime/`)
- `CanvasState` / `NewCanvasState` / `GetVar` / `SetVar` / `ReadVars`:
same `{{cpn_id@param}}` resolution model.
- `ResolveTemplate` (regex fast path + gonja fallback) — Python
Jinja-style semantics.
- `selector.go`, `metrics.go`, `component.go`: shared runtime contracts.

## Out of scope (intentionally)

- **`Retrieval` component logic** — wrapped only; full parity lands in a
follow-up PR.
- **Frontend** — only minor dsl-bridge / canvas UX fixes ride along.
- **CLI / admin / model registry** — orthogonal to agent behavior.

## How alignment is verified

`internal/service/agent_run_e2e_test.go` exercises the **full production
chain** against real Python-shaped DSL fixtures:
```
loadCanvasForUser → versionDAO.GetLatest → decodeCanvasFromDSL →
canvas.Compile → cc.Workflow.Invoke → answer extraction
```
using in-memory SQLite + miniredis (no Docker). Covers:
- `TestRunAgent_RealCanvas_BeginMessage` — happy path, `{{sys.query}}`
resolution
- `TestRunAgent_RealCanvas_WaitForUserResume` — two-run resume cycle
(Python-equivalent)
- `TestRunAgent_RealCanvas_CompileFails` — unknown component name →
sanitized error (Python-equivalent)
- `TestRunAgent_RealCanvas_InvokeFails` — unresolvable template ref
(Python-equivalent)
- `TestRunAgent_RunTracker_AttachCheckpoint_CallSequence` —
Start→AttachCheckpoint→MarkSucceeded lifecycle

`internal/handler/agent_test.go` — SSE streaming parity (`Content-Type:
text/event-stream`, `data: {…}\n\n`, trailing `data: [DONE]\n\n`,
OpenAI-compatible non-stream `choices`).

`internal/agent/canvas/fixture_compile_test.go` + per-component tests
pin the Python-equivalent outputs.

```
go test -count=1 -v -run 'TestRunAgent_RealCanvas|TestRunAgent_RunTracker' ./internal/service/
```

## Design reference

`docs/develop/agent-go-port-design.md` (1329 lines, last cross-checked
2026-06-17) — module layout, per-component / per-tool inventory,
corner-case catalogue, and the actionable backlog (Section 14, including
the retrieval alignment follow-up).

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 11:58:29 +08:00
Liu An 4379269374 Docs: Update version references to v0.26.1 in READMEs and docs (#16158)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.26.0 to v0.26.1
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-06-17 19:35:32 +08:00
Zhichang YuandClaude e45659868a feat(agent): ship the Go agent canvas port — eino interrupt/resume + Redis check-pointing (#16035)
Replaces the Python agent canvas runtime with a Go implementation that
runs inside `cmd/server_main`.

The canvas compiles into an eino Workflow that pauses on wait-for-user
via native Interrupt/Resume (no sentinel flag) and resumes from a
Redis-backed CheckPointStore.

All 21 Python agent components and ~35 tools are ported with functional
parity.

Sandbox providers now read their JSON config from the admin-panel
system_settings table with env fallback.

234 files / +35,413 / -6,111. All Go files are gofmt-clean (CI gate
added); drops the v2 DSL E2E step and the gap-analysis plan (both
redundant after the port ships).

## Type of change

- [x] Refactoring
- [x] New feature
- [x] Bug fix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-17 13:24:03 +08:00
Zhichang Yu 3fa15c0e2f feat(agent): Go port — canvas engine, 22 components, DSL v2, 13 endpoints (#15952)
Ports the agent canvas subsystem from Python to Go.

## What's included

### Canvas Engine (Phase 0/1)
- State engine, scheduler, variable resolver, Redis checkpoint store,
cancel protocol
- **209 tests** across canvas / component / io packages

### 22 Components (P0–P4)
| Tier | Components |
|---|---|
| P0 T1+T2+T3 | LLM, Agent, ExitLoop, Switch, Categorize, Begin,
Message, Invoke |
| P1 T3 | VariableAggregator, VariableAssigner, StringTransform,
ListOperations, DataOperations |
| P2 T3 | Iteration, IterationItem, Loop, LoopItem |
| P3 T3 | UserFillUp, Fillup |
| P4 T5 | Browser, ExcelProcessor, DocsGenerator |

### DSL v2 Schema (Phase 2.5)
- Typed v2 in-memory model with v1-to-v2 auto-detect converter
- v1 legacy field stripping per plan §2.11.7

### HTTP Endpoints & Bug Fixes (Plans PR1–PR3)
- **DELETE SQL bug fix**: gorm v2 `Where("id = ?", id).Delete(...)`
pattern
- **CreateAgent validation**: title/DSL required, duplicate check, 103
envelope
- **13 new endpoints**: templates, prompts, tags, sessions CRUD,
chat/completions (SSE + non-stream stubs), rerun, test_db_connection,
logs, webhook/logs
- **756 Go unit tests** (745 → 756, +18)
- **17 → 0 Python integration test failures** (test_agents.py +
test_session_management/)

### Tools
21 eino tools: HTTPHelper, search tools, financial/data tools, mandatory
stubs

### Infrastructure
OTel observability, NATS message queue, DeepDoc gRPC client, SSRF
guards, IDOR mitigation
2026-06-12 22:58:28 +08:00
Liu An 92c4b7688b Docs: Update version references to v0.26.0 in READMEs and docs (#15941)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.6 to v0.26.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-06-11 18:34:26 +08:00
Jack bea8092007 Update developer doc (#15336)
### What problem does this PR solve?

update developer doc

### Type of change

- [x] Documentation Update
2026-05-28 15:58:09 +08:00
Liu An 0639dba89a Docs: Update version references to v0.25.6 in READMEs and docs (#15248)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.5 to v0.25.6
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-26 19:45:43 +08:00
Jin Hai 775ea55679 Docs: update python version to 3.13 (#15103)
### What problem does this PR solve?

1. update python version to 3.13
2. upgrade ormsgpack to 1.6.0

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 19:09:19 +08:00
Jin Hai 90c76e73d0 Docs: Update version references to v0.25.5 in READMEs and docs (#15059)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-20 20:05:45 +08:00
Liu An f038a34154 Docs: Update version references to v0.25.4 in READMEs and docs (#14912)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.3 to v0.25.4
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-14 11:07:08 +08:00
Liu An 3182fd0789 Docs: Update version references to v0.25.3 in READMEs and docs (#14896)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.2 to v0.25.3
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-13 18:42:42 +08:00
Liu An 57b24be6d6 Docs: Update version references to v0.25.2 in READMEs and docs (#14731)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.1 to v0.25.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-09 19:06:05 +08:00