antirez
599e49d253
Add ds4-agent as a native terminal coding agent with session KV storage, DSML tool execution, edit/read/search/write/list/bash tools, async bash jobs, compaction support, history replay, and a linenoise-based interactive UI. The agent defaults to CRC-guarded edits, streams tool visualization, ignores tool calls emitted inside thinking, improves bash output observations for long-running commands, and styles prompts, spacing, and prefill progress for interactive use.
82 lines
3.5 KiB
Markdown
82 lines
3.5 KiB
Markdown
# DS4 Agent Context Compaction
|
||
|
||
The agent should compact context before long sessions become brittle, and also
|
||
when a tool result would otherwise not fit.
|
||
|
||
## Triggers
|
||
|
||
- **Soft trigger:** compact before a new user turn when the current transcript
|
||
is at least 85% of the configured context, or when no more than 8192 tokens
|
||
remain. For small contexts the free-token threshold is capped to one quarter
|
||
of the context size, so test runs with tiny `--ctx` values do not compact
|
||
after every short turn.
|
||
- **Hard trigger:** if a tool result cannot be appended with enough room left
|
||
for the model to answer, compact immediately and retry the append once.
|
||
|
||
## Model
|
||
|
||
Compaction uses the live session once, before rewriting the prompt:
|
||
|
||
1. Remember the current transcript length as `BOTTOM`.
|
||
2. Ask the model, internally, to emit a durable task-state summary of the
|
||
conversation up to `BOTTOM`.
|
||
3. Roll the real transcript back to `BOTTOM`; the summary request and response
|
||
are not part of the user-visible session.
|
||
4. Take the recent tail verbatim by scanning backward from `BOTTOM` up to 10%
|
||
of the configured context, capped at 50000 tokens. Align the tail to a
|
||
rendered `<｜User｜>` boundary when possible.
|
||
5. Rebuild the transcript as:
|
||
- system prompt and tool contract
|
||
- compacted durable-state summary
|
||
- recent verbatim tail
|
||
6. Re-sync the live KV state to the rebuilt transcript.
|
||
|
||
This keeps the part of the conversation that is most likely to contain active
|
||
work exactly as sampled, while summarizing older task state instead of merely
|
||
dropping it.
|
||
|
||
## Summary Contract
|
||
|
||
The summary should preserve durable task state only:
|
||
|
||
- user goals and constraints
|
||
- files inspected or edited
|
||
- commands run and important results
|
||
- decisions, rejected approaches, and known bugs
|
||
- current next steps
|
||
|
||
It should not invent facts or summarize raw file contents unless the assistant
|
||
already used those contents to reach a conclusion. Reloadable bulky data should
|
||
be described as such, with enough path/range information for the model to fetch
|
||
it again using tools.
|
||
|
||
The summary prompt explicitly tells the model to stop after the summary, and not
|
||
to continue the user task. This matters because the summarizer is invoked from a
|
||
live agent transcript where "next step" often means "call the next tool"; during
|
||
compaction, that next step must be recorded, not executed.
|
||
|
||
The compaction loop treats `</think>` and DSML as hard control boundaries. If
|
||
the model tries to close thinking or emit a tool call while summarizing, the
|
||
summary stops before that token. Internal compaction must never execute tools or
|
||
store DSML markup in the rebuilt system state.
|
||
|
||
## UI
|
||
|
||
During compaction the terminal should make the state explicit:
|
||
|
||
- show a high-intensity `COMPACTING` marker
|
||
- stream the generated summary so the user can see what is retained
|
||
- then show the normal prefill progress while the compacted context is rebuilt
|
||
|
||
## Validation Log
|
||
|
||
- With `--ctx 12000`, read `README.md` lines 1-500, then let the model call
|
||
`more count=500`. The second tool result did not fit, so hard compaction ran,
|
||
rebuilt the context from a summary plus a 1200-token verbatim tail, appended
|
||
the delayed tool result, and the model continued with the second chunk
|
||
available.
|
||
- Saved the same near-full state as a reusable session, then asked a follow-up
|
||
question. Soft compaction ran before appending the new user turn, preserved
|
||
that only lines 1-500 of `README.md` had been read, and the model answered the
|
||
follow-up correctly after the compacted context was rebuilt.
|