unclecode/toolshrink
Cut large agent tool output by what it means, not by where it was cut. 13 content-aware reducers + DeepSeek Harness plugin.
编辑导读Editor's Brief
## 核心特性
- toolshrink 通过识别文本形态并保留承载信息的部分，来缩减大体量 agent 工具输出，而不是仅按头部/尾部大小截断。
- 它包含 13 种 cut，分别用于 diff、JSON、测试、构建输出、堆栈跟踪、日志、文件树、重复行、lint 输出、安装输出、CSV/TSV 表格、git log，以及大小回退。
- 每次缩减都会在标记中报告被省略的字符或行数，避免产生不完整行和拆分 UTF-16 代理对，并且在第二次处理时保持幂等。
- 它可以通过 spill store 保存完整原始输出，并追加定位标记，使被移除内容能够逐字节取回。
- 它可以作为 DeepSeek Harness 插件使用，也可以作为 JavaScript 库使用，并支持可选的 tool、command 和 path 提示。
## 使用场景
- 在 DeepSeek Harness 中使用它，在长 shell、测试、构建或日志工具结果进入 agent 上下文前进行缩减。
- 在需要带预算文本缩减，并输出 strategy、note 和统计信息的 JavaScript 工具中，将其作为库使用。
- 使用自定义 cut 文件处理特定宿主环境的输出形态，例如 kubectl 或其他结构化命令结果。
## 技术细节
- Harness 适配器可通过 dsh plugin --profile web add github:unclecode/toolshrink 安装，并以默认 50,000 字符预算挂载。
- 可以通过 YAML 配置 maxChars、maxLines、maxLineChars、禁用的 cut、spill 目录和日志路径。
- cut 是暴露 name、detect 和 reduce 的纯文件，自定义 cut 可从目录加载，并会在内置 cut 之前被尝试。
- 库 API 返回 content、reduced、strategy、note 和 stats，而 spill store 接口支持基于文件或由宿主提供的存储。
## 注意事项
- 仓库将 pi 和 Codex 适配器列为待完成工作，而非已包含的集成。
- TODO 部分将 semantic、SQL 和 Docker cut 描述为计划中的功能，而非已交付功能。
- 使用默认的基于文件的 spill store 时，spill 文件会在 24 小时后清理。
## Core Features
- toolshrink reduces large agent tool output by recognizing text shape and keeping information-carrying parts instead of using only head/tail size cuts.
- It includes thirteen cuts for diff, JSON, tests, build output, stack traces, logs, file trees, repeated lines, lint output, install output, CSV/TSV tables, git log, and a size fallback.
- Every reduction reports omitted characters or lines in a marker, avoids partial lines and split UTF-16 surrogate pairs, and is idempotent on a second pass.
- It can save the complete original output through a spill store and append a locator marker so removed content can be retrieved byte for byte.
- It can be used as a DeepSeek Harness plugin or as a JavaScript library with optional hints for tool, command, and path.
## Use Cases
- Use it in DeepSeek Harness to shrink long shell, test, build, or log tool results before they enter the agent context.
- Use it as a library in JavaScript tools that need budgeted text reduction with strategy, note, and statistics output.
- Use custom cut files to handle host-specific output shapes such as kubectl or other structured command results.
## Technical Details
- The Harness adapter installs with dsh plugin --profile web add github:unclecode/toolshrink and mounts with a default 50,000-character budget.
- Configuration can set maxChars, maxLines, maxLineChars, disabled cuts, spill directory, and log path through YAML.
- Cuts are plain files exposing name, detect, and reduce, and custom cuts can be loaded from a directory and tried before built-in cuts.
- The library API returns content, reduced, strategy, note, and stats, while the spill store interface allows file-based or host-provided storage.
## Notes
- The repository lists adapters for pi and Codex as open work rather than included integrations.
- The TODO section describes semantic, SQL, and Docker cuts as planned rather than shipped features.
- Spilled files are cleaned after 24 hours when using the default file-based spill store.
安装Install
dsh plugin --profile web add github:unclecode/toolshrink把 unclecode/toolshrink 加入你的 DSH 配置（web profile）即可启用。
READMEREADME
toolshrink
Cut large agent tool output by what it means, not by where it was cut.
I use Claude Code every day, and I always wanted to intervene in how it manages context. In the early days you could edit the session JSONL directly. Then that door closed.
When DeepSeek open-sourced Harness, where everything is a plugin, I looked inside. Tool output there is cut by size: keep the head, keep the tail, drop the middle. I read Codex and pi, and they do the same. None of them look at what the text contains.
That fails in a predictable way. Your test suite prints 5,000 passing lines and 3 failures in the middle. A size cut keeps the passes and throws away the failures. The model reads it, believes the run, and answers wrong.
So I built the shrinker I always wished Claude Code had. It reads the output first, recognizes its shape, and keeps the part that carries the information:
input: a vitest run, 31,958 chars, 805 lines, budget 2,000 chars
head+tail cut: 1,904 chars the model learns: the summary
toolshrink: 255 chars the model learns: which test failed,
why, at which line, and the summary
Everything removed is counted in a marker the model can read, and the complete original is saved to disk with a locator. Nothing is lost silently.
The cuts
Each cut recognizes one shape of text. The first one that recognizes the input runs. When none does, the size fallback runs, so the result always fits the budget.
| Cut | Recognizes | Keeps | Drops |
|---|---|---|---|
diff |
git diff, patches | changed lines, file and hunk headers, 1 context line each side | unchanged context |
json |
one JSON value | the structure, 3 samples per long array, 5 keys per wide object, counts | repeated records |
tests |
vitest, jest, pytest, cargo test, go test | failures with their explanation, the summary | passing tests |
build |
tsc, cargo, gcc, webpack, esbuild | errors and warnings with their code frame, the summary | build progress |
stacktrace |
node, Python, Java, Ruby traces | the message and frames in YOUR code | dependency frames, counted |
log |
timestamped logs | errors and warnings with the lines before them, the ending | routine lines |
tree |
find, ls -R, file listings | the structure, 8 entries per directory, counts | crowded directories |
repeat |
retry storms, progress spam | 2 samples per run plus "2,998 similar lines omitted" | consecutive near-identical lines |
lint |
eslint, ruff, clippy | each rule with its count and example locations, worst files | repeated occurrences of the same rule |
install |
npm, pip, pnpm, cargo | the summary, versions, deprecations, vulnerabilities, errors | fetch and download progress |
csv |
CSV, TSV, pipe tables | header, 5 rows from the start, 2 from the end, row and column counts | the rows between |
gitlog |
git log, both formats | the 15 newest commits, the total, the authors with counts | older commits |
size |
everything (fallback) | bash: the end · grep/read: the start · unknown: both ends | the rest, counted |
Thirteen cuts ship today. Each one is a plain file with a shared interface, so adding your own is one file, not a fork.
Every cut follows four rules, taken from the three agents I read:
- never return a partial line (from pi)
- never split a UTF-16 surrogate pair (from DeepSeek Harness)
- say exactly how much was removed:
... 15,903 characters, 401 lines omitted ...(from Codex) - a second pass changes nothing
Use it with DeepSeek Harness
One command:
dsh plugin --profile web add github:unclecode/toolshrink
That is the whole install. The package carries a dsh.bundle manifest, so
the plugin mounts with a 50,000-character default budget on the next start.
Change the budget from your own layer, ~/.dsh/cordis.patch.yml:
- id: toolshrink
config:
maxChars: 20000
log: /tmp/toolshrink.log
Hacking on it instead? Clone, npm install && npm run build, and mount the
adapter file by path with an insert row (see Adapter config below).
Adapter config
- insert:
- id: toolshrink
name: /path/to/toolshrink/adapters/harness/toolshrink.mjs
config:
maxChars: 50000 # cut above this many characters (default 50000)
maxLines: 2000 # or above this many lines (default 2000)
maxLineChars: 0 # cap single long lines, 0 = off (default 0)
disable: [json] # skip named cuts (default none)
spillDir: ~/.dsh-toolshrink # where full originals go
log: /tmp/toolshrink.log # one line per cut, omit for silence
The log line format: bash 64151 -> 2942 via tree+size.
Use it as a library
import { shrink, FileSpillStore } from 'toolshrink'
const out = shrink(bigText, { tool: 'bash', command: 'npm test' }, {
budget: { maxChars: 20_000 },
spill: new FileSpillStore({ dir: '/tmp/spills' }), // optional
})
out.content // the text to give the model
out.reduced // false when the input already fit
out.strategy // "tests", "diff+size", "size:tail", "none", ...
out.note // one human-readable line about what happened
out.stats // inputChars, outputChars, keptLines, droppedLines, ...
The hint (second argument) is optional and improves routing: tool picks the
size direction, command helps detect test runs and diffs, path helps detect
JSON and logs.
Write your own cut
A cut is one file that default-exports three members. The file name is the cut name.
// mycut.mjs
export default {
name: 'mycut',
// Cheap and certain. When unsure, return false: a wrong match is worse
// than the size fallback.
detect(text, hint) {
return hint.command?.startsWith('kubectl') ?? false
},
// Return null to decline after a closer look; the next cut then tries.
reduce(text, hint, budget) {
const content = text.slice(0, budget.maxChars) // your real logic here
return {
content,
reduced: true,
strategy: 'mycut',
note: 'kept the part I know matters',
stats: {
inputChars: text.length, inputLines: 0,
outputChars: content.length, outputLines: 0,
},
}
},
}
Use it:
import { shrink, loadReducers } from 'toolshrink'
const mine = await loadReducers('/path/to/my-cuts') // reads the directory
shrink(text, hint, { extra: mine }) // tried BEFORE built-ins
Or control the built-ins: { only: ['tests', 'diff'] } restricts and orders,
{ disable: ['json'] } skips.
Spill: nothing is lost
With a spill store, the complete original is saved before any cut, and the cut text ends with:
[full output saved as spill:bash-d63d2aebb643: directories sampled to 8 entries each]
store.load('spill:bash-d63d2aebb643') returns the original, byte for byte.
Files are cleaned after 24 hours. The store is an interface; the default writes
files, a host can plug its own storage.
What I saw in live use
With a 3,000-character budget, the agent got a 60,000-character find result
cut down to its head. Its reply began: "The output was truncated. Let me get a
count by directory" - it saw the omission marker, re-queried with aggregation,
and answered correctly from 4,000 total characters instead of 60,000.
That is the design working: an honest marker turns a cut from silent data loss into a signal the model acts on. This is the intervention I always wanted, and now it is a YAML row.
TODO: cuts I want next
Each of these is one file with the same interface. Pick one and send a pull request.
| Cut | Recognizes | Would keep |
|---|---|---|
semantic |
anything, given the agent's current goal | the chunks most relevant to the goal. Two stages: lexical scoring (BM25, no model needed), then optional embedding scoring for meaning beyond shared words |
sql |
query results, EXPLAIN plans | the plan's expensive nodes, sampled result rows |
docker |
build and compose output | the failing layer, the final image, dropped build chatter |
The semantic cut is the interesting one: every cut above decides by SHAPE,
this one would decide by RELEVANCE. It needs one extra input, a query for what
the agent is working on right now, which the host adapter can pass through the
hint.
Adapters for other agents
The library knows nothing about any agent. The Harness adapter is 70 lines:
catch the result event, call shrink, return the replacement.
- pi (
earendil-works/pi) has an extension API with tool-result access. - Codex (
openai/codex) has a plugin system incodex-rs/core-plugins.
Both adapters are open work. If you write one, a pull request is welcome.
License
MIT. Use it, change it, no need to ask.
Built by @unclecode, author of
Crawl4AI
相关推荐Related
- deepseek-ai/deepseek-harness DeepSeek Harness (dsh) — 由 DeepSeek AI 开发的开源 agent harness（智能体框架）。一切皆插件：核心 + 插件 + 技能 = 可热插拔的 AI 编程工作台。DeepSeek Harness: …
-
amruthpillai/reactive-resume
A one-of-a-kind resume builder that keeps your privacy in mind. Completely secure, customizable, portable, open-source a…
-
tt-a1i/archify
Agent skill for beautiful, verifiable architecture, workflow, sequence, data-flow, and lifecycle diagrams—self-contained…
-
strukto-ai/mirage
The World's First Unified Virtual Filesystem For AI Agents
-
liustack/modlens
The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, ge…
-
omdsh-dev/DSH-better-sidebar
开放的侧边栏底座，支持三方拓展注册新侧边栏页面。内置文件渲染编辑/终端/Git/子代理页面 ｜ Open sidebar foundation, supports third-party extensions to register new…
-
ccch1mneyyy/dsh-TUI
DSH 官方公众号收录的 TUI 补位插件：Claude Code 风，鲸鱼顶栏/实时状态/流式思考/双击 Esc 回滚/上下文进度+TPS。npm 一键装。 DSH official WeChat featured TUI plugin…
-
dsh-market/dsh-market
The plugin market inside DeepSeek Harness — browse, search, one-click install · DSH 可视化插件市场