dongsheng123132/dsh-benchmark

Name: dongsheng123132/dsh-benchmark
Author: dongsheng123132

Plugin插件 Native原生 ⭐ 3 MIT Vision & Media视觉与多媒体

DeepSeek Harness 的确定性修订锁定基准与回归证据

Project Overview项目介绍

Edited by Zhiwei Agent (zhiwei)由知微Agent(zhiwei)编辑

dsh-benchmark is a native benchmarking plugin built exclusively for DeepSeek Harness (DSH) that provides reproducible, deterministic benchmark evidence for DSH tools and plugins. To install the plugin in DSH, you can run the simple command dsh plugin --profile benchmark add github:dongsheng123132/dsh-benchmark directly from your DSH command line interface. It registers three core commands after installation: dsh_benchmark_inspect for metadata and fingerprint inspection without execution, dsh_benchmark_run for executing fixed test cases and writing a content-addressed report, and dsh_benchmark_compare for comparing current and baseline reports against manifest thresholds. It also works as a standalone CLI and a proof-only MCP server that validates manifests and computes report hashes.

dsh-benchmark is designed for DSH plugin developers who need to run consistent regression tests on their own plugins before releasing public version updates. Each benchmark run records separate observations for warmup and measured runs, tracking duration in nanoseconds, exit code, timeout status, output hashes, expectation checks and other key metrics. It excludes raw command inputs, outputs, environment variables and timestamps from the final report to keep results fully deterministic and secure. Developers define all test parameters in an explicit manifest that freezes test case details, execution constraints, scoring rules and passing thresholds.

dsh-benchmark requires Node.js version 22 or higher to run, and has no runtime dependencies beyond the optional DSH tools SDK peer dependency. It uses a strict safety model that disables shell execution and PATH lookup, prevents working directory traversal escapes, rejects secret-bearing fields in manifests, and restricts artifact writes to explicit content-addressed directories. This model prevents accidental shell expansion and environment credential leakage, but it is not an operating system-level sandbox for malicious code. The project is released under the permissive MIT open source license, and developers should only run benchmark executables from trusted sources.

这是一个专为DeepSeek Harness（DSH）开发的原生插件，用于为DSH工具和插件提供可复现、确定性的基准测试证据。它不重复dsh-batch-regression的功能，而是围绕固定测试用例定义了一套完整的证据协议，包含明确的版本修订、文件指纹、受限执行、版本化评分和基线回归对比等能力。当前版本遵循DSH官方加载契约，同时支持作为独立命令行工具和MCP服务器运行。

该插件面向DSH插件开发者，帮助他们在发布版本更新前对自己开发的插件进行回归测试，确保代码修改不会破坏原有功能。开发者可以通过指定测试套件名称、用例版本、目标修订版、执行参数、预期结果和评分阈值等信息，创建一个明确的基准测试清单，每次运行测试都会生成带指纹的内容寻址报告，方便对比不同版本的表现。

本项目采用MIT许可证开源，运行依赖Node.js 22及以上版本，除了可选的DSH工具SDK外没有额外运行时依赖。它内置严格的安全模型，禁止使用shell执行、PATH查找，限制工作目录逃逸，不允许包含密钥信息，能够防止意外的环境泄露，但不提供针对恶意代码的操作系统级沙箱，开发者仅可运行可信的基准测试可执行文件。

View on GitHub在 GitHub 查看

Send to your DSH发给你的 DSH 安装

请帮我安装这个 DSH 插件。安装前先完成【兼容性检查 + 安全性检查】，检查通过再动手。

插件：dsh-benchmark（dongsheng123132/dsh-benchmark）
仓库：https://github.com/dongsheng123132/dsh-benchmark
本站详情页：https://www.yhbd.top/plugins/dongsheng123132-dsh-benchmark/
本站登记：类型 plugin · 归类 原生 DSH 插件 · 许可证 MIT · ⭐ 3 · 最近提交 2026-09-07 · 主语言 JavaScript

按下面顺序执行，每步先把结论告诉我，再进入下一步：

【1 兼容性检查】
① 我这边：DSH 版本、Node 版本、操作系统、当前 profile（web / desktop）。
② 读它的 README、package.json、插件 manifest，列出它要求的 DSH 版本 / Node 版本 / 操作系统 / 外部依赖 / 需要另外先装的运行时。
③ 逐条比对，结论只写「满足 / 不满足 / 未知」三种；不满足的给出可行替代方案。
④ 检查是否和我已装的插件冲突：命令名重复、skill / tool 重名、端口占用、重复注册的 MCP server。

【2 安全性检查】
① 仓库可信度：和上面「本站登记」是否一致；star / fork 数、创建时间、最近提交，是否归档或长期停更。
② 安装脚本：逐行看 package.json 的 preinstall / install / postinstall，以及 install.sh、setup.ps1 之类脚本。出现 curl|bash、下载后直接执行、混淆代码、访问与插件功能无关的域名，立刻停下来告诉我，不要继续装。
③ 依赖：列出新增依赖，标出无人维护、或与知名包拼写近似的可疑包（typosquatting）。
④ 权限与副作用：它会读写哪些目录、访问哪些域名、需要哪些 DSH 权限（filesystem / network / shell / clipboard 等），以及怎么卸载和回滚。
⑤ 如果它要求 sudo / 管理员权限，或权限明显超出功能所需，先停下来问我。

【3 安装】
上面两步没有「不满足」和「高危项」时才执行；用官方推荐方式安装，不要自行提权。

【4 汇报】
用表格输出：检查项 / 结论 / 依据 / 是否需要我决策。拿不准的一律写「未知」并说明要我怎么确认——不要猜，也不要替我决定。

Send this message to DSH in your current session: it verifies compatibility and security first (answering met / not met / unknown item by item) and only installs once everything checks out — it will stop and ask you if it finds a high-risk item. The box scrolls; the copy is the full prompt. CLI install commands may not be accurate across systems, so DSH is the safer route.把上面这条消息直接发给当前会话里的 DSH：它会先核对兼容性与安全性（逐条给「满足 / 不满足 / 未知」），确认没问题再安装，有高危项会停下来问你。框内可滚动，复制到的是完整提示词；安装命令不一定准确，发给 DSH 更稳。

Pre-install check安装前体检Compatibility · Security兼容性 · 安全性 1 warning1 项注意

Only 3 stars - very few users, little community feedback星标只有 3，几乎没人在用，遇到问题缺少社区反馈

DSH walks through these 9 checksDSH 会逐条核对这 9 项

Compatibility兼容性

DSH, Node, OS and profile requirementsDSH 版本 / Node 版本 / 操作系统 / profile 是否满足要求
External dependencies and runtimes (Electron / Python / Docker, ...)外部依赖与运行时（Electron / Python / Docker 等）是否齐备
Conflicts with installed plugins: command names, skill / tool names, ports, duplicate MCP registration与已装插件是否冲突：命令名、skill / tool 重名、端口占用、重复 MCP 注册

Security安全性

Repo matches the facts registered here; archived or abandoned?仓库是否与页面登记一致，是否归档或长期停更
Safety of preinstall / install / postinstall and install.sh / setup.ps1preinstall / install / postinstall 与 install.sh、setup.ps1 是否安全
curl|bash, download-then-execute, obfuscation, unrelated domains → stop immediatelycurl|bash、下载即执行、混淆代码、无关域名 → 立刻停止
Typosquatting or unmaintained packages among the new dependencies新增依赖里有没有 typosquatting 或无人维护的包
Requested permissions vs. what the feature actually needs申请了哪些权限、是否超出功能所需（filesystem / network / shell / clipboard）
Any sudo / admin requirement, plus uninstall and rollback是否要求 sudo / 管理员权限，以及卸载与回滚方式

Anything uncertain must be marked unknown with a note on how to confirm it. This site's signal screen is a static snapshot, not a security audit.拿不准的必须标「未知」并说明要我怎么确认。本站的信号筛查是静态快照，不能替代安全审计。

Or use CLI install (for developers)或使用命令行安装（适合开发者）

CLI Install命令行安装

dsh plugin --profile benchmark add github:dongsheng123132/dsh-benchmark

把 dongsheng123132/dsh-benchmark 加入你的 DSH 配置（web profile）即可启用。

READMEREADME

5.7 KB · 缓存于 2026-08-23 03:42:15 · 5.7 KB

dsh-benchmark

Reproducible, deterministic benchmark evidence for DeepSeek Harness tools and plugins.

This project deliberately does not duplicate dsh-batch-regression, which runs one shell command repeatedly for median/distribution statistics. dsh-benchmark defines an evidence protocol around fixed cases: explicit target and suite revisions, file-derived target fingerprints, bounded argv-only subprocesses, raw measurements, versioned deterministic scoring, content-addressed reports, and baseline regression comparison.

The first release evaluates commands and JSONL runners, not subjective LLM quality.

Version 0.2.0 is a formal Codex plugin and standalone proof-only MCP server, and uses the namespace export shape required by the stock DSH Web Loader. A real Cordis boot regression test guards that loader contract.

Adjacent benchmark skills often grade Skill or LLM quality. This project stays at the deterministic execution-evidence layer: fixed target revisions and cases, raw bounded measurements without raw business output, versioned scoring, content-addressed reports, and baseline regression decisions.

Evidence model

An explicit manifest freezes:

suite name and case revision;
target name, claimed revision, and files used to recompute its fingerprint;
executable, constrained working directory, warmup/repeat counts, timeout, output cap, and concurrency cap;
fixed argv and optional JSONL stdin for every case;
expected exit code, stdout/stderr SHA-256, and optional JSONL line count;
scorer version, minimum pass rate, output-stability rule, and maximum median-latency regression.

Showing the opening section of the README — the full document lives in the repository以上为 README 开头摘要，完整文档在仓库内 · View the full README on GitHub →在 GitHub 查看完整 README →

渲染基于仓库 dongsheng123132/dsh-benchmark 的 README.md。在 GitHub 查看原文 →