CAI-MH/dsh-quality-review

Plugin插件 Native原生 ⭐ 2 MIT Approval & Security审批与安全

AI response quality review for DeepSeek Harness: audits each finished assistant turn with an independent reviewer model and steers the agent to fix unreasonable output (up to 2 review rounds).

Project Overview项目介绍

This is a quality review plugin built exclusively for DeepSeek Harness (DSH) that audits AI assistant outputs before each turn closes. It integrates with DSH’s agent/turn-stopping event hook, capturing the most recent assistant reply before the turn ends and running it through a reviewer model to check for quality issues. You can install it via the DSH CLI with the command dsh plugin --profile web add dsh-quality-review, or add it manually to your profile’s package.json dependencies and restart the DSH web interface.

The plugin checks four core quality aspects by default: factual accuracy, completeness, logical consistency, and instruction following. Each of these aspects can be toggled on or off individually via configuration, so you can tailor the review to your specific use case. For common SOP-style tasks, you can add custom SOP files to the default DSH_HOME/quality-review/sop folder, and the plugin will automatically match tasks by filename and add an extra check for SOP compliance.

All configuration options come with safe defaults, so you can run the plugin immediately after install without any extra setup. The plugin allows a maximum of 5 review rounds per turn, with the default set to 2, and you can configure an exempt pattern list to skip review for prompts that match specific keywords. It is released under the permissive MIT license, and developers can modify the source code after installing npm dependencies and compiling the TypeScript source with tsc.

这是一个专为 DeepSeek Harness (DSH) 开发的原生质量审核插件,用于在每轮对话结束前检查AI助手输出是否符合预设质量要求。它通过agent/turn-stopping钩子接入DSH的对话流程,获取助手最新回复后调用审核模型检查四项核心质量维度,不通过则引导代理修正输出,每轮对话最多允许两次修正,避免不合格输出流到用户端。

安装后插件会自动在每轮对话结束前触发审核,默认开启所有四项质量维度,同时支持针对常规SOP类任务自定义审核标准。用户只需将任务对应的SOP规范放入指定文件夹,插件就能自动匹配任务类型,额外增加是否符合SOP规范的审核维度。

该插件使用MIT许可证开源,开发时需要先安装npm依赖,再通过tsc编译TypeScript源码得到可运行产物。插件限制每轮对话最多只能进行5次修正,用户还可以配置豁免关键词列表,匹配到关键词的对话会直接跳过审核流程不进行检查。

Pre-install check安装前体检Compatibility · Security兼容性 · 安全性 1 warning1 项注意
  • Only 2 stars - very few users, little community feedback星标只有 2,几乎没人在用,遇到问题缺少社区反馈
DSH walks through these 9 checksDSH 会逐条核对这 9 项

Compatibility兼容性

  • DSH, Node, OS and profile requirementsDSH 版本 / Node 版本 / 操作系统 / profile 是否满足要求
  • External dependencies and runtimes (Electron / Python / Docker, ...)外部依赖与运行时(Electron / Python / Docker 等)是否齐备
  • Conflicts with installed plugins: command names, skill / tool names, ports, duplicate MCP registration与已装插件是否冲突:命令名、skill / tool 重名、端口占用、重复 MCP 注册

Security安全性

  • Repo matches the facts registered here; archived or abandoned?仓库是否与页面登记一致,是否归档或长期停更
  • Safety of preinstall / install / postinstall and install.sh / setup.ps1preinstall / install / postinstall 与 install.sh、setup.ps1 是否安全
  • curl|bash, download-then-execute, obfuscation, unrelated domains → stop immediatelycurl|bash、下载即执行、混淆代码、无关域名 → 立刻停止
  • Typosquatting or unmaintained packages among the new dependencies新增依赖里有没有 typosquatting 或无人维护的包
  • Requested permissions vs. what the feature actually needs申请了哪些权限、是否超出功能所需(filesystem / network / shell / clipboard)
  • Any sudo / admin requirement, plus uninstall and rollback是否要求 sudo / 管理员权限,以及卸载与回滚方式

Anything uncertain must be marked unknown with a note on how to confirm it. This site's signal screen is a static snapshot, not a security audit.拿不准的必须标「未知」并说明要我怎么确认。本站的信号筛查是静态快照,不能替代安全审计。

Or use CLI install (for developers)或使用命令行安装(适合开发者)

CLI Install命令行安装

dsh plugin --profile web add dsh-quality-review

把 CAI-MH/dsh-quality-review 加入你的 DSH 配置(web profile)即可启用。

READMEREADME

dsh-quality-review

English | 中文

AI response quality review for DeepSeek Harness (DSH): audits each finished assistant turn with an independent reviewer model and steers the agent to fix unreasonable output (up to 2 review rounds).

How it works

On agent/turn-stopping — the moment a turn is about to close — the plugin takes the assistant's latest visible reply, audits it with a reviewer model, and when the verdict is fail it steers the agent with a concrete fix request so the turn stays open and the model revises. Each turn allows at most maxRounds steered follow-ups (default 2), which is the hard loop guard.

Reviewed aspects (each can be toggled):

  • Factual accuracy — obvious factual errors and fabricated information
  • Completeness — any part of the user's question left unanswered
  • Logical consistency — contradictory reasoning and unsupportable conclusions
  • Instruction following — violations of explicit user format/language/constraint requirements

Install

dsh plugin --profile web add dsh-quality-review

Or add it to your profile's package.json dependencies and list it under dsh.profile.bundles, then restart dsh web.

Configuration

Every field has a safe default, so the bundle row can be inserted with no config at all. Override individual keys from your profile's cordis.patch.yml:

- id: quality-review
  config:
    enabled: true            # master switch; the plugin stays loaded but inert when false
    reviewer:
      provider: ''           # reviewer provider id; empty reuses the agent's own provider
      model: ''              # reviewer model id; empty reuses the agent's own model
    maxRounds: 2             # max steered fix rounds per turn (loop guard, max 5)
    aspects:
      factualAccuracy: true
      completeness: true
      logicalConsistency: true
      instructionFollowing: true
    minReplyChars: 200       # skip replies shorter than this
    sop:
      enabled: true          # SOP reference-standard switch (default on)
      dir: ''                # SOP folder; empty uses DSH_HOME/quality-review/sop
    exemptPatterns:          # extra static keywords: skip review on match (optional)
      - 代码 review

Showing the opening section of the README — the full document lives in the repository以上为 README 开头摘要,完整文档在仓库内 · View the full README on GitHub →在 GitHub 查看完整 README →

← 上一个 Prev dsh-remote-tunnel-easy 下一个 Next dsh-live2d-afterglow →