🛩️ dsh-pilot — give your DSH agent hands

中文说明 · DeepSeek Harness plugin

Drive a real browser from the DeepSeek Harness chat: the agent opens pages, reads them as structured text with a numbered element list, clicks and types by ref (no CSS guessing), presses keys, navigates back/reload, waits, evaluates JS, and takes screenshots — while you watch a live draggable cockpit panel in the Web GUI and can take over at any time.

• 🚀 One command install — dsh plugin --profile web add github:guo6x/dsh-pilot
• ⚡ Zero runtime dependencies — talks CDP over the native Node ≥ 22 WebSocket, uses the Edge/Chrome already on your machine
• 🔑 No API key — nothing leaves your machine; no vision model required
• 📖 Text-first by design — the agent reads DOM snapshots (title/URL/text/links + numbered elements), so text-only models browse without burning vision tokens
• 🎯 Ref-driven interaction — every click/type targets a snapshot ref, not a guessed selector; stale refs fail loudly with a hint
• 📝 Human-named forms — fill several fields by label, aria-label, placeholder, name, or id, and upload files by ref
• 🧭 Full navigation set — back, reload, and wait tools for real browsing flows, with page-settling waits built in
• 👀 Human in the loop — live screenshot, URL bar, action log, and a session switcher in the cockpit; you see everything the agent does
• 🧩 Per-session isolation — every agent session gets its own browser instance; parallel sessions never fight over one page

Watch it work

Open the ✈️ cockpit, start its isolated browser, and watch the page change live. When several agents are browsing, the demo also pins one session, then returns to automatic following.

Install — copy, paste, confirm

# Install from GitHub — this is the supported release channel.
dsh plugin --profile web add github:guo6x/dsh-pilot

Restart a running dsh web process, then refresh the page. Installation is complete when a ✈️ button appears at the bottom of the sidebar. Click it to open the cockpit.

Requirements: the DeepSeek Harness web profile, Node ≥ 22, and Edge or Chrome installed. The plugin needs no extra account, API key, or browser download.

Developing from a checkout instead? Run dsh plugin --profile web add . from the repository directory. The committed lib/ files mean GitHub installs do not run a build script.

First safe task

Start a new chat and paste this:

Open https://example.com, tell me what the page says, then verify that the “Learn more” link is visible.

You should see a structured answer with the page title and a numbered link, while the cockpit shows the same page. This task only reads a public page.

If the ✈️ button is missing

• Confirm the plugin is installed in the web profile: dsh plugin --profile web list dsh-pilot.
• Restart the dsh web process after installing; a browser refresh alone cannot load new host code.
• Check that Node is version 22 or newer and that Edge or Chrome is installed; the cockpit reports a launch error when neither is available.

See it work in 60 seconds

For a real staging form, give the agent one bounded task:

Open our staging application form. Fill the contact fields, upload the approved resume at C:\\path\\to\\resume.pdf, verify the entered values, and stop before final submission.

The agent gets structured page evidence, uses labels and snapshot refs instead of brittle guesses, and the cockpit lets you watch or stop the browser at any time.

Why it feels different

What the agent gets

The agent just says what it needs: "open the login page, fill the form, click submit, and read the result" — the tools are the same verbs.

What the human gets

A draggable cockpit overlay: live screenshot (2 s refresh), current URL + title, start/stop buttons, an address bar, and the recent action log. When several sessions are browsing, choose one to inspect or leave it on Latest activity to follow the agent automatically. Everything the agent does is visible; close the browser or take over whenever you like.

Known limitations

• One tab per session. Refs are pinned to the current page, so a tab switcher would invalidate them. Need a second context? Spawn a subagent — each agent session gets its own browser.
• Headless only. The cockpit shows the headless view; there is no headed mode (a human driving the same browser is a different product).
• The panel follows the most recently used session by default. When several sessions exist, you can pin the cockpit to any one of them; switch back to Latest activity to resume automatic following.

How it works

DSH chat ──pilot_* tools──▶ host plugin ──CDP (native WebSocket)──▶ headless Edge/Chrome
    ▲                              │
    └── structured text snapshots ◀┘
GUI cockpit ◀──/dsh-pilot/state + /dsh-pilot/shot.png (loopback)──┘

• Launches msedge/chrome headless with an isolated --user-data-dir under the OS temp dir and a dynamically picked debugging port (9222+); the whole tree is killed and the profile removed on stop.
• The host registers 17 tools plus a loopback-only HTTP API (/dsh-pilot/*, 403 for non-loopback clients).
• The client is a small overlay panel registered in sidebar.footer.action + shell.overlay.

Security

• Browser runs headless with an isolated profile; it never touches your real browser session.
• The HTTP API binds to the DSH server (loopback by default) and rejects non-loopback clients explicitly.
• pilot_open accepts http(s) URLs only; pilot_eval runs page-context JS (same trust as opening DevTools yourself — do not point the agent at pages you don't trust).
• pilot_upload accepts existing absolute regular files only; upload only files the user has authorized for the current site.
• No telemetry, no network calls to third parties, no API keys.

Develop

pnpm install
pnpm test             # build, real-headless-Edge flow, and package-content check

MIT licensed. Found a bug or an idea? Open an issue.

Related

• Chinese dev log (掘金): 我给我的 agent 装了双手：零依赖浏览器操控插件开发记