Fisfzy/dsh-ego-browser

资源Resource ⭐ 27 automationbrowserweb-ui

DSH(DeepSeek Harness)插件:把 ego-lite 浏览器(给 AI Agent 用的 Chromium)接入 HARNESS——13 个结构化 ego_* 工具(文本语义快照、语义定位点击、表单填充、截图、CDP 控制、任务空间隔离),内置 ego 运行时,Linux + Chrome 开箱即用,无需克隆官方仓库或手动构建。

编辑导读Editor's Brief

## 核心特性
- 该插件将 CitroLabs/ego-lite 集成到 DeepSeek Harness 中,并提供 32 个结构化 ego_* 工具,覆盖空间、页面读取、导航、等待、交互、执行、输出、会话处理与诊断。
- 它提供实时观察窗口,可串流显示智能体当前操作的页面,并包含标签栏、历史抽屉,以及对同一浏览器实例的直接鼠标和键盘控制。
- 它支持 CDP JPEG 截屏串流和可选的 FFmpeg H.264 采集,默认使用 CDP 后端;只有在兼容安装通过能力检查后,才可选择 FFmpeg。
- 它内置 ego 运行时和 --no-sandbox 包装器,可自动检测 Chrome、Chromium、Brave 或 Edge,并处理 root、Docker 和无头环境回退,无需手动构建。
- 它包含 ego_downloadego_captchaego_auth_flushego_httpego_doctor 等运维工具,并提供 worker 单实例守护、崩溃恢复、冷启动重试和前端帧缓存限制。

## 使用场景
- 智能体可执行需要登录的文献或数据采集,操作人员可在观察窗口中查看滚动、翻页和 PDF 下载过程。
- 智能体可填写表单或处理登录流程;当出现验证码或其他阻塞式交互时,用户可以直接接管。
- 智能体可对 Web 产品运行 QA 或冒烟测试,历史抽屉可用于回看已访问的操作轨迹。
- 智能体可以操作 DSH 界面本身,同时同一观察路径可显示会话管理、任务面板或设置变更。

## 技术细节
- 每个工具会将参数转换为 JavaScript 脚本,通过 ctx.subprocess 使用 ego-browser nodejs 运行,经由 CDP 驱动共享的 Chromium,并从 @@DSH_RESULT@@ 哨兵行解析结果。
- 所有 ego_* 操作均由进程内互斥锁串行化,错误会在返回宿主前进行规范化处理。
- 观察栈使用 lib/client.js 处理 watcher 租约、JPEG 图像和 MediaSource 视频,同时由 lib/cast-server.js 代理元数据 SSE、观察 API 以及带背压的二进制视频。
- CDP 采集使用 Page.startScreencast 输出 JPEG,默认 20 FPS,并带帧 ID 确认;FFmpeg 采集在 Windows 上使用 gfxcapture(hwnd),在其他平台上使用显示源裁剪,并交付分片 MP4 数据块。

## 注意事项
- 观察窗口需要图形化 DSH Web 会话;无头会话仍可使用 ego_* 工具,但无法实时查看。
- Windows 支持是在插件层适配的,但底层 ego-lite 宿主仍是社区移植版本,在复杂多步骤流程中可能稳定性较低。
- 运行时 Cookie 仅在优雅关闭时写入磁盘,因此强制重启后需要重新登录;多任务空间之间的 Cookie 保持隔离。

## Core Features
- The plugin integrates CitroLabs/ego-lite into DeepSeek Harness and exposes 32 structured ego_* tools for spaces, page reading, navigation, waiting, interaction, execution, output, session handling, and diagnostics.
- It provides a real-time observation window that streams the page currently operated by an agent and includes a tab bar, a history drawer, and direct mouse and keyboard control over the same browser instance.
- It supports CDP JPEG screencast and optional FFmpeg H.264 capture, with CDP as the default backend and FFmpeg selectable only after a compatible installation passes capability checks.
- It bundles an ego runtime and a --no-sandbox wrapper, automatically detects Chrome, Chromium, Brave, or Edge, and handles root, Docker, and headless fallbacks without requiring a manual build.
- It includes operational tools such as ego_download, ego_captcha, ego_auth_flush, ego_http, and ego_doctor, plus worker single-instance guarding, crash recovery, cold-start retry, and front-end frame-cache limits.

## Use Cases
- Agents can perform authenticated literature or data collection while an operator watches scrolling, pagination, and PDF downloads in the observation window.
- Agents can fill forms or handle login flows, and a user can take over directly when a captcha or other blocking interaction appears.
- Agents can run QA or smoke tests on web products while the history drawer allows review of visited trajectories.
- Agents can operate the DSH interface itself while the same observation path shows session management, task boards, or settings changes.

## Technical Details
- Each tool converts parameters into a JavaScript script, runs it through ctx.subprocess with ego-browser nodejs, drives a shared Chromium through CDP, and parses results from an @@DSH_RESULT@@ sentinel line.
- All ego_* actions are serialized by an in-process mutex, and errors are normalized before being returned to the host.
- The observation stack uses lib/client.js for watcher leases, JPEG images, and MediaSource video, while lib/cast-server.js proxies metadata SSE, watch APIs, and binary video with backpressure.
- CDP capture uses Page.startScreencast JPEG at a default 20 FPS with frame-ID acknowledgment, while FFmpeg capture uses Windows gfxcapture(hwnd) or display-source cropping on other platforms and delivers fragmented MP4 chunks.

## Notes
- The observation window requires a graphical DSH web session; headless sessions can still use ego_* tools without live viewing.
- Windows support is adapted at the plugin layer, but the underlying ego-lite host remains a community port and may be less stable for complex multi-step flows.
- Runtime cookies are written to disk only during graceful shutdown, so forced restarts require logging in again, and multi-task spaces keep cookies isolated.

安装Install

git clone https://github.com/Fisfzy/dsh-ego-browser.git

Fisfzy/dsh-ego-browser 加入你的 DSH 配置(web profile)即可启用。

READMEREADME

上一个 Prev dsh-code 下一个 Next dsh-popout-sidebar