HuanLinOTO/dsh-plugin-android-use

让模型通过 adb 操作安卓手机的 DSH 插件(截图、UI 树、点击、滑动、输入文本、按键、打开应用) | DSH plugin exposing adb-based tools that let the model operate an Android phone (screenshot, UI dump, tap, swipe, input text, press keys, open apps).

Project Overview项目介绍

dsh-plugin-android-use is a DSH-native plugin that lets a model drive an Android phone through adb. The repository ships a real bundle manifest and a cordis.patch.yml, declares dsh.bundle.patch in package.json, and is installed with dsh plugin --profile web add link:... for a local checkout or dsh plugin --profile web add @huanlin/dsh-plugin-android-use once published to npm. The prebuilt lib/ folder is committed, there is no prepare script, and the plugin therefore does not need allowBuilds. It contributes ten tools covering device discovery, screen perception, input actions, and app management, and registers skills/android-use/SKILL.md as a runtime skill via ctx.skills.register in apply() so no extra file copying is required.

The typical loop is screenshot-first: the model calls android_screenshot and receives an image with a coordinate grid burned in, then reads the printed grid label to issue android_tap({x, y}) or android_swipe directly on screenshot pixels. region plus magnify zoom in on small UI elements such as icons or buttons, android_ui_dump exposes accessibility nodes when the app supports them, and android_open_app, android_input_text, and android_press_key round out the interaction. Every frame is also written to captureDir (default <temp>/dsh-android-use/) and pruned past captureKeep (default 200), so a human can later replay what the model actually saw via the saved_path returned by each tool call.

Prerequisites and limits: adb must be on PATH (or pointed to via adbPath), and an Android device must already be attached through USB or adb connect <ip:port>. The input text mode is ASCII-only, so Unicode input requires installing ADBKeyboard IME on the device and switching inputTextMode to adbkeyboard. Images are downscaled so the longest side is at most imageMaxDimension (default 1280) and encoded as JPEG quality imageQuality (default 80), and coordinates must always be read from the grid, never back-computed from the original device resolution. Typecheck and the unit tests need junctions from node_modules/@deepseek-ai/* to a local DSH source tree, the license field is not declared in the README, and on first run you should tune adbPath, defaultSerial, and captureDir in the DSH settings page or directly inside cordis.patch.yml.

dsh-plugin-android-use 是一款 DSH 原生插件,让模型通过 adb 操控 Android 设备。仓库自带 bundle-manifest 与 cordis.patch.yml,安装方式为 dsh plugin --profile web add link:...(开发态)或 dsh plugin --profile web add @huanlin/dsh-plugin-android-use(npm 发布后),并声明 dsh.bundle.patch 作为 bundle 插入入口,预构建 lib/ 入库、无 prepare 脚本,无需 allowBuilds。插件为 DSH 工具栏注入 10 个工具,覆盖设备发现、屏幕感知、输入操作与应用管理,并将 skills/android-use/SKILL.md 作为 runtime skill 注册到 ctx.skills。

典型工作流:模型先调用 android_screenshot 拿到带坐标网格的截图像素图,再读取网格标签直接发起 android_tap 或 android_swipe;需要确认小目标时配合 region+magnify 放大,无法走无障碍树的应用可用 android_ui_dump 转 UI 节点;最后用 android_open_app 启动、android_input_text 填表、android_press_key 按键收尾,每一帧按默认 200 张上限留存到 captureDir 便于事后复盘。面向需要让 DSH 模型在真机或远程调试机上自主操作安卓 App 的开发者与测试人员。

依赖与限制:本机需已装 adb 且设备已通过 adb connect 或 USB 连入;输入模式 input 仅支持 ASCII,要输中文需安装 ADBKeyboard IME 并切到 adbkeyboard;图像管线默认长边压到 1280、JPEG q80,发图时坐标按网格标签读、不可按像素比例换算;cordis 与各 @deepseek-ai/* peer 依赖需 junction 到本地 DSH 源码树才能跑 pnpm typecheck,license 字段 README 未声明。首次使用请先在 DSH 设置页或 cordis.patch.yml 中按需调整 adbPath、defaultSerial、captureDir 等字段。

Pre-install check安装前体检Compatibility · Security兼容性 · 安全性 2 warnings2 项注意
  • No license declared - all rights reserved by default; ask the author before commercial use or redistribution未声明开源许可证 —— 默认「保留所有权利」,商用或再分发前先问作者
  • Only 3 stars - very few users, little community feedback星标只有 3,几乎没人在用,遇到问题缺少社区反馈
DSH walks through these 9 checksDSH 会逐条核对这 9 项

Compatibility兼容性

  • DSH, Node, OS and profile requirementsDSH 版本 / Node 版本 / 操作系统 / profile 是否满足要求
  • External dependencies and runtimes (Electron / Python / Docker, ...)外部依赖与运行时(Electron / Python / Docker 等)是否齐备
  • Conflicts with installed plugins: command names, skill / tool names, ports, duplicate MCP registration与已装插件是否冲突:命令名、skill / tool 重名、端口占用、重复 MCP 注册

Security安全性

  • Repo matches the facts registered here; archived or abandoned?仓库是否与页面登记一致,是否归档或长期停更
  • Safety of preinstall / install / postinstall and install.sh / setup.ps1preinstall / install / postinstall 与 install.sh、setup.ps1 是否安全
  • curl|bash, download-then-execute, obfuscation, unrelated domains → stop immediatelycurl|bash、下载即执行、混淆代码、无关域名 → 立刻停止
  • Typosquatting or unmaintained packages among the new dependencies新增依赖里有没有 typosquatting 或无人维护的包
  • Requested permissions vs. what the feature actually needs申请了哪些权限、是否超出功能所需(filesystem / network / shell / clipboard)
  • Any sudo / admin requirement, plus uninstall and rollback是否要求 sudo / 管理员权限,以及卸载与回滚方式

Anything uncertain must be marked unknown with a note on how to confirm it. This site's signal screen is a static snapshot, not a security audit.拿不准的必须标「未知」并说明要我怎么确认。本站的信号筛查是静态快照,不能替代安全审计。

Or use CLI install (for developers)或使用命令行安装(适合开发者)

CLI Install命令行安装

dsh plugin --profile web add @huanlin/dsh-plugin-android-use

把 HuanLinOTO/dsh-plugin-android-use 加入你的 DSH 配置(web profile)即可启用。

READMEREADME

dsh-plugin-android-use card

dsh-android-use

DSH 插件:让 AI 通过 adb 操作安卓手机。提供 10 个工具覆盖设备发现、屏幕感知、输入操作和应用管理,让模型能够自主操控 Android 设备。

安装

# 从本地 checkout 开发安装:
dsh plugin --profile web add link:D:\Projects\deepseek-harness\dsh-plugin-android-use

# 从 npm 安装(发布后):
dsh plugin --profile web add @huanlin/dsh-plugin-android-use

预构建策略:lib/ 入库,无 prepare 脚本,npm 安装开箱即用,无需 allowBuilds。

前提条件:本机已安装 adb 并在 PATH 上(或通过 adbPath 配置指定路径),已通过 adb connect <ip:port> 或 USB 连接 Android 设备。

配置

在 DSH GUI 设置页或 cordis.patch.yml 中配置:

字段 类型 默认值 说明
adbPath string 'adb' adb 可执行路径。默认从 PATH 查找。
defaultSerial string? - 默认设备 serial。省略则单设备自动选择;多设备时模型需传 serial 参数。
inputTextMode 'input' | 'adbkeyboard' 'input' 文本输入模式。input 仅支持 ASCII(用 adb shell input text);adbkeyboard 支持 Unicode(需设备安装 ADBKeyboard IME)。
captureDir string? <系统临时目录>/dsh-android-use 每张发给模型的帧都会在此留一份,便于事后复查。
captureKeep number 200 留存上限,超出按时间删除最旧的;0 表示不清理。
imageMaxDimension number 1280 发给模型的图片长边上限(像素):1080x2414 的截图会压到 573x1280。
imageFormat `'jpeg' 'webp' 'png'`
imageQuality number 80 jpeg / webp 质量。
showGrid boolean true 是否在发送的帧上绘制坐标网格。
provideSkill boolean true 是否把插件自带 skill 注册到 ctx.skills(profile 未挂 skills 服务时自动跳过)。

工具

设备发现

工具 参数 说明
android_list_devices 无 列出所有已连接的安卓设备(serial、state、product、model)。
android_device_info serial? 获取设备详细信息:型号、品牌、Android 版本、SDK、屏幕分辨率、密度、屏幕状态。

屏幕感知

工具 参数 说明
android_screenshot region?, magnify?, serial? 截屏 → 裁剪/放大 → 压缩(默认长边 ≤1280、JPEG)→ 烧入坐标网格 → 留存本地 → 发图。region 用截图像素;省略 magnify 时自动放大到上限(≤4×)。返回 saved_path、设备分辨率、压缩比、网格步长。
android_ui_dump serial? 转储无障碍树,节点 center 即截图像素坐标。App 未暴露无障碍节点时返回 empty: true 并明确建议改用截图(不再静默返回 0 节点 + screen 0x0)。

Showing the opening section of the README — the full document lives in the repository以上为 README 开头摘要,完整文档在仓库内 · View the full README on GitHub →在 GitHub 查看完整 README →

← 上一个 Prev dsh-ximalaya 下一个 Next dsh-task-control →