Unofficial community project. Not affiliated with, endorsed by, or sponsored by DeepSeek or Apple. "DeepSeek", "Apple", "macOS", and "Vision" are trademarks of their respective owners, used here only to describe compatibility.

Overview

A DeepSeek Harness (dsh) plugin that extracts text from screen captures, clipboard images, and image files. It calls Apple's Vision framework (the engine behind macOS Live Text) directly: no vision model, no API key, no network.

Comparable plugins (visionDS, dsh-vision-router, dsh-vision-sidecar) route the image to a vision model. LocalLens uses no model at all, which removes keys, quotas, and uploads. The cost is that it reads text rather than understanding images, and runs on macOS only.

Requirements

• macOS 13 (Ventura) or later. Below 13, recognitionLanguages is unavailable, the languages option is ignored, and Chinese accuracy degrades.
• Xcode Command Line Tools (xcode-select --install). A Swift helper is compiled with /usr/bin/swiftc on first call. Full Xcode is not required.
• DeepSeek Harness, Node.js 22+.

Install

dsh plugin --profile web add github:uknowmyface/locallens

The package ships its own cordis.patch.yml (declared via dsh.bundle.patch), so dsh plugin add installs and registers it in one step without writing to your profile's patch layer. Replace web with your profile name. Takes effect in a new session.

Usage

Ask in the chat box. No hotkey, no menu bar item, no background process.

Clipboard (most common): press Cmd+Ctrl+Shift+4 and drag a region, which copies the capture to the clipboard, then say "OCR the clipboard".

Image file: "read the text in ~/Desktop/bug.png"

Screen: "read what's on my screen". Captures the entire main display and requires Screen Recording permission.

Results land in the conversation context, so the next instruction can act on them:

OCR the clipboard and turn the table into markdown
read this error screenshot, then grep the project for where it's thrown

Parameters

Filled in by the agent. Listed for reference.

Returns { text, source, lineCount, warning? }.

Configuration

Override by id in your profile's own cordis.patch.yml rather than forking. The user layer is applied after every bundle layer:

- id: tool-screenshot-ocr
  config:
    languages: 'ja,en-US'

Privacy boundary

Stays local: image files and screen captures are read by Vision in a local process and never uploaded. The OCR computation involves no network call. The project contains no telemetry and no network code.

Does not stay local: the recognized text is returned as a tool result, enters the model context, and is sent to your configured model provider along the same path as anything you type.

The accurate claim is "images never reach a third-party vision service", not "no data leaves the machine". If a screenshot holds something you would not paste into the chat box, do not OCR it.

Security notes

OCR output is untrusted input. The text originates from arbitrary screen or image content and may be authored by a third party. dsh agents hold shell and filesystem access, so the risk is real. Results are wrapped in an <untrusted-ocr-text> marker and the tool description instructs the model to treat them as data rather than commands. This is a prompt-level mitigation, not a security boundary. Treat OCR of unknown images with the same caution as letting an agent browse an untrusted site.

source=screen captures the entire main display. No region or window scoping, including everything behind the chat window (password managers, 2FA codes). Use clipboard when the screen holds sensitive material. Region and window capture are planned.

source=file is not sandboxed. Any image file readable by the current user can be read.

Screen Recording permission is granted per application. Granting it to a terminal grants it to every process launched from that terminal. This is a macOS design constraint the plugin cannot narrow. clipboard and file require no permission.

Report vulnerabilities via SECURITY.md.

Scope

Vision also provides an image classifier of roughly 1300 categories, cat/dog detection, face and barcode detection. This plugin uses only VNRecognizeTextRequest.

The classifier returns coarse labels (dog, document, beach), not descriptions. Use a vision model when you need the picture understood. LocalLens does not attempt to substitute for one.

How it works

1. assets/ocr.swift is a Swift CLI built on VNRecognizeTextRequest at .accurate with language correction enabled.
2. It compiles to bin/ocr on first call (10 to 20 seconds) and is reused by mtime afterwards.
3. screencapture grabs the display, Vision extracts the text, and the result returns as plain text. The pipeline runs entirely on device.

The binary is cached alongside the source, so anyone able to write to the plugin directory can execute code in the dsh host process. This is the same trust boundary that applies to the rest of ~/.dsh.

Uninstall

dsh plugin --profile web remove @locallens/dsh-tool-screenshot-ocr

Removes the package and the bundle layer together, leaving nothing in the profile config.

Contributing

Issues and PRs welcome. These directions are wanted, in rough order of value:

Windows port. Windows 10+ ships Windows.Media.Ocr, structurally the same as Apple Vision: system-level OCR with no model, no upload, no key. The tool contract in lib/index.js carries over unchanged; only the engine layer needs replacing. This is the largest gap today.

Region and window capture. screencapture supports -R x,y,w,h and -l <windowid>. Wiring these into source would substantially narrow the data exposed by full-screen capture.

Linux port. No system-level equivalent exists. Tesseract is the obvious candidate, at the cost of an external dependency.

PDF input. Only bitmaps are accepted today; PDFs need rasterizing first.

Language testing. zh-Hans,en-US is the default. Accuracy in other languages has not been systematically verified.

lib/ocr-engine.js is the seam between the engine and the tool layer. Cross-platform ports start there.

Related projects

• ModLens and similar cloud-vision plugins provide genuine image understanding (scene description, layout reasoning, chart interpretation) that OCR cannot cover. Use them when you need the model to see rather than read.
• macOS has shipped Live Text since Monterey, which selects text from images directly in Preview and Quick Look. This plugin exposes the same capability to an agent programmatically.

Acknowledgements

• Built on Apple's Vision framework.
• Plugin and hot-reload mechanics come from cordis.
• Thanks to the DeepSeek Harness team for the plugin architecture.

The code was written by AI. DeepSeek produced the initial implementation; Claude performed the security review, corrected the package naming and install path, and rewrote the documentation. The author does not write code, and no human has reviewed it line by line. The project is roughly 800 lines with no runtime dependencies. Read it before trusting it.

License

MIT, see LICENSE.