tianji-qingtian/dsh-model-router

dsh-model-router

Model Router & Cost Optimizer for DeepSeek Harness (dsh). Answers simple questions directly on the cheap model (zero prefix, no cache tax), degrades gracefully on transient provider failures, and shows live per-session token / cache-hit / cost figures right under the composer.

The harness is in developer preview and iterates quickly — expect compatibility-breaking changes.

中文说明见 README.zh.md。

Features

• Cheap-model judge routing — clearly heavy work (strong keywords / long payloads) goes straight to the main model with zero added latency. Everything else is decided by a zero-prefix flash judge call (SIMPLE / AGENTIC, one word, 64-token cap, thinking off). The judge also sees the last assistant reply, so context-dependent follow-ups (它 / 这个 / 继续 …) are never misrouted to a context-free answer.
• Ask before quick-answering — in auto mode, every SIMPLE hit asks the user through the built-in question UI: ⚡ 快速回答（flash） or 主模型回答. Choosing the main model (or dismissing the question) takes the normal flow; subagent sessions fall back automatically. The question text follows the request language.
• Direct quick-answer — when the user picks the quick option (or a quick answer is otherwise produced), the router rejects the step, runs a zero-prefix one-shot stream on the cheap model (no cache-miss tax), and writes the question + answer straight into the session log inside a step envelope — the user sees an ordinary Q&A exchange prefixed with a ⚡ 快速回答 / Quick answer · <model> marker (matched to the question language), the main model never runs for it, and no subagent sessions, relay cards, or toasts are created. The main session's model and prefix cache are never touched.
• Vision-aware routing — messages carrying image blocks are routed to the catalog's image-capable model when one exists (e.g. deepseek-v4-flash-vision-exp), including the quick-answer judge/stream; without an image-capable model they fall back to the main model instead of being answered blind on a text-only flash.
• Model picker respected — the router never overrides the model chosen in the composer picker or via selectModel, including relay/proxy providers (e.g. kabuai): agent/request passes the harness's requested config through unchanged, and quick answers always run on the session provider's own cheap model. (No drift healing — its original cause, a forged request/header, was removed at the root in v0.9.2.)
• Auto / off toggle — the dock panel and /router auto|off (plus the route_model tool) simply enable or disable quick-answering for the session. There is no per-request model flipping to configure.
• Automatic fallback — transient failures (RATE_LIMIT, SERVER, TIMEOUT, EMPTY_RESPONSE) degrade the turn to the cheap model and retry once; anything else delegates to the provider's own retry policy.
• Real usage metering — a session projection folds the durable log: real adapter token usage (input / output / cache read / cache write / reasoning), per-model breakdown, and estimated cost from a model-class price table. Projection-based, so the numbers are replay-safe and survive cold sessions.
• Composer dock panel (i18n zh/en) — Auto / 关闭 toggle, current model, miss/out/cache%/≈$ line, a QA×N quick-answer counter (with a brief inline highlight on each direct answer), and a per-model usage breakdown. Reactively driven by useProjection; the toggle reuses the built-in commands remote — no custom wire protocol. UI strings are localized through the harness locale service.