mirror of
https://github.com/0xWheatyz/handler.git
synced 2026-08-30 04:36:24 +00:00
Add the pi harness: lightweight local-model agents with full gate parity
Model backend rows gain a harness column (claude | pi). A pi-harness row runs the agent through the pi coding agent instead of the claude binary — pi speaks the OpenAI Completions API natively, so a bare vLLM/llama.cpp/Ollama endpoint needs no LiteLLM/claude-code-router translation proxy, and the loop is far lighter for slow local token throughput. The Claude subscription and existing claude-harness backends are untouched. Parity comes from generated per-agent artifacts under ~/.handler-pi (outside the repo tree, so the clean-tree gate never trips): models.json + settings.json render the row as a pi provider pinned as the default model; a bundled bridge extension (pi_bridge.ts) adapts pi's events to the exact stdin/stdout contract of `python -m handler.hooks` — the Stop/completion gate re-prompts pi with blockers via a follow-up message, git push runs the test/build/approval gates and denies on failure, questions defer through an ask_operator tool into the normal answer/resume flow, and memory recall is injected at session start. The memory tools are registered natively (pi has no MCP), shelling to a new `python -m handler.mcpserver --call <tool>` seam that reuses the MCP server's implementations. Skills reuse the same ~/.claude/skills sync (pi implements the same SKILL.md standard) plus the repo's committed .claude/skills. Sessions are single JSONL files pre-assigned via --session, so cross-worker resume archives/materializes exactly like claude's; the prompt travels on stdin (pi has no -- separator). The supervisor normalizes pi's event stream on the fly: assistant message_end feeds last_output, the final agent_end becomes the run result. The whole chain was validated live against pi 0.84.1 with a stub OpenAI endpoint: memory injection, push-gate denial (including the protected- branch approval gate), stop-gate block loop, and ask_operator pause all ran end to end through the real hooks and DB. Also: harness selector in the dashboard Models form, pi baked into the control image (NodeSource 22 for pi's node >= 22.19 floor), PI_BIN override, docs in docs/local-models.md, fake_pi fixture + 12 tests (361 total green). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KdGv3u3DfTsP1S188KDhVH
This commit is contained in:
+81
-4
@@ -1,9 +1,25 @@
|
||||
# Local model backends (Qwen-Coder & friends)
|
||||
|
||||
Handler can run agents on locally-hosted models without changing anything about how an
|
||||
agent works: it is still the same `claude` binary with the same generated
|
||||
`settings.json`, hooks, skills, MCP connectors, plugins, and permission gates. The only
|
||||
thing a **model backend** changes is the environment of that one agent's process:
|
||||
Handler runs agents on locally-hosted models through a **model backend** row, and every
|
||||
backend picks one of two **harnesses**:
|
||||
|
||||
| Harness | Binary | Endpoint it needs | When to pick it |
|
||||
|---|---|---|---|
|
||||
| `claude` (default) | Claude Code | **Anthropic Messages API** incl. tool use — put LiteLLM / claude-code-router in front of a local server | You want the exact Claude Code toolchain (MCP connectors, plugins, permission modes) |
|
||||
| `pi` | [pi coding agent](https://github.com/badlogic/pi-mono) | **bare OpenAI-compatible** (`/v1/chat/completions`) — vLLM, llama.cpp, Ollama directly, no proxy | You want the lightest loop for slow local token throughput |
|
||||
|
||||
Both harnesses keep handler's contract intact: the same hooks (test/completion gate,
|
||||
push gate, approval gate), the same checkmark/log streaming, the same memory layer, the
|
||||
same skills, kill/resume, and schedules. The Claude subscription (no backend selected)
|
||||
always launches `claude` — pi is only ever used when you point an agent at a backend row
|
||||
that says so.
|
||||
|
||||
## The claude harness
|
||||
|
||||
Nothing about how an agent works changes: it is still the same `claude` binary with the
|
||||
same generated `settings.json`, hooks, skills, MCP connectors, plugins, and permission
|
||||
gates. The only thing the backend changes is the environment of that one agent's
|
||||
process:
|
||||
|
||||
| Variable | From |
|
||||
|---|---|
|
||||
@@ -88,6 +104,67 @@ Then register the backend in Handler: base URL `http://<host>:4000`, model
|
||||
[claude-code-router](https://github.com/musistudio/claude-code-router) in front as
|
||||
the Anthropic translator.
|
||||
|
||||
## The pi harness
|
||||
|
||||
Set **Harness: pi** on the backend row (or `"harness": "pi"` via `POST /claude/models`)
|
||||
and point `base_url` straight at the OpenAI-compatible endpoint — no LiteLLM, no
|
||||
claude-code-router:
|
||||
|
||||
```bash
|
||||
# vLLM with the Qwen tool parser is all you need:
|
||||
vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct \
|
||||
--enable-auto-tool-choice --tool-call-parser qwen3_coder --port 8000
|
||||
```
|
||||
|
||||
Backend row: base URL `http://<host>:8000/v1`, model
|
||||
`Qwen/Qwen3-Coder-30B-A3B-Instruct`, harness `pi`, API key optional (pi requires *some*
|
||||
credential, so handler injects a placeholder when none is stored). The same
|
||||
tool-parser/template caveats apply as ever — the model's tool calls must come back as
|
||||
structured `tool_calls`, so use vLLM's parser flags, `--jinja` on `llama-server`, or an
|
||||
Ollama model whose template declares `.Tools`.
|
||||
|
||||
### What the control layer generates
|
||||
|
||||
At every launch (spawn *and* resume) the backend row is materialized into a per-agent
|
||||
`PI_CODING_AGENT_DIR` under `~/.handler-pi/` — outside the repo tree, so the clean-tree
|
||||
completion gate never sees generated files:
|
||||
|
||||
- **`models.json` + `settings.json`** — the row as a pi provider (`openai-completions`
|
||||
by default) pinned as the default model. Row `env` keys `PI_PROVIDER_API`,
|
||||
`PI_CONTEXT_WINDOW`, and `PI_MAX_TOKENS` tune it; everything else in the env map
|
||||
passes through to the process.
|
||||
- **`extensions/handler-bridge.ts`** — the bundled bridge extension that adapts pi's
|
||||
events to the same `python -m handler.hooks` contract claude uses. The gates are the
|
||||
*same tested Python code*: the Stop/completion gate re-prompts pi with the blockers,
|
||||
`git push` runs the test + image-build + protected-branch approval gates and denies on
|
||||
failure, and questions go through an `ask_operator` tool that pauses the agent for the
|
||||
normal answer/resume flow. Memory recall is injected at session start, and the memory
|
||||
tools (`memory_search/get/save/link`) are registered directly — pi has no MCP by
|
||||
design, so the bridge shells to `python -m handler.mcpserver --call <tool>` instead.
|
||||
- **`APPEND_SYSTEM.md`** — the handler conventions (completion contract, ask_operator,
|
||||
memory usage) appended to pi's system prompt.
|
||||
|
||||
Skills work unchanged: pi implements the same SKILL.md standard as Claude Code, and the
|
||||
generated `settings.json` points pi's discovery at the web-managed `~/.claude/skills`
|
||||
sync plus the repo's committed `.claude/skills` (the forge role skills). pi also reads
|
||||
`AGENTS.md` / `CLAUDE.md` context files natively.
|
||||
|
||||
Sessions are single JSONL files pre-assigned by handler (`--session <path>`), so
|
||||
cross-worker resume works exactly like claude's: archived to the DB, materialized by
|
||||
whichever worker claims the resume, continued by launching pi again on the same file.
|
||||
|
||||
### What differs from the claude harness
|
||||
|
||||
- **MCP connectors and plugins don't apply** — pi has no MCP client or plugin system.
|
||||
The bundled memory server is bridged as native tools; other connectors are
|
||||
claude-harness-only for now.
|
||||
- **Permission modes don't apply** — pi has no permission system. The hard gates
|
||||
(PreToolUse-equivalent blocking, Stop gate) are enforced by the bridge, which is the
|
||||
layer handler actually relies on for claude too.
|
||||
- **`--max-budget-usd` doesn't apply** — local tokens are free; pi has no budget flag.
|
||||
- The `pi` binary must be on the worker's PATH (the control image bakes it in;
|
||||
`PI_BIN` overrides, same as `CLAUDE_BIN`).
|
||||
|
||||
## Expectations and tips for small models
|
||||
|
||||
- **Keep the harness light.** Handler's agents run tool-heavy (hooks, MCP connectors,
|
||||
|
||||
Reference in New Issue
Block a user