mirror of
https://github.com/0xWheatyz/handler.git
synced 2026-08-30 10:46:25 +00:00
docs: split MVP plan into docs/PLAN.md and rewrite README
Move the original plan-of-action out of README.md into docs/PLAN.md (verbatim, with Phase 1 checkboxes ticked and MVP-resolved open questions annotated). Replace the README with a proper project readme grounded in the implemented MVP: what it is, architecture, install/configure/run, curl walkthrough, control CLI, API reference, hooks, development, and layout. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W5ZuS5pV1NS6eKsRZHXonY
This commit is contained in:
Claude
@@ -1,242 +1,232 @@
|
|||||||
# handler
|
# handler
|
||||||
|
|
||||||
# Remote Control Wrapper — Plan of Action
|
A remote control wrapper for [Claude Code](https://claude.com/claude-code) agents.
|
||||||
|
Run many `claude` agents across many projects — each isolated, each leaving behind a
|
||||||
|
**checkmark** (its current state) and an entry in a **big log** (the complete history) —
|
||||||
|
all backed by a centralized database, driven entirely through an HTTP API.
|
||||||
|
|
||||||
**Status:** Living document — update phase checkboxes and status in place as work completes. Don't append new copies of this file; overwrite it, the way a checkmark gets overwritten.
|
Every agent process is a real `claude` binary invocation. There is no hard dependency on
|
||||||
**Last updated:** 2026-07-07 (revision 7: accepted `forge`'s maturity as a risk worth taking — pin to a released version, fork/vendor if it ever stalls)
|
any particular git host or network layer: you bring your own Claude Code login, your own
|
||||||
|
git remote, and your own network exposure.
|
||||||
|
|
||||||
|
> **Status: Phase 1 MVP.** The control layer, HTTP API, database, migrations, and
|
||||||
|
> verification hooks are implemented and tested (45 tests, SQLite). Live end-to-end agent
|
||||||
|
> spawning against a real `claude` binary + tmux is stubbed behind mockable seams and
|
||||||
|
> wired but not yet exercised against production binaries. See
|
||||||
|
> [`docs/PLAN.md`](docs/PLAN.md) for the full design and roadmap.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 1. What "done" looks like
|
## Why
|
||||||
|
|
||||||
Production release means:
|
One operator running several of their own projects wants to fan work out to background
|
||||||
|
Claude Code agents and keep a reliable, queryable picture of what each one is doing —
|
||||||
|
without babysitting a wall of tmux panes. `handler` gives every agent:
|
||||||
|
|
||||||
- **Runs anywhere.** No hard dependency on Gitea or Tailscale — git hosting and network exposure are both pluggable, not assumed.
|
- **A checkmark** — one small, always-current row: where it stopped, what's next, any
|
||||||
- A **web UI**, not just tmux/CLI, for spawning, monitoring, and interacting with agents.
|
open question for you. Overwritten on every checkpoint, like a file you keep saving.
|
||||||
- **All communication goes through an API.** The UI is a client of it; tmux/CLI become optional local conveniences, not the source of truth.
|
- **A big log** — the append-only history of everything every agent has ever done.
|
||||||
- Every agent leaves a **checkmark** behind: a small, current-state record of where it stopped, what's needed next, and any open questions for you.
|
- **A verification gate** — an agent never reaches `done` on its own say-so. A `Stop`
|
||||||
- Every checkmark links to an entry in a **big log**: the append-only, complete history of everything every agent has ever done.
|
hook runs the project's own test task and blocks the turn on failure, so `done` in the
|
||||||
- State lives in a **centralized database** that the CLI wrapper (backend) writes to and the API reads from — not scattered flat files or a git-hosting-specific store. Control layer and API containers hold nothing persistent themselves; they can restart, redeploy, or scale out without losing data because all of it lives in the database.
|
database means *a test run passed*.
|
||||||
- **Multiple projects run concurrently, each isolated from the others by default.** One control layer can host any number of projects, each with its own agents, working directory, and history — with an explicit, opt-in mechanism for the edge cases where something genuinely needs to cross that boundary.
|
- **A push gate** — a `git push` doesn't leave until tests pass *and* a throwaway image
|
||||||
|
build succeeds locally, so a push already known to fail CI never goes out.
|
||||||
|
- **Isolation** — each project has its own working directory, agents, history, and
|
||||||
|
credentials; nothing crosses the boundary unless you explicitly share it.
|
||||||
|
|
||||||
## 2. Non-negotiable constraints
|
## Architecture
|
||||||
|
|
||||||
- Every agent process is a real `claude` binary invocation. No OAuth handling, no protocol reimplementation.
|
Three components over one database. The database is the only thing that holds state, so
|
||||||
- **No hard dependency on Tailscale.** The API authenticates itself with a bearer token; users choose their own network exposure — Tailscale, a VPN, a reverse proxy, or plain localhost. The wrapper doesn't assume any of them.
|
the control layer and API are disposable compute that can restart or scale out freely.
|
||||||
- **No hard dependency on Gitea.** Git operations use plain `git` by default. Forge-specific niceties (PRs, issues, CI status) go through `forge` (git-pkgs/forge) as an optional, pluggable layer — one CLI that works the same against GitHub, GitLab, Gitea/Forgejo, or Bitbucket, never a requirement just to run the tool.
|
|
||||||
- **The database never stores raw credentials.** `projects.credential_ref` is a pointer (an env var name, a file path, a command to run) — never a token. The control layer resolves it to an actual secret only at spawn time, injected into that container's environment for that run.
|
|
||||||
- Design decisions above a stakes threshold get surfaced via `plan` mode + `AskUserQuestion`, not silently guessed.
|
|
||||||
- **Projects are isolated by default.** An agent only sees its own project's working directory, checkmarks, and log — nothing crosses that boundary unless something is explicitly marked shared. This is one operator running many of their own projects, not a multi-tenant service for other people — don't let the isolation model drift into looking like the latter.
|
|
||||||
- **Agents don't get to self-report "done."** A verification gate actually runs the project's own test task and blocks completion on failure — "done" in the database means a test run passed, not that the agent stopped talking.
|
|
||||||
- **A `git push` doesn't leave until a local build check passes too.** Same hard-block pattern as the test gate, and it runs after tests, not instead of them — cheap checks run before the more expensive one that would obviously fail anyway.
|
|
||||||
- If this ships open-source: no embedded credentials, no implied Anthropic affiliation, README states plainly that each user brings their own Claude Code login, their own git remote, and their own network layer.
|
|
||||||
|
|
||||||
## 3. Control layer + API (build this first — this is the MVP)
|
```
|
||||||
|
writes reads (+ answer backfill)
|
||||||
### 3.1 Data model — Postgres primary, SQLite fallback
|
┌──────────────────┐ ┌──────────────┐ ┌──────────────────┐
|
||||||
|
│ control layer │───────▶│ database │◀───────│ HTTP API │
|
||||||
Two supported backends behind one data-access layer, not two separate code paths:
|
│ (CLI + hooks) │ │ PG / SQLite │ │ (FastAPI) │
|
||||||
|
└──────────────────┘ └──────────────┘ └──────────────────┘
|
||||||
- **Postgres (default for real deployments).** A live, centralized server is what actually makes "stateless containers" true — the control layer and API are just compute that can restart, redeploy, or scale to N replicas because none of them hold state locally. This is the deployment target for anyone running more than a single node, or wanting one datastore behind multiple agent hosts.
|
│ ▲ ▲
|
||||||
- **SQLite (minimal-infra fallback).** A single file, zero services to stand up — for a single-node/homelab-scale run where standing up Postgres is overkill. Explicitly a fallback, not the default: it doesn't give you the centralization the Postgres path does, and single-writer semantics limit it to one control-layer instance at a time.
|
│ spawns │ Stop / PreToolUse / Notification hooks │ curl, UI, any client
|
||||||
|
▼ │ write checkmark + log rows │ (bearer token)
|
||||||
The schema is the same shape on both; only types differ slightly (Postgres gets proper `SERIAL`/`TIMESTAMPTZ`/`JSONB`, SQLite uses its looser dynamic typing). Pick one query layer/ORM that supports both dialects rather than hand-maintaining two schemas — see open questions.
|
tmux + claude binary (one working dir / worktree per agent)
|
||||||
|
|
||||||
```sql
|
|
||||||
-- Postgres
|
|
||||||
CREATE TABLE projects (
|
|
||||||
id TEXT PRIMARY KEY, -- slug, e.g. "leeworks-api"
|
|
||||||
root_dir TEXT NOT NULL,
|
|
||||||
git_remote TEXT,
|
|
||||||
credential_ref TEXT, -- pointer to a secret, e.g. "env:LEEWORKS_TOKEN" — never the token itself, see 3.7
|
|
||||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE TABLE agents (
|
|
||||||
id BIGSERIAL PRIMARY KEY,
|
|
||||||
project_id TEXT NOT NULL REFERENCES projects(id),
|
|
||||||
name TEXT NOT NULL, -- unique within a project, not globally
|
|
||||||
working_dir TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL, -- working | paused_for_input | blocked | done
|
|
||||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
|
||||||
UNIQUE (project_id, name)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE TABLE checkmarks (
|
|
||||||
agent_id BIGINT PRIMARY KEY REFERENCES agents(id),
|
|
||||||
checkpoint_at TIMESTAMPTZ NOT NULL,
|
|
||||||
status TEXT NOT NULL,
|
|
||||||
where_it_stopped TEXT,
|
|
||||||
next_steps JSONB,
|
|
||||||
open_question TEXT,
|
|
||||||
log_entry_id BIGINT REFERENCES log_entries(id),
|
|
||||||
tests_status TEXT NOT NULL DEFAULT 'unknown', -- pass | fail | unknown — see 3.5
|
|
||||||
tested_at TIMESTAMPTZ,
|
|
||||||
build_status TEXT NOT NULL DEFAULT 'unknown', -- pass | fail | unknown — see 3.6
|
|
||||||
built_at TIMESTAMPTZ
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE TABLE log_entries (
|
|
||||||
id BIGSERIAL PRIMARY KEY,
|
|
||||||
agent_id BIGINT NOT NULL REFERENCES agents(id),
|
|
||||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
|
||||||
session_id TEXT,
|
|
||||||
status TEXT NOT NULL,
|
|
||||||
summary TEXT,
|
|
||||||
decisions TEXT,
|
|
||||||
question TEXT,
|
|
||||||
answer TEXT, -- filled in on resume; only field ever touched post-insert
|
|
||||||
visibility TEXT NOT NULL DEFAULT 'project', -- project | global — see 3.4
|
|
||||||
push_sha TEXT, -- set if this checkpoint pushed; null if it didn't
|
|
||||||
ci_status TEXT NOT NULL DEFAULT 'not_applicable', -- not_applicable | pending | pass | fail — see 3.6
|
|
||||||
ci_checked_at TIMESTAMPTZ
|
|
||||||
);
|
|
||||||
```
|
```
|
||||||
|
|
||||||
```sql
|
- **Control layer** (`handler.control`) — the only writer. Spawns/lists/attaches/kills
|
||||||
-- SQLite (minimal-infra fallback) — same shape, simpler types
|
agents as `tmux` sessions running the `claude` binary, one working directory or git
|
||||||
CREATE TABLE projects (
|
worktree per agent, namespaced `project__agent`. Stateless; every write goes straight
|
||||||
id TEXT PRIMARY KEY,
|
to the database.
|
||||||
root_dir TEXT NOT NULL,
|
- **Hooks** (`handler.hooks`) — run inside each agent via a generated `settings.json`.
|
||||||
git_remote TEXT,
|
They write the checkpoint/log rows and enforce the test and push gates.
|
||||||
credential_ref TEXT,
|
- **API** (`handler.api`) — a thin, read-mostly HTTP layer over the same database (the
|
||||||
created_at TEXT NOT NULL
|
one write it does is backfilling an operator's answer). Bearer-token auth on every
|
||||||
);
|
route; every agent route is nested under `/projects/:project/` so nothing leaks across
|
||||||
-- agents / checkmarks / log_entries: identical columns, JSONB becomes TEXT,
|
a project boundary.
|
||||||
-- BIGSERIAL becomes INTEGER PRIMARY KEY AUTOINCREMENT, TIMESTAMPTZ becomes TEXT.
|
|
||||||
|
### One schema, two backends
|
||||||
|
|
||||||
|
The data model is defined once (SQLAlchemy Core) and renders correctly on both:
|
||||||
|
|
||||||
|
- **Postgres** (default for real deployments) — `BIGSERIAL`, `TIMESTAMPTZ`, `JSONB`.
|
||||||
|
A live central server is what makes the stateless-container story true.
|
||||||
|
- **SQLite** (minimal-infra fallback) — a single file, zero services. Same schema shape,
|
||||||
|
simpler types (`INTEGER PRIMARY KEY`, TEXT, JSON).
|
||||||
|
|
||||||
|
Portable column types bridge the two, and the checkmark upsert uses native
|
||||||
|
`INSERT … ON CONFLICT DO UPDATE` on both dialects. Migrations are Alembic, dual-dialect.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- Python 3.11+
|
||||||
|
- `git` and `tmux` (for live spawning)
|
||||||
|
- A `claude` binary, authenticated (for live spawning)
|
||||||
|
- `mise` in each managed project, with a `.mise.toml` defining at least a `test` task
|
||||||
|
- Postgres (default) — or nothing but a file path for the SQLite fallback
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m venv .venv && source .venv/bin/activate
|
||||||
|
pip install -e ".[dev]"
|
||||||
```
|
```
|
||||||
|
|
||||||
`checkmarks` is a literal upsert (`INSERT ... ON CONFLICT DO UPDATE` / `INSERT OR REPLACE`) keyed by `agent_id` — the "small file that gets overwritten," just as a row instead of a file. `log_entries` is insert-only except for the one `answer` backfill on resume, which is the DB equivalent of "linked directly to this checkmark." Agent identity is now `(project_id, name)`, not a bare name — two projects can each have an agent called `api` without colliding.
|
## Configure
|
||||||
|
|
||||||
### 3.2 Backend (CLI wrapper) — the only thing that writes
|
Configuration is entirely environment-driven (see [`.env.example`](.env.example)):
|
||||||
|
|
||||||
- Spawns/lists/attaches/kills agents within a project (tmux + `claude` binary, one working directory or git worktree per agent, nested under that project's root)
|
| Variable | Purpose | Default |
|
||||||
- Owns every write to the database: creates the agent row, upserts the checkmark, inserts log entries
|
|---|---|---|
|
||||||
- Hooked via `Stop`/`SessionEnd` (checkpoint), `PreToolUse` (defer `AskUserQuestion`), `Notification` (fires a generic webhook — the user points that at ntfy, Pushover, Slack, email, whatever; the wrapper doesn't pick for them)
|
| `DATABASE_URL` | `sqlite:////abs/path.db` or `postgresql+psycopg://…` | `sqlite:///./handler.db` |
|
||||||
|
| `AUTH_TOKEN` | Global bearer token gating every API route | *(required for the API)* |
|
||||||
|
| `SHARED_CONTEXT_WRITE_TOKEN` | Higher-trust token gating `PUT /shared/context/:key` | falls back to `AUTH_TOKEN` |
|
||||||
|
| `WEBHOOK_URL` | Generic target for the `Notification` hook (ntfy, Slack, …) | unset → no-op |
|
||||||
|
| `PROJECTS_ROOT` | Base dir for per-project roots / worktrees | `./projects` |
|
||||||
|
| `CLAUDE_BIN` / `MISE_BIN` / `TMUX_BIN` | Binary overrides | `claude` / `mise` / `tmux` |
|
||||||
|
|
||||||
### 3.3 API — the only thing that reads (plus writes the answer back)
|
## Run
|
||||||
|
|
||||||
- Thin HTTP layer over the same database (Postgres in centralized deployments, SQLite for minimal-infra): `GET /projects`, `POST /projects`, `GET /projects/:project/agents`, `POST /projects/:project/agents`, `GET /projects/:project/agents/:name/checkmark`, `GET /projects/:project/agents/:name/log`, `POST /projects/:project/agents/:name/answer`, `POST /projects/:project/agents/:name/resume`
|
Apply migrations, then start the API:
|
||||||
- Auth: a generated bearer token checked on every request. No assumption about what network the request arrives over — Tailscale, a VPN, or a bare reverse proxy are all just transport underneath it.
|
|
||||||
- The UI (Phase 3) and any future integration are just clients of this — same contract as `curl`.
|
|
||||||
|
|
||||||
### 3.4 Multi-project isolation and cross-project sharing
|
```bash
|
||||||
|
export DATABASE_URL="sqlite:///$PWD/handler.db"
|
||||||
|
export AUTH_TOKEN="$(openssl rand -hex 32)"
|
||||||
|
|
||||||
Every agent belongs to exactly one project. Isolation is the default; sharing is a deliberate, visible act, never an accident.
|
alembic upgrade head
|
||||||
|
uvicorn handler.api.app:app --host 0.0.0.0 --port 8000
|
||||||
**Isolation:**
|
|
||||||
- Filesystem: each project gets its own root directory; agents in that project work only within it (subdirectories or git worktrees underneath), never reaching into another project's tree.
|
|
||||||
- Naming: tmux sessions follow `project__agent`, matching the `(project_id, name)` uniqueness in the database — no cross-project collisions, and it's visually obvious which project a session belongs to from `tmux ls`.
|
|
||||||
- Config: git remote, credential reference, and any other per-project settings, live on the `projects` row — not global config — so projects can point at entirely different repos, hosts, or forges independently.
|
|
||||||
- API: every agent route is nested under `/projects/:project/...`. There's no endpoint that returns another project's data by accident.
|
|
||||||
|
|
||||||
**The explicit sharing mechanism, for the edge cases:**
|
|
||||||
- `log_entries.visibility` — defaults to `project`, can be set to `global` when an agent (or you) decides a specific checkpoint genuinely matters beyond its own project. A `GET /shared/log` endpoint surfaces only the entries marked `global`, across all projects — a deliberate opt-in feed, not a merged view of everything.
|
|
||||||
- `shared_context` table — a small key-value store for standing facts multiple projects need to reference (a shared staging URL, a schema version, a convention decision), independent of any single checkpoint:
|
|
||||||
```sql
|
|
||||||
CREATE TABLE shared_context (
|
|
||||||
key TEXT PRIMARY KEY,
|
|
||||||
value TEXT NOT NULL,
|
|
||||||
set_by_agent_id BIGINT REFERENCES agents(id),
|
|
||||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
|
||||||
);
|
|
||||||
```
|
|
||||||
Any agent can read it (`GET /shared/context`); writing (`PUT /shared/context/:key`) is the one place worth gating behind an explicit flag or higher-trust token, since it's the one table every project implicitly trusts.
|
|
||||||
|
|
||||||
Nothing here allows one project to read another project's checkmarks or private log entries directly — the only paths across the boundary are the two explicit ones above.
|
|
||||||
|
|
||||||
### 3.5 Toolchains and test verification
|
|
||||||
|
|
||||||
Environment reliability and result reliability are two different problems — solve both, don't conflate them.
|
|
||||||
|
|
||||||
**Environment: mise, not nix.** A single generic base image (mise + the `claude` binary + `git` + `gh`) with no baked-in language runtimes — one image for every project regardless of stack. Each project carries its own `.mise.toml`, pinning exact tool versions (Rust, Python, Node, whatever) and a lockfile committed to the repo for strict, reproducible installs. On container start, `mise install --non-interactive` reads that lockfile and materializes the project's exact toolchain — the container doesn't need to know or care what language it's about to run. Nix would get similar pinning with more ceremony and a heavier runtime; since Docker already provides OS-level isolation, mise's language-level pinning is the right amount of tool for this.
|
|
||||||
|
|
||||||
Every project defines a canonical task, regardless of stack:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[tasks.test]
|
|
||||||
description = "Run the test suite"
|
|
||||||
run = "pytest" # or `cargo test`, `npm test`, whatever the project actually uses
|
|
||||||
|
|
||||||
[tasks.verify]
|
|
||||||
depends = ["lint", "test"]
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Result: a Stop hook, not self-reporting.** After each turn, a `Stop` hook runs `mise run test` (or `verify`) in the agent's working directory and checks the exit code. Nonzero → `decision: "block"`, with the failure output fed back to the agent, so the turn cannot end on a broken test suite. This is what "correct" actually rests on: it's the same command for every project because mise's task abstraction is the uniform interface, and it's enforced by the harness, not reported by the model.
|
The API is just a client contract — everything below works with plain `curl`:
|
||||||
|
|
||||||
The result feeds straight into the schema already in place: the hook sets `checkmarks.tests_status` and `tested_at` on every checkpoint, so `status = 'done'` in the database is only ever true alongside a passing test run — not a claim taken on faith.
|
```bash
|
||||||
|
TOKEN="Authorization: Bearer $AUTH_TOKEN"
|
||||||
|
BASE="http://127.0.0.1:8000"
|
||||||
|
|
||||||
### 3.6 Push gate and CI follow-through
|
# Register a project and an agent
|
||||||
|
curl -s -X POST $BASE/projects -H "$TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"id":"leeworks-api","root_dir":"/srv/projects/leeworks"}'
|
||||||
|
|
||||||
Tests catch broken code; they don't catch a broken container. A Dockerfile with a stale `COPY` path passes `pytest` fine and then fails to build — worth catching locally, before it costs a CI run.
|
curl -s -X POST $BASE/projects/leeworks-api/agents -H "$TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"name":"api","working_dir":"/srv/projects/leeworks/api"}'
|
||||||
|
|
||||||
**Local build check — cheap, throwaway, no registry involved.** Since CI/CD (on its own, beefier runner) owns the real build-and-push, the local check only needs to prove the Dockerfile is sound: a `kaniko`/`buildah` build with no push target, output discarded. No Docker socket, no `--privileged`, nothing the agent's own sandbox needs elevated trust for — it's the same category of tool CI systems adopted specifically to avoid handing out real Docker-in-Docker access. This is `mise run build-image` alongside the existing `test` task, same interface, same project regardless of stack.
|
# Read an agent's checkmark and log
|
||||||
|
curl -s $BASE/projects/leeworks-api/agents/api/checkmark -H "$TOKEN"
|
||||||
|
curl -s $BASE/projects/leeworks-api/agents/api/log -H "$TOKEN"
|
||||||
|
|
||||||
**Enforcement — same mechanism as tests, one step later.** A `PreToolUse` hook matching `Bash(git push*)` runs the verification chain (tests, then the throwaway build — cheap first, expensive second) and returns `permissionDecision: "deny"` on the first failure, with the reason surfaced to the agent. A push that's already known to fail CI doesn't leave. `checkmarks.build_status`/`built_at` record the result the same way `tests_status` does.
|
# Answer a paused question, then resume the agent
|
||||||
|
curl -s -X POST $BASE/projects/leeworks-api/agents/api/answer -H "$TOKEN" \
|
||||||
|
-H 'Content-Type: application/json' -d '{"answer":"use Postgres"}'
|
||||||
|
curl -s -X POST $BASE/projects/leeworks-api/agents/api/resume -H "$TOKEN" \
|
||||||
|
-H 'Content-Type: application/json' -d '{}'
|
||||||
|
```
|
||||||
|
|
||||||
**CI is still the authoritative gate — the wrapper just closes the loop on it.** Once a push clears the local checks and actually goes out, whatever CI system the host runs (GitHub Actions, Gitea/Forgejo Actions, GitLab CI) does the real build-and-push on its own runner, same as today. The wrapper records `push_sha` and sets `ci_status = 'pending'` on that log entry, then a background poller in the control layer — not a hook, since this can take minutes — checks the run tied to that commit with `forge ci list` / `forge ci log <id>`. Same two commands regardless of which host the project is on; `forge` detects the forge type from the remote, so this isn't per-host integration work, just one poller calling one interface. Once the run resolves, the poller backfills `ci_status` and `ci_checked_at` on that same log entry — the DB equivalent of "recording why it did or didn't ultimately land," without ever exposing the wrapper to inbound webhook traffic.
|
## Control CLI
|
||||||
|
|
||||||
### 3.7 Credentials: forge and git
|
The `handler` command manages agent processes (the write side):
|
||||||
|
|
||||||
Two things need to authenticate per project — the `forge` CLI (for PRs, issues, CI status) and `git` itself (for push/pull) — and the design goal is one secret servicing both, not two to manage.
|
```bash
|
||||||
|
handler spawn --project leeworks-api --name api --worktree feature/auth --task "add login"
|
||||||
|
handler list [--project leeworks-api]
|
||||||
|
handler attach --project leeworks-api --name api
|
||||||
|
handler kill --project leeworks-api --name api
|
||||||
|
```
|
||||||
|
|
||||||
**Resolution, not storage.** `projects.credential_ref` is a pointer (`env:VAR_NAME`, `file:/path`, `cmd:some command`), never the token itself — mirrors `forge`'s own `--token-cmd` pattern, just one level up. At spawn time, the control layer resolves the reference to an actual value and injects it into that agent's container environment only — nothing persisted in Postgres/SQLite, nothing baked into an image.
|
`spawn` refuses any project whose working directory has no `.mise.toml` with a
|
||||||
|
`[tasks.test]` task — the verification gate is a hard requirement, not a convention. It
|
||||||
|
resolves the working directory (a subdirectory or a fresh git worktree, always under the
|
||||||
|
project root), writes a per-agent `.claude/settings.json` wiring the hooks, and launches a
|
||||||
|
`tmux` session with the agent's identity and `DATABASE_URL` injected into its environment.
|
||||||
|
|
||||||
**One token, two consumers, where the host allows it.** `forge` reads the resolved token from the environment directly (`GITHUB_TOKEN`, `GITEA_TOKEN`, `FORGE_TOKEN`, per host) with zero extra config once it's injected. For git's own HTTPS auth, the wrapper writes a small credential-helper at container start that hands back that same value — most self-hosted forges (Gitea/Forgejo, GitLab) accept a PAT as the HTTP password, so one token covers both `forge pr create` and `git push` for a given project.
|
## API reference
|
||||||
|
|
||||||
**SSH deploy keys as the per-project alternative.** Some operators won't want a single token holding both git-write and forge-API scope. Where that matters, a project can use an SSH deploy key for git transport and a separately-scoped, more limited token for `forge`'s PR/issue/CI operations — configured per project, not mandated globally, since this is a real preference split among self-hosters rather than a settled question.
|
All routes require `Authorization: Bearer <AUTH_TOKEN>`. `GET /health` is unauthenticated.
|
||||||
|
|
||||||
## 4. Phased roadmap
|
| Method & path | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `GET /projects` · `POST /projects` | List / register projects |
|
||||||
|
| `GET /projects/:p/agents` · `POST …` | List / register agents (project-scoped) |
|
||||||
|
| `GET /projects/:p/agents/:name/checkmark` | The agent's current-state checkmark |
|
||||||
|
| `GET /projects/:p/agents/:name/log` | The agent's log history (paginated) |
|
||||||
|
| `POST /projects/:p/agents/:name/answer` | Backfill the operator's answer to an open question |
|
||||||
|
| `POST /projects/:p/agents/:name/resume` | Feed the answer back via `claude --resume` |
|
||||||
|
| `GET /shared/log` | Cross-project feed of entries explicitly marked `global` |
|
||||||
|
| `GET /shared/context` · `GET /shared/context/:key` | Read shared key/value facts |
|
||||||
|
| `PUT /shared/context/:key` | Write a shared fact — requires the shared-write token |
|
||||||
|
|
||||||
### Phase 0 — Prerequisites
|
## Hooks
|
||||||
- [ ] `claude` binary installed and authenticated (per user)
|
|
||||||
- [ ] `git` installed; `forge` (git-pkgs/forge) optional
|
|
||||||
- [ ] `mise` installed in the base agent image; each project supplies its own `.mise.toml` with, at minimum, a `test` task
|
|
||||||
- [ ] A Postgres connection string (default target) — or nothing at all if running the SQLite fallback for a single-node/minimal-infra setup
|
|
||||||
|
|
||||||
### Phase 1 — Control layer + API (the MVP)
|
Wired into each agent as `python -m handler.hooks <event>`:
|
||||||
- [ ] Database schema + migrations for both backends, Postgres default / SQLite fallback, including `projects` and project-scoped agents (section 3.1)
|
|
||||||
- [ ] Control script: spawn/list/attach/kill, tmux + `claude` binary, one working dir/worktree per agent, namespaced by project — itself stateless, all state written straight to the database
|
|
||||||
- [ ] Hooks writing checkmark/log rows (`Stop`/`SessionEnd`, `PreToolUse` defer, `Notification` → generic webhook), all scoped to the owning project
|
|
||||||
- [ ] HTTP API over the same database, bearer-token auth, all agent routes nested under `/projects/:project/`
|
|
||||||
- [ ] Resume flow: API endpoint takes an answer, feeds it back via `claude --resume`
|
|
||||||
- [ ] Shared-context and shared-log endpoints for the explicit cross-project edge case (section 3.4)
|
|
||||||
- [ ] `Stop` hook verification gate: `mise install` on checkout, `mise run test` on every checkpoint, block completion on failure (section 3.5)
|
|
||||||
- [ ] `PreToolUse` hook on `git push*`: throwaway `kaniko`/`buildah` build (no registry) after tests pass, hard-deny the push on either failure (section 3.6)
|
|
||||||
|
|
||||||
**Definition of done:** run several projects side by side, each with its own agents, working directories, toolchain, and history, entirely through `curl` + a token, against either a live Postgres instance or the SQLite fallback — no Gitea, no Tailscale, no UI required, no state stored anywhere the control layer or API containers themselves live, nothing crosses a project boundary unless explicitly shared, no agent reaches `done` without a passing test run to show for it, and no push leaves that a local build already proved would fail.
|
- **`Stop` / `SessionEnd`** — checkpoint. On `Stop`, run `mise run test`; on failure,
|
||||||
|
return `decision: "block"` with the output so the turn cannot end on red. Records
|
||||||
|
`tests_status` / `tested_at`; `status = 'done'` only ever accompanies a pass.
|
||||||
|
- **`PreToolUse`** — two jobs. An `AskUserQuestion` is *deferred*: the question is
|
||||||
|
persisted, the checkmark set to `paused_for_input`, and the tool call denied so control
|
||||||
|
hands off to the async answer/resume flow. A `Bash` command running `git push` triggers
|
||||||
|
the push gate — tests first, then a throwaway image build (`mise run build-image`) — and
|
||||||
|
is denied on the first failure.
|
||||||
|
- **`Notification`** — POSTs a small JSON payload to `WEBHOOK_URL` (no-op when unset).
|
||||||
|
Never blocks the agent on delivery failure.
|
||||||
|
|
||||||
### Phase 2 — Forge integration
|
Hook identity travels via environment variables injected at spawn (`HANDLER_AGENT_ID`,
|
||||||
- [ ] Repo-scoped actions via `forge`: branch creation, PR open, issue linking — one interface across GitHub/GitLab/Gitea/Forgejo/Bitbucket instead of a per-host integration
|
`HANDLER_PROJECT_ID`, `HANDLER_AGENT_NAME`, `DATABASE_URL`), since hook stdin doesn't
|
||||||
- [ ] Pin `forge` to a specific released version, not `@latest` — accepted risk given the project's youth, mitigated by not floating on a moving target
|
carry it; the wiring itself lives in the generated `settings.json`.
|
||||||
- [ ] Credential resolution (section 3.7): `credential_ref` → injected env var at spawn, shared by `forge` and git's credential helper
|
|
||||||
- [ ] Background poller reading back CI run results via `forge ci list` / `forge ci log`, backfilling `ci_status`/`ci_checked_at` on the log entry that recorded the push (section 3.6)
|
|
||||||
|
|
||||||
### Phase 3 — Production UI
|
## Development
|
||||||
- [ ] Web frontend, API-backed only (same contract as `curl`)
|
|
||||||
- [ ] Project switcher, agent list per project, live checkmark view, log history, "answer this question" form, plus a view for the shared/global feed
|
|
||||||
|
|
||||||
**Definition of done:** open a URL, see every agent's state, answer a paused question, no terminal required.
|
```bash
|
||||||
|
pytest # 45 tests, entirely on SQLite — no live claude/tmux/mise needed
|
||||||
|
ruff check . # lint
|
||||||
|
# or, via the project's own mise tasks:
|
||||||
|
mise run verify # lint + test
|
||||||
|
```
|
||||||
|
|
||||||
### Phase 4 — Observability (moved back, now optional)
|
The suite drives every API route through FastAPI's `TestClient`, exercises all four hook
|
||||||
- [ ] Prometheus metrics endpoint on the API (agent counts, pending questions, checkpoint rate)
|
types, and runs a real `alembic upgrade head` per test so the migration path itself is
|
||||||
- [ ] Grafana/Loki wiring documented as an optional add-on for self-hosters who already run that stack — not a dependency for anyone else
|
covered. Three seams — `control.tmux`, `hooks.verify`, and `control.spawn.resume` — are
|
||||||
|
the mock points that stand in for live `claude`/`tmux`/`mise`, and the drop-in points for
|
||||||
|
wiring them up for real.
|
||||||
|
|
||||||
### Phase 5 — Open-source release
|
## Project layout
|
||||||
- [ ] Strip any remaining homelab-specific defaults into config
|
|
||||||
- [ ] README: bring-your-own Claude Code login, bring-your-own network, bring-your-own git host, bring-your-own credential source (env var, file, or command)
|
|
||||||
- [ ] Publish
|
|
||||||
|
|
||||||
## 5. Open questions
|
```
|
||||||
|
src/handler/
|
||||||
|
config.py # env-driven settings, shared by every entrypoint
|
||||||
|
db/ # SQLAlchemy Core schema, engine, portable types, upsert, DAL
|
||||||
|
api/ # FastAPI app, auth deps, pydantic schemas, routes
|
||||||
|
control/ # CLI, tmux/worktree/settings-gen seams, spawn orchestration
|
||||||
|
hooks/ # Stop/SessionEnd, PreToolUse gate, Notification, verify seam
|
||||||
|
migrations/ # Alembic env + versions
|
||||||
|
tests/ # DB, API, hook, and control tests (SQLite)
|
||||||
|
docs/PLAN.md # full design + phased roadmap (the original plan of action)
|
||||||
|
```
|
||||||
|
|
||||||
- Project name — still Cutout / Handler / Dead Drop / Backchannel / Umbra.
|
## Roadmap
|
||||||
- API implementation language/framework — wants a good story for *both* Postgres and SQLite (a query layer or ORM that speaks both dialects) plus minimal runtime deps, given "portable" is still a hard requirement.
|
|
||||||
- Default webhook target for `Notification` — ship a zero-config adapter (ntfy needs no account) or leave it fully bring-your-own from day one?
|
Phase 1 (this MVP) is the control layer + API. Still ahead: **Phase 2** forge integration
|
||||||
- Reference deployment for Postgres — user-supplied external instance only, or does the repo also ship a docker-compose/StatefulSet example for people with nowhere else to put it? (Either way, Postgres is the one stateful component in the system — everything else stays a disposable container.)
|
and credential resolution (PRs/issues/CI status via `forge`, one interface across GitHub /
|
||||||
- Who can write to `shared_context` — any agent by default, or does it need a higher-trust token than the per-project routes get?
|
GitLab / Gitea / Forgejo / Bitbucket), **Phase 3** a web UI, **Phase 4** optional
|
||||||
- Per-project API tokens vs one global token — least-privilege argues for scoping tokens to a project, but this is single-operator, not multi-tenant, so a global token may just be simpler and sufficient.
|
observability, and **Phase 5** open-source release. Details and design rationale live in
|
||||||
- Pre-bake common toolchains into the base image (faster container start, bigger image) vs. always `mise install` on checkout against a cached, mounted data dir (smaller image, first-run latency) — worth benchmarking rather than guessing.
|
[`docs/PLAN.md`](docs/PLAN.md).
|
||||||
- Is a `test` task in `.mise.toml` a hard requirement (container refuses to run the agent without one) or a soft convention the Stop hook just skips if missing? Leaning hard requirement, since a silent skip defeats the point of the gate.
|
|
||||||
- `forge` also ships as a Go library, not just a CLI — if the API implementation language lands on Go, worth importing it directly instead of shelling out to a subprocess. Relevant input to the language question above, not a decision on its own.
|
## License
|
||||||
- One token for both `forge` and git push vs. separate SSH deploy key + scoped API token per project — reasonable defaults differ by how much a given operator trusts a single credential with both git-write and forge-API scope.
|
|
||||||
- Polling cadence for the CI status backfill — fixed interval with backoff, or piggyback on the agent's own next checkpoint? A dedicated poller is simpler to reason about but is one more always-on process in an otherwise mostly-idle system.
|
MIT.
|
||||||
|
|||||||
+247
@@ -0,0 +1,247 @@
|
|||||||
|
# Remote Control Wrapper — Plan of Action
|
||||||
|
|
||||||
|
**Status:** Living document — update phase checkboxes and status in place as work completes. Don't append new copies of this file; overwrite it, the way a checkmark gets overwritten.
|
||||||
|
**Last updated:** 2026-07-08 (revision 8: Phase 1 MVP vertical slice implemented — see checkboxes and README.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. What "done" looks like
|
||||||
|
|
||||||
|
Production release means:
|
||||||
|
|
||||||
|
- **Runs anywhere.** No hard dependency on Gitea or Tailscale — git hosting and network exposure are both pluggable, not assumed.
|
||||||
|
- A **web UI**, not just tmux/CLI, for spawning, monitoring, and interacting with agents.
|
||||||
|
- **All communication goes through an API.** The UI is a client of it; tmux/CLI become optional local conveniences, not the source of truth.
|
||||||
|
- Every agent leaves a **checkmark** behind: a small, current-state record of where it stopped, what's needed next, and any open questions for you.
|
||||||
|
- Every checkmark links to an entry in a **big log**: the append-only, complete history of everything every agent has ever done.
|
||||||
|
- State lives in a **centralized database** that the CLI wrapper (backend) writes to and the API reads from — not scattered flat files or a git-hosting-specific store. Control layer and API containers hold nothing persistent themselves; they can restart, redeploy, or scale out without losing data because all of it lives in the database.
|
||||||
|
- **Multiple projects run concurrently, each isolated from the others by default.** One control layer can host any number of projects, each with its own agents, working directory, and history — with an explicit, opt-in mechanism for the edge cases where something genuinely needs to cross that boundary.
|
||||||
|
|
||||||
|
## 2. Non-negotiable constraints
|
||||||
|
|
||||||
|
- Every agent process is a real `claude` binary invocation. No OAuth handling, no protocol reimplementation.
|
||||||
|
- **No hard dependency on Tailscale.** The API authenticates itself with a bearer token; users choose their own network exposure — Tailscale, a VPN, a reverse proxy, or plain localhost. The wrapper doesn't assume any of them.
|
||||||
|
- **No hard dependency on Gitea.** Git operations use plain `git` by default. Forge-specific niceties (PRs, issues, CI status) go through `forge` (git-pkgs/forge) as an optional, pluggable layer — one CLI that works the same against GitHub, GitLab, Gitea/Forgejo, or Bitbucket, never a requirement just to run the tool.
|
||||||
|
- **The database never stores raw credentials.** `projects.credential_ref` is a pointer (an env var name, a file path, a command to run) — never a token. The control layer resolves it to an actual secret only at spawn time, injected into that container's environment for that run.
|
||||||
|
- Design decisions above a stakes threshold get surfaced via `plan` mode + `AskUserQuestion`, not silently guessed.
|
||||||
|
- **Projects are isolated by default.** An agent only sees its own project's working directory, checkmarks, and log — nothing crosses that boundary unless something is explicitly marked shared. This is one operator running many of their own projects, not a multi-tenant service for other people — don't let the isolation model drift into looking like the latter.
|
||||||
|
- **Agents don't get to self-report "done."** A verification gate actually runs the project's own test task and blocks completion on failure — "done" in the database means a test run passed, not that the agent stopped talking.
|
||||||
|
- **A `git push` doesn't leave until a local build check passes too.** Same hard-block pattern as the test gate, and it runs after tests, not instead of them — cheap checks run before the more expensive one that would obviously fail anyway.
|
||||||
|
- If this ships open-source: no embedded credentials, no implied Anthropic affiliation, README states plainly that each user brings their own Claude Code login, their own git remote, and their own network layer.
|
||||||
|
|
||||||
|
## 3. Control layer + API (build this first — this is the MVP)
|
||||||
|
|
||||||
|
### 3.1 Data model — Postgres primary, SQLite fallback
|
||||||
|
|
||||||
|
Two supported backends behind one data-access layer, not two separate code paths:
|
||||||
|
|
||||||
|
- **Postgres (default for real deployments).** A live, centralized server is what actually makes "stateless containers" true — the control layer and API are just compute that can restart, redeploy, or scale to N replicas because none of them hold state locally. This is the deployment target for anyone running more than a single node, or wanting one datastore behind multiple agent hosts.
|
||||||
|
- **SQLite (minimal-infra fallback).** A single file, zero services to stand up — for a single-node/homelab-scale run where standing up Postgres is overkill. Explicitly a fallback, not the default: it doesn't give you the centralization the Postgres path does, and single-writer semantics limit it to one control-layer instance at a time.
|
||||||
|
|
||||||
|
The schema is the same shape on both; only types differ slightly (Postgres gets proper `SERIAL`/`TIMESTAMPTZ`/`JSONB`, SQLite uses its looser dynamic typing). Pick one query layer/ORM that supports both dialects rather than hand-maintaining two schemas — see open questions.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Postgres
|
||||||
|
CREATE TABLE projects (
|
||||||
|
id TEXT PRIMARY KEY, -- slug, e.g. "leeworks-api"
|
||||||
|
root_dir TEXT NOT NULL,
|
||||||
|
git_remote TEXT,
|
||||||
|
credential_ref TEXT, -- pointer to a secret, e.g. "env:LEEWORKS_TOKEN" — never the token itself, see 3.7
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE agents (
|
||||||
|
id BIGSERIAL PRIMARY KEY,
|
||||||
|
project_id TEXT NOT NULL REFERENCES projects(id),
|
||||||
|
name TEXT NOT NULL, -- unique within a project, not globally
|
||||||
|
working_dir TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL, -- working | paused_for_input | blocked | done
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||||
|
UNIQUE (project_id, name)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE checkmarks (
|
||||||
|
agent_id BIGINT PRIMARY KEY REFERENCES agents(id),
|
||||||
|
checkpoint_at TIMESTAMPTZ NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
where_it_stopped TEXT,
|
||||||
|
next_steps JSONB,
|
||||||
|
open_question TEXT,
|
||||||
|
log_entry_id BIGINT REFERENCES log_entries(id),
|
||||||
|
tests_status TEXT NOT NULL DEFAULT 'unknown', -- pass | fail | unknown — see 3.5
|
||||||
|
tested_at TIMESTAMPTZ,
|
||||||
|
build_status TEXT NOT NULL DEFAULT 'unknown', -- pass | fail | unknown — see 3.6
|
||||||
|
built_at TIMESTAMPTZ
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE log_entries (
|
||||||
|
id BIGSERIAL PRIMARY KEY,
|
||||||
|
agent_id BIGINT NOT NULL REFERENCES agents(id),
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||||
|
session_id TEXT,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
summary TEXT,
|
||||||
|
decisions TEXT,
|
||||||
|
question TEXT,
|
||||||
|
answer TEXT, -- filled in on resume; only field ever touched post-insert
|
||||||
|
visibility TEXT NOT NULL DEFAULT 'project', -- project | global — see 3.4
|
||||||
|
push_sha TEXT, -- set if this checkpoint pushed; null if it didn't
|
||||||
|
ci_status TEXT NOT NULL DEFAULT 'not_applicable', -- not_applicable | pending | pass | fail — see 3.6
|
||||||
|
ci_checked_at TIMESTAMPTZ
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- SQLite (minimal-infra fallback) — same shape, simpler types
|
||||||
|
CREATE TABLE projects (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
root_dir TEXT NOT NULL,
|
||||||
|
git_remote TEXT,
|
||||||
|
credential_ref TEXT,
|
||||||
|
created_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
-- agents / checkmarks / log_entries: identical columns, JSONB becomes TEXT,
|
||||||
|
-- BIGSERIAL becomes INTEGER PRIMARY KEY AUTOINCREMENT, TIMESTAMPTZ becomes TEXT.
|
||||||
|
```
|
||||||
|
|
||||||
|
`checkmarks` is a literal upsert (`INSERT ... ON CONFLICT DO UPDATE` / `INSERT OR REPLACE`) keyed by `agent_id` — the "small file that gets overwritten," just as a row instead of a file. `log_entries` is insert-only except for the one `answer` backfill on resume, which is the DB equivalent of "linked directly to this checkmark." Agent identity is now `(project_id, name)`, not a bare name — two projects can each have an agent called `api` without colliding.
|
||||||
|
|
||||||
|
### 3.2 Backend (CLI wrapper) — the only thing that writes
|
||||||
|
|
||||||
|
- Spawns/lists/attaches/kills agents within a project (tmux + `claude` binary, one working directory or git worktree per agent, nested under that project's root)
|
||||||
|
- Owns every write to the database: creates the agent row, upserts the checkmark, inserts log entries
|
||||||
|
- Hooked via `Stop`/`SessionEnd` (checkpoint), `PreToolUse` (defer `AskUserQuestion`), `Notification` (fires a generic webhook — the user points that at ntfy, Pushover, Slack, email, whatever; the wrapper doesn't pick for them)
|
||||||
|
|
||||||
|
### 3.3 API — the only thing that reads (plus writes the answer back)
|
||||||
|
|
||||||
|
- Thin HTTP layer over the same database (Postgres in centralized deployments, SQLite for minimal-infra): `GET /projects`, `POST /projects`, `GET /projects/:project/agents`, `POST /projects/:project/agents`, `GET /projects/:project/agents/:name/checkmark`, `GET /projects/:project/agents/:name/log`, `POST /projects/:project/agents/:name/answer`, `POST /projects/:project/agents/:name/resume`
|
||||||
|
- Auth: a generated bearer token checked on every request. No assumption about what network the request arrives over — Tailscale, a VPN, or a bare reverse proxy are all just transport underneath it.
|
||||||
|
- The UI (Phase 3) and any future integration are just clients of this — same contract as `curl`.
|
||||||
|
|
||||||
|
### 3.4 Multi-project isolation and cross-project sharing
|
||||||
|
|
||||||
|
Every agent belongs to exactly one project. Isolation is the default; sharing is a deliberate, visible act, never an accident.
|
||||||
|
|
||||||
|
**Isolation:**
|
||||||
|
- Filesystem: each project gets its own root directory; agents in that project work only within it (subdirectories or git worktrees underneath), never reaching into another project's tree.
|
||||||
|
- Naming: tmux sessions follow `project__agent`, matching the `(project_id, name)` uniqueness in the database — no cross-project collisions, and it's visually obvious which project a session belongs to from `tmux ls`.
|
||||||
|
- Config: git remote, credential reference, and any other per-project settings, live on the `projects` row — not global config — so projects can point at entirely different repos, hosts, or forges independently.
|
||||||
|
- API: every agent route is nested under `/projects/:project/...`. There's no endpoint that returns another project's data by accident.
|
||||||
|
|
||||||
|
**The explicit sharing mechanism, for the edge cases:**
|
||||||
|
- `log_entries.visibility` — defaults to `project`, can be set to `global` when an agent (or you) decides a specific checkpoint genuinely matters beyond its own project. A `GET /shared/log` endpoint surfaces only the entries marked `global`, across all projects — a deliberate opt-in feed, not a merged view of everything.
|
||||||
|
- `shared_context` table — a small key-value store for standing facts multiple projects need to reference (a shared staging URL, a schema version, a convention decision), independent of any single checkpoint:
|
||||||
|
```sql
|
||||||
|
CREATE TABLE shared_context (
|
||||||
|
key TEXT PRIMARY KEY,
|
||||||
|
value TEXT NOT NULL,
|
||||||
|
set_by_agent_id BIGINT REFERENCES agents(id),
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
);
|
||||||
|
```
|
||||||
|
Any agent can read it (`GET /shared/context`); writing (`PUT /shared/context/:key`) is the one place worth gating behind an explicit flag or higher-trust token, since it's the one table every project implicitly trusts.
|
||||||
|
|
||||||
|
Nothing here allows one project to read another project's checkmarks or private log entries directly — the only paths across the boundary are the two explicit ones above.
|
||||||
|
|
||||||
|
### 3.5 Toolchains and test verification
|
||||||
|
|
||||||
|
Environment reliability and result reliability are two different problems — solve both, don't conflate them.
|
||||||
|
|
||||||
|
**Environment: mise, not nix.** A single generic base image (mise + the `claude` binary + `git` + `gh`) with no baked-in language runtimes — one image for every project regardless of stack. Each project carries its own `.mise.toml`, pinning exact tool versions (Rust, Python, Node, whatever) and a lockfile committed to the repo for strict, reproducible installs. On container start, `mise install --non-interactive` reads that lockfile and materializes the project's exact toolchain — the container doesn't need to know or care what language it's about to run. Nix would get similar pinning with more ceremony and a heavier runtime; since Docker already provides OS-level isolation, mise's language-level pinning is the right amount of tool for this.
|
||||||
|
|
||||||
|
Every project defines a canonical task, regardless of stack:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[tasks.test]
|
||||||
|
description = "Run the test suite"
|
||||||
|
run = "pytest" # or `cargo test`, `npm test`, whatever the project actually uses
|
||||||
|
|
||||||
|
[tasks.verify]
|
||||||
|
depends = ["lint", "test"]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result: a Stop hook, not self-reporting.** After each turn, a `Stop` hook runs `mise run test` (or `verify`) in the agent's working directory and checks the exit code. Nonzero → `decision: "block"`, with the failure output fed back to the agent, so the turn cannot end on a broken test suite. This is what "correct" actually rests on: it's the same command for every project because mise's task abstraction is the uniform interface, and it's enforced by the harness, not reported by the model.
|
||||||
|
|
||||||
|
The result feeds straight into the schema already in place: the hook sets `checkmarks.tests_status` and `tested_at` on every checkpoint, so `status = 'done'` in the database is only ever true alongside a passing test run — not a claim taken on faith.
|
||||||
|
|
||||||
|
### 3.6 Push gate and CI follow-through
|
||||||
|
|
||||||
|
Tests catch broken code; they don't catch a broken container. A Dockerfile with a stale `COPY` path passes `pytest` fine and then fails to build — worth catching locally, before it costs a CI run.
|
||||||
|
|
||||||
|
**Local build check — cheap, throwaway, no registry involved.** Since CI/CD (on its own, beefier runner) owns the real build-and-push, the local check only needs to prove the Dockerfile is sound: a `kaniko`/`buildah` build with no push target, output discarded. No Docker socket, no `--privileged`, nothing the agent's own sandbox needs elevated trust for — it's the same category of tool CI systems adopted specifically to avoid handing out real Docker-in-Docker access. This is `mise run build-image` alongside the existing `test` task, same interface, same project regardless of stack.
|
||||||
|
|
||||||
|
**Enforcement — same mechanism as tests, one step later.** A `PreToolUse` hook matching `Bash(git push*)` runs the verification chain (tests, then the throwaway build — cheap first, expensive second) and returns `permissionDecision: "deny"` on the first failure, with the reason surfaced to the agent. A push that's already known to fail CI doesn't leave. `checkmarks.build_status`/`built_at` record the result the same way `tests_status` does.
|
||||||
|
|
||||||
|
**CI is still the authoritative gate — the wrapper just closes the loop on it.** Once a push clears the local checks and actually goes out, whatever CI system the host runs (GitHub Actions, Gitea/Forgejo Actions, GitLab CI) does the real build-and-push on its own runner, same as today. The wrapper records `push_sha` and sets `ci_status = 'pending'` on that log entry, then a background poller in the control layer — not a hook, since this can take minutes — checks the run tied to that commit with `forge ci list` / `forge ci log <id>`. Same two commands regardless of which host the project is on; `forge` detects the forge type from the remote, so this isn't per-host integration work, just one poller calling one interface. Once the run resolves, the poller backfills `ci_status` and `ci_checked_at` on that same log entry — the DB equivalent of "recording why it did or didn't ultimately land," without ever exposing the wrapper to inbound webhook traffic.
|
||||||
|
|
||||||
|
### 3.7 Credentials: forge and git
|
||||||
|
|
||||||
|
Two things need to authenticate per project — the `forge` CLI (for PRs, issues, CI status) and `git` itself (for push/pull) — and the design goal is one secret servicing both, not two to manage.
|
||||||
|
|
||||||
|
**Resolution, not storage.** `projects.credential_ref` is a pointer (`env:VAR_NAME`, `file:/path`, `cmd:some command`), never the token itself — mirrors `forge`'s own `--token-cmd` pattern, just one level up. At spawn time, the control layer resolves the reference to an actual value and injects it into that agent's container environment only — nothing persisted in Postgres/SQLite, nothing baked into an image.
|
||||||
|
|
||||||
|
**One token, two consumers, where the host allows it.** `forge` reads the resolved token from the environment directly (`GITHUB_TOKEN`, `GITEA_TOKEN`, `FORGE_TOKEN`, per host) with zero extra config once it's injected. For git's own HTTPS auth, the wrapper writes a small credential-helper at container start that hands back that same value — most self-hosted forges (Gitea/Forgejo, GitLab) accept a PAT as the HTTP password, so one token covers both `forge pr create` and `git push` for a given project.
|
||||||
|
|
||||||
|
**SSH deploy keys as the per-project alternative.** Some operators won't want a single token holding both git-write and forge-API scope. Where that matters, a project can use an SSH deploy key for git transport and a separately-scoped, more limited token for `forge`'s PR/issue/CI operations — configured per project, not mandated globally, since this is a real preference split among self-hosters rather than a settled question.
|
||||||
|
|
||||||
|
## 4. Phased roadmap
|
||||||
|
|
||||||
|
### Phase 0 — Prerequisites
|
||||||
|
- [ ] `claude` binary installed and authenticated (per user)
|
||||||
|
- [ ] `git` installed; `forge` (git-pkgs/forge) optional
|
||||||
|
- [ ] `mise` installed in the base agent image; each project supplies its own `.mise.toml` with, at minimum, a `test` task
|
||||||
|
- [ ] A Postgres connection string (default target) — or nothing at all if running the SQLite fallback for a single-node/minimal-infra setup
|
||||||
|
|
||||||
|
### Phase 1 — Control layer + API (the MVP)
|
||||||
|
|
||||||
|
Implemented as a vertical slice — see `README.md` for what runs today. Live end-to-end
|
||||||
|
agent spawning against a real `claude` binary + tmux is deferred behind mocked seams
|
||||||
|
(`control.tmux`, `hooks.verify`, `control.spawn.resume`).
|
||||||
|
|
||||||
|
- [x] Database schema + migrations for both backends, Postgres default / SQLite fallback, including `projects` and project-scoped agents (section 3.1)
|
||||||
|
- [x] Control script: spawn/list/attach/kill, tmux + `claude` binary, one working dir/worktree per agent, namespaced by project — itself stateless, all state written straight to the database
|
||||||
|
- [x] Hooks writing checkmark/log rows (`Stop`/`SessionEnd`, `PreToolUse` defer, `Notification` → generic webhook), all scoped to the owning project
|
||||||
|
- [x] HTTP API over the same database, bearer-token auth, all agent routes nested under `/projects/:project/`
|
||||||
|
- [x] Resume flow: API endpoint takes an answer, feeds it back via `claude --resume` (control seam)
|
||||||
|
- [x] Shared-context and shared-log endpoints for the explicit cross-project edge case (section 3.4)
|
||||||
|
- [x] `Stop` hook verification gate: `mise run test` on every checkpoint, block completion on failure (section 3.5). `mise install` on checkout deferred to live spawn wiring.
|
||||||
|
- [x] `PreToolUse` hook on `git push*`: throwaway build (no registry) after tests pass, hard-deny the push on either failure (section 3.6). Wired to `mise run build-image`; the real `kaniko`/`buildah` invocation is deferred behind the verify seam.
|
||||||
|
|
||||||
|
**Definition of done:** run several projects side by side, each with its own agents, working directories, toolchain, and history, entirely through `curl` + a token, against either a live Postgres instance or the SQLite fallback — no Gitea, no Tailscale, no UI required, no state stored anywhere the control layer or API containers themselves live, nothing crosses a project boundary unless explicitly shared, no agent reaches `done` without a passing test run to show for it, and no push leaves that a local build already proved would fail.
|
||||||
|
|
||||||
|
### Phase 2 — Forge integration
|
||||||
|
- [ ] Repo-scoped actions via `forge`: branch creation, PR open, issue linking — one interface across GitHub/GitLab/Gitea/Forgejo/Bitbucket instead of a per-host integration
|
||||||
|
- [ ] Pin `forge` to a specific released version, not `@latest` — accepted risk given the project's youth, mitigated by not floating on a moving target
|
||||||
|
- [ ] Credential resolution (section 3.7): `credential_ref` → injected env var at spawn, shared by `forge` and git's credential helper
|
||||||
|
- [ ] Background poller reading back CI run results via `forge ci list` / `forge ci log`, backfilling `ci_status`/`ci_checked_at` on the log entry that recorded the push (section 3.6)
|
||||||
|
|
||||||
|
### Phase 3 — Production UI
|
||||||
|
- [ ] Web frontend, API-backed only (same contract as `curl`)
|
||||||
|
- [ ] Project switcher, agent list per project, live checkmark view, log history, "answer this question" form, plus a view for the shared/global feed
|
||||||
|
|
||||||
|
**Definition of done:** open a URL, see every agent's state, answer a paused question, no terminal required.
|
||||||
|
|
||||||
|
### Phase 4 — Observability (moved back, now optional)
|
||||||
|
- [ ] Prometheus metrics endpoint on the API (agent counts, pending questions, checkpoint rate)
|
||||||
|
- [ ] Grafana/Loki wiring documented as an optional add-on for self-hosters who already run that stack — not a dependency for anyone else
|
||||||
|
|
||||||
|
### Phase 5 — Open-source release
|
||||||
|
- [ ] Strip any remaining homelab-specific defaults into config
|
||||||
|
- [ ] README: bring-your-own Claude Code login, bring-your-own network, bring-your-own git host, bring-your-own credential source (env var, file, or command)
|
||||||
|
- [ ] Publish
|
||||||
|
|
||||||
|
## 5. Open questions
|
||||||
|
|
||||||
|
Decisions made for the Phase 1 MVP are marked **[resolved]**; the rest remain open.
|
||||||
|
|
||||||
|
- Project name — still Cutout / Handler / Dead Drop / Backchannel / Umbra. **[resolved: `handler` for now — the package and repo name.]**
|
||||||
|
- API implementation language/framework — wants a good story for *both* Postgres and SQLite (a query layer or ORM that speaks both dialects) plus minimal runtime deps, given "portable" is still a hard requirement. **[resolved: Python + FastAPI + SQLAlchemy Core + Alembic.]**
|
||||||
|
- Default webhook target for `Notification` — ship a zero-config adapter (ntfy needs no account) or leave it fully bring-your-own from day one? **[resolved: fully bring-your-own via `WEBHOOK_URL`; no-op when unset.]**
|
||||||
|
- Reference deployment for Postgres — user-supplied external instance only, or does the repo also ship a docker-compose/StatefulSet example for people with nowhere else to put it? (Either way, Postgres is the one stateful component in the system — everything else stays a disposable container.)
|
||||||
|
- Who can write to `shared_context` — any agent by default, or does it need a higher-trust token than the per-project routes get? **[resolved: gated behind an optional higher-trust `SHARED_CONTEXT_WRITE_TOKEN`, falling back to the global token when unset.]**
|
||||||
|
- Per-project API tokens vs one global token — least-privilege argues for scoping tokens to a project, but this is single-operator, not multi-tenant, so a global token may just be simpler and sufficient. **[resolved: single global bearer token for the MVP.]**
|
||||||
|
- Pre-bake common toolchains into the base image (faster container start, bigger image) vs. always `mise install` on checkout against a cached, mounted data dir (smaller image, first-run latency) — worth benchmarking rather than guessing.
|
||||||
|
- Is a `test` task in `.mise.toml` a hard requirement (container refuses to run the agent without one) or a soft convention the Stop hook just skips if missing? Leaning hard requirement, since a silent skip defeats the point of the gate. **[resolved: hard requirement — `spawn` refuses a project with no `[tasks.test]`.]**
|
||||||
|
- `forge` also ships as a Go library, not just a CLI — if the API implementation language lands on Go, worth importing it directly instead of shelling out to a subprocess. Relevant input to the language question above, not a decision on its own. **[resolved moot: language is Python; `forge` will be shelled out to in Phase 2.]**
|
||||||
|
- One token for both `forge` and git push vs. separate SSH deploy key + scoped API token per project — reasonable defaults differ by how much a given operator trusts a single credential with both git-write and forge-API scope.
|
||||||
|
- Polling cadence for the CI status backfill — fixed interval with backoff, or piggyback on the agent's own next checkpoint? A dedicated poller is simpler to reason about but is one more always-on process in an otherwise mostly-idle system.
|
||||||
Reference in New Issue
Block a user