fix(control,ui): close review findings - heartbeat starvation, resume race, stale UI writes

Fixes from the post-migration code review (3 major, 4 minor):

- worker: heartbeat between every drained command so a long queue can't
  starve proof-of-life into a false reap; worker_stale_after default
  60s -> 300s (one slow sync/login command must not look like a crash)
- repository.create_run: enforces one running run per agent atomically
  (agent-row FOR UPDATE on Postgres; SQLite's single writer suffices) -
  two workers claiming resumes for the same agent can no longer both
  launch claude on one session; resume surfaces the loss loudly
- headless._settle: upload the final session archive BEFORE marking the
  run finished - a resume claimed the instant a run leaves 'running'
  materializes from session_archives, and the old order let it race an
  incomplete archive into needless context re-injection (found as a
  test flake, real in production)
- store.tsx: generation token drops in-flight loadRun writes after the
  user switches runs (run A's events/log/checkmark no longer land on
  run B), plus id-keyed dedup on event appends from overlapping polls
- credsync: credential files written 0600 from the first byte
- headless: seq counter locked (reader thread + supervisor both emit
  events); proc.stdout closed after reader join
- login: submit pins to the latest CLAIMED login_start (a still-running
  one previously pinned to the wrong worker)

Suite 296 green (new: create_run conflict coverage); reaper tests track
the new staleness default.
This commit is contained in:
2026-07-21 23:41:36 -04:00
parent 1517e4dca8
commit 478d178542
18 changed files with 145 additions and 57 deletions
+3 -3
View File
@@ -36,7 +36,7 @@ def _hb(worker_id, age_seconds=0.0):
def test_reaper_marks_stale_workers_runs_crashed(env):
agent, run = _seed_run("w-dead")
_hb("w-dead", age_seconds=120) # stale: default worker_stale_after is 60s
_hb("w-dead", age_seconds=600) # stale: default worker_stale_after is 300s
assert worker.reap_dead_workers() == 1
@@ -64,7 +64,7 @@ def test_reaper_preserves_paused_agent_status(env):
"""paused_for_input still accurately describes what the agent needs — only agents
stuck in 'working' get flipped to crashed."""
agent, run = _seed_run("w-dead2", agent_name="paused", agent_status="paused_for_input")
_hb("w-dead2", age_seconds=120)
_hb("w-dead2", age_seconds=600)
assert worker.reap_dead_workers() == 1
with get_engine().begin() as conn:
@@ -74,7 +74,7 @@ def test_reaper_preserves_paused_agent_status(env):
def test_reaper_idempotent_against_races(env):
agent, run = _seed_run("w-dead3", agent_name="raced")
_hb("w-dead3", age_seconds=120)
_hb("w-dead3", age_seconds=600)
assert worker.reap_dead_workers() == 1
# A second reaper (or the same one next pass) finds nothing left to settle.
assert worker.reap_dead_workers() == 0