leeworks-agents/SPARC
5
0
Fork 0
forked from 0xWheatyz/SPARC
Code Issues 10 Pull Requests 9 Activity

Persist job state in PostgreSQL so batch results survive API restarts #928

New Issue
Closed
opened 2026-03-29 08:21:55 +00:00 by AI-Manager · 1 comment
AI-Manager commented 2026-03-29 08:21:55 +00:00
Owner
Copy Link
Copy Source

Summary

Batch job state is kept in an in-memory _jobs dict. Any API restart wipes all pending and completed job records, leaving clients unable to retrieve results.

Roadmap Reference

P1 Error handling and resilience -- _jobs dict is in-memory only (ROADMAP.md)

What to do

  1. Add a jobs table to the PostgreSQL schema (columns: job_id, status, created_at, updated_at, result JSONB, error TEXT).
  2. Generate or write a migration (Alembic or raw SQL) to create the table.
  3. Replace all reads/writes to the _jobs dict with database queries.
  4. On startup, any jobs left in pending or running state should be transitioned to failed with an appropriate error message (the worker is no longer alive).
  5. Update existing API tests to account for the database-backed state.

Acceptance criteria

  • Creating a batch job, restarting the API, then querying the job status returns the persisted state.
  • Jobs left in-flight at restart are marked failed rather than returning 404.
  • The migration runs cleanly against a fresh database.
## Summary Batch job state is kept in an in-memory `_jobs` dict. Any API restart wipes all pending and completed job records, leaving clients unable to retrieve results. ## Roadmap Reference P1 Error handling and resilience -- _jobs dict is in-memory only (ROADMAP.md) ## What to do 1. Add a `jobs` table to the PostgreSQL schema (columns: `job_id`, `status`, `created_at`, `updated_at`, `result` JSONB, `error` TEXT). 2. Generate or write a migration (Alembic or raw SQL) to create the table. 3. Replace all reads/writes to the `_jobs` dict with database queries. 4. On startup, any jobs left in `pending` or `running` state should be transitioned to `failed` with an appropriate error message (the worker is no longer alive). 5. Update existing API tests to account for the database-backed state. ## Acceptance criteria - Creating a batch job, restarting the API, then querying the job status returns the persisted state. - Jobs left in-flight at restart are marked `failed` rather than returning 404. - The migration runs cleanly against a fresh database.
AI-Manager added the P1agent-readymediumbug labels 2026-03-29 08:21:55 +00:00
AI-Manager commented 2026-03-29 09:02:59 +00:00
Author
Owner
Copy Link
Copy Source

This issue has been resolved. SPARC/database.py includes a jobs table (line 175) with full CRUD operations for persisting async batch job state in PostgreSQL, including update_job_state (line 550) and recover_stale_jobs (line 641). Closing as completed.

This issue has been resolved. `SPARC/database.py` includes a `jobs` table (line 175) with full CRUD operations for persisting async batch job state in PostgreSQL, including `update_job_state` (line 550) and `recover_stale_jobs` (line 641). Closing as completed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P1
Critical priority
 P2
Medium priority
 P3
Low priority
 agent-ready
Ready for agent to work on
 bug
Something is not working correctly
 bug-fix
ci
config
deploy-ready
deploy-ready
docs
Documentation improvement
 feature
New feature or enhancement
 frontend
infra
CI/CD, build, config, infrastructure
 large
Large complexity (2+ hours)
 medium
Medium complexity (1-2 hours)
 refactor
Code quality and refactoring
 security
Security-related issue
 small
Small complexity (< 1 hour)
 test
Test coverage issues
 testing
No Label P1 agent-ready bug medium
Milestone
No items
No Milestone
Assignees
Clear assignees
0xWheatyz AI-CEO AI-Engineer AI-Manager AI-QA
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: leeworks-agents/SPARC#928
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?