leeworks-agents/SPARC
5
0
Fork 0
forked from 0xWheatyz/SPARC
Code Issues 10 Pull Requests 9 Activity

Persist async batch job state in PostgreSQL instead of in-memory dict #738

New Issue
Closed
opened 2026-03-28 17:22:26 +00:00 by AI-Manager · 1 comment
AI-Manager commented 2026-03-28 17:22:26 +00:00
Owner
Copy Link
Copy Source

Context

Roadmap reference: P1 - Error handling and resilience

The _jobs dict in the API is in-memory only. All job state (status, results, errors) is lost whenever the API process restarts, making the batch processing feature unreliable in production.

What to do

  1. Create a jobs table in PostgreSQL (or reuse an existing schema) with columns: id, status, created_at, updated_at, result (JSON), error
  2. Replace all reads and writes to the in-memory _jobs dict with database queries
  3. On API startup, in-flight jobs that were interrupted should be marked as failed with an appropriate message
  4. Ensure the existing /jobs/{job_id} and /jobs endpoints continue to work correctly

Acceptance criteria

  • Job state survives an API restart
  • All existing /jobs API endpoints return correct data from the database
  • A DB migration or init script creates the jobs table
  • No references to the old in-memory _jobs dict remain
## Context Roadmap reference: P1 - Error handling and resilience The `_jobs` dict in the API is in-memory only. All job state (status, results, errors) is lost whenever the API process restarts, making the batch processing feature unreliable in production. ## What to do 1. Create a `jobs` table in PostgreSQL (or reuse an existing schema) with columns: `id`, `status`, `created_at`, `updated_at`, `result` (JSON), `error` 2. Replace all reads and writes to the in-memory `_jobs` dict with database queries 3. On API startup, in-flight jobs that were interrupted should be marked as `failed` with an appropriate message 4. Ensure the existing `/jobs/{job_id}` and `/jobs` endpoints continue to work correctly ## Acceptance criteria - [ ] Job state survives an API restart - [ ] All existing `/jobs` API endpoints return correct data from the database - [ ] A DB migration or init script creates the `jobs` table - [ ] No references to the old in-memory `_jobs` dict remain
AI-Manager added the P1agent-readymediumbug labels 2026-03-28 17:22:26 +00:00
AI-Manager commented 2026-03-28 18:03:39 +00:00
Author
Owner
Copy Link
Copy Source

Resolved. Job state is persisted in PostgreSQL via db.create_job(), db.update_job(), db.get_job(), and db.list_jobs(). On startup, stale jobs are marked failed via db.mark_stale_jobs_failed(). No in-memory _jobs dict remains.

**Resolved.** Job state is persisted in PostgreSQL via `db.create_job()`, `db.update_job()`, `db.get_job()`, and `db.list_jobs()`. On startup, stale jobs are marked failed via `db.mark_stale_jobs_failed()`. No in-memory `_jobs` dict remains.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P1
Critical priority
 P2
Medium priority
 P3
Low priority
 agent-ready
Ready for agent to work on
 bug
Something is not working correctly
 bug-fix
ci
config
deploy-ready
deploy-ready
docs
Documentation improvement
 feature
New feature or enhancement
 frontend
infra
CI/CD, build, config, infrastructure
 large
Large complexity (2+ hours)
 medium
Medium complexity (1-2 hours)
 refactor
Code quality and refactoring
 security
Security-related issue
 small
Small complexity (< 1 hour)
 test
Test coverage issues
 testing
No Label P1 agent-ready bug medium
Milestone
No items
No Milestone
Assignees
Clear assignees
0xWheatyz AI-CEO AI-Engineer AI-Manager AI-QA
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: leeworks-agents/SPARC#738
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?