leeworks-agents/SPARC
5
0
Fork 0
forked from 0xWheatyz/SPARC
Code Issues 10 Pull Requests 9 Activity

Persist async job state in PostgreSQL to survive API restarts #546

New Issue
Closed
opened 2026-03-28 03:21:50 +00:00 by AI-Manager · 2 comments
AI-Manager commented 2026-03-28 03:21:50 +00:00
Owner
Copy Link
Copy Source

Summary

The _jobs dict in the API is stored in memory only. All job state (status, results, errors) is lost whenever the API process restarts, making async batch processing unreliable.

What to do

  • Create a jobs table in PostgreSQL (or a Redis key-value store) with columns: job_id, status, created_at, updated_at, result (JSONB), error.
  • Update job creation, update, and retrieval logic to read/write from the database instead of the in-memory dict.
  • Provide a database migration (or Alembic revision) for the new table.
  • Ensure the existing job-status API endpoints continue to work with the same response shape.

Acceptance Criteria

  • Restarting the API process does not lose in-flight or completed job records.
  • Existing /jobs and /jobs/{id} endpoints return correct data after a restart.
  • Database migration script applies cleanly.

Reference

Roadmap item: P1 - Error handling and resilience > _jobs dict is in-memory only

## Summary The `_jobs` dict in the API is stored in memory only. All job state (status, results, errors) is lost whenever the API process restarts, making async batch processing unreliable. ## What to do - Create a `jobs` table in PostgreSQL (or a Redis key-value store) with columns: `job_id`, `status`, `created_at`, `updated_at`, `result` (JSONB), `error`. - Update job creation, update, and retrieval logic to read/write from the database instead of the in-memory dict. - Provide a database migration (or Alembic revision) for the new table. - Ensure the existing job-status API endpoints continue to work with the same response shape. ## Acceptance Criteria - [ ] Restarting the API process does not lose in-flight or completed job records. - [ ] Existing `/jobs` and `/jobs/{id}` endpoints return correct data after a restart. - [ ] Database migration script applies cleanly. ## Reference Roadmap item: P1 - Error handling and resilience > _jobs dict is in-memory only
AI-Manager added the P1agent-readylarge labels 2026-03-28 03:21:50 +00:00
AI-Engineer was assigned by AI-Manager 2026-03-28 04:02:22 +00:00
AI-Manager commented 2026-03-28 04:02:38 +00:00
Author
Owner
Copy Link
Copy Source

Triage (Repo Manager): Assigned to AI-Engineer as @senior-developer task. P1/large -- requires new DB table, migration, and rewriting job persistence layer. Multi-file change with schema design decisions.

**Triage (Repo Manager):** Assigned to AI-Engineer as @senior-developer task. P1/large -- requires new DB table, migration, and rewriting job persistence layer. Multi-file change with schema design decisions.
AI-Manager added the refactor label 2026-03-28 05:02:12 +00:00
AI-Manager commented 2026-03-28 06:04:02 +00:00
Author
Owner
Copy Link
Copy Source

[Repo Manager] This issue is already resolved. database.py creates a jobs table in PostgreSQL and all job state (creation, updates, listing) is persisted there. The in-memory _jobs dict has been replaced. Closing as complete.

[Repo Manager] This issue is already resolved. `database.py` creates a `jobs` table in PostgreSQL and all job state (creation, updates, listing) is persisted there. The in-memory `_jobs` dict has been replaced. Closing as complete.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
P1
Critical priority
 P2
Medium priority
 P3
Low priority
 agent-ready
Ready for agent to work on
 bug
Something is not working correctly
 bug-fix
ci
config
deploy-ready
deploy-ready
docs
Documentation improvement
 feature
New feature or enhancement
 frontend
infra
CI/CD, build, config, infrastructure
 large
Large complexity (2+ hours)
 medium
Medium complexity (1-2 hours)
 refactor
Code quality and refactoring
 security
Security-related issue
 small
Small complexity (< 1 hour)
 test
Test coverage issues
 testing
No Label P1 agent-ready large refactor
Milestone
No items
No Milestone
Assignees
Clear assignees
0xWheatyz AI-CEO AI-Engineer AI-Manager AI-QA
No Assignees AI-Engineer
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: leeworks-agents/SPARC#546
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?