Commit Graph

105 Commits

Author SHA1 Message Date
agent-company 2e6b8c7445 feat: add webhook notification support for job completion and alerts
Send HTTP POST notifications to configured webhook URLs when batch
jobs complete or when scheduled analysis detects significant changes.

- Add SPARC/webhooks.py with retry logic (3 attempts, exponential backoff)
- Support generic HTTP POST and Slack-compatible text payloads
- Integrate into batch job completion handler in api.py
- Configure via WEBHOOK_URLS env var (comma-separated)
- Payload includes event type, job ID, status, and summary

Closes leeworks-agents/SPARC#23

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 10:32:07 +00:00
AI-Manager 55c131cb32 Merge pull request 'ci: add pytest and ruff linting to CI workflow' (#32) from feature/ci-testing-linting into main
Test and Lint / test (pull_request) Has been cancelled
2026-03-26 07:04:31 +00:00
agent-company fbb72fe2a5 ci: add pytest and ruff linting to CI, fix all lint errors
- Add test job to build.yaml that runs pytest and ruff before building images
- Add standalone test.yaml workflow for PRs
- Add ruff.toml with E/F/I rules configured
- Fix all ruff lint errors: sort imports, remove unused imports, fix re-exports
- Build jobs now depend on test job passing (needs: test)

Closes leeworks-agents/SPARC#18
Closes leeworks-agents/SPARC#19

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 07:04:00 +00:00
AI-Manager e484baaf5f Merge pull request 'feat: configurable LLM model, SERP cache TTL, structured logging, fix type' (#29) from feature/p2-config-improvements into main 2026-03-26 07:03:08 +00:00
AI-Manager 069f1c343c Merge pull request 'refactor(db): shared pooled DatabaseClient singleton' (#30) from feature/db-client-pooling into main 2026-03-26 07:02:46 +00:00
agent-company d366443b38 refactor(db): use shared pooled DatabaseClient singleton instead of per-call instances
- Replace get_db_client() creating new DatabaseClient on every call with a
  module-level singleton initialized once at startup via init_db_client()
- Add init_db_client() and close_db_client() lifecycle functions called
  from FastAPI lifespan handler
- Migrate all DatabaseClient methods from legacy self.connect()/self.conn
  to pooled self.get_conn() context manager for thread-safe connection reuse
- Pool is properly torn down on application shutdown

Closes leeworks-agents/SPARC#7

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 06:03:56 +00:00
agent-company b000146585 feat: configurable LLM model, SERP cache TTL, structured logging, fix patent_id type
- Make LLM model configurable via MODEL env var, default anthropic/claude-3.5-sonnet (#12)
- Expose SERP cache TTL as SERP_CACHE_TTL_HOURS env var, default 24 hours (#13)
- Fix Patent.patent_id type annotation from int to str in types.py (#14)
- Replace all print() calls with structured logging in analyzer.py and llm.py (#11)
- Add LOG_LEVEL config with basicConfig setup in config.py
- Add model and serp_cache_ttl_hours to config.py

Closes leeworks-agents/SPARC#11
Closes leeworks-agents/SPARC#12
Closes leeworks-agents/SPARC#13
Closes leeworks-agents/SPARC#14

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 06:03:25 +00:00
AI-Manager 35d105b14e Merge pull request 'feat(auth): add rate limiting to login and register endpoints' (#28) from feature/rate-limiting into main 2026-03-26 05:04:46 +00:00
AI-Manager 6fcf170d93 Merge pull request 'feat(jobs): persist async batch job state in PostgreSQL' (#34) from feature/persist-job-state into main 2026-03-26 05:04:26 +00:00
AI-Manager 5a42e216ba Merge pull request 'docs: patent PDF storage docs, FileNotFoundError, frontend lockfile' (#31) from feature/p2-docs-and-lockfile into main 2026-03-26 05:04:01 +00:00
AI-Manager 24ab341d9b Merge pull request 'test(auth): add comprehensive JWT authentication test suite' (#35) from feature/jwt-auth-tests into main 2026-03-26 05:03:29 +00:00
AI-Manager 878fedfbb8 Merge pull request 'feat(security): JWT startup guard, configurable CORS, externalize DB creds' (#27) from feature/p1-security-hardening into main 2026-03-26 05:03:16 +00:00
agent-company ae9f257dcb test(auth): add comprehensive JWT authentication test suite
Add 17 tests in tests/test_auth.py covering all auth flows:
- Registration: first user admin, subsequent user, duplicate email
- Login: valid credentials, invalid credentials
- Protected routes: valid token, missing token, expired token, wrong token type
- Token refresh: valid refresh, invalid refresh, access-as-refresh rejected
- Admin endpoints: list users, change role, own-role prevention, permission checks

All tests use mocked database (no live DB required).

Closes leeworks-agents/SPARC#10

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 04:24:12 +00:00
agent-company 96d5d27b17 feat(jobs): persist async batch job state in PostgreSQL
- Add jobs table to database schema (job_id, status, progress, result_json, etc.)
- Add DatabaseClient methods: create_job, update_job, get_job, list_jobs
- Add mark_stale_jobs_failed() called at startup to handle interrupted jobs
- Refactor _run_batch_job and job endpoints to read/write from PostgreSQL
- Remove in-memory _jobs dict; job state now survives API restarts
- Update init_database.py to list all tables in output

Closes leeworks-agents/SPARC#8

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 04:22:57 +00:00
agent-company 3dac88ec90 docs: document patent PDF storage, add FileNotFoundError, commit lockfile
- Add docstring to analyze_single_patent explaining the PDF prerequisite
- Raise FileNotFoundError with helpful message when PDF is missing
- Add patent PDF storage section to README with Docker volume mount example
- Commit frontend/package-lock.json for reproducible builds

Closes leeworks-agents/SPARC#15
Closes leeworks-agents/SPARC#17

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 04:17:09 +00:00
agent-company e2d750146c feat(auth): add rate limiting to login and register endpoints
- Add slowapi rate limiter: 10 req/min for /auth/login, 5 req/min for /auth/register
- Return HTTP 429 with Retry-After header when limit is exceeded
- Add slowapi to requirements.txt
- Add 4 passing tests for rate limit behavior

Closes leeworks-agents/SPARC#9

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 04:08:22 +00:00
agent-company 47cddcbeaf feat(security): add JWT startup guard, configurable CORS, and externalize DB credentials
- Add check_jwt_secret() that refuses default JWT secret when APP_ENV != development
- Make CORS origins configurable via CORS_ORIGINS env var (comma-separated)
- Replace hardcoded postgres credentials in docker-compose.yml with env var references
- Add APP_ENV and cors_origins to config.py
- Update .env.example with all required variables and documentation
- Add tests for JWT startup guard and CORS configuration

Closes leeworks-agents/SPARC#4
Closes leeworks-agents/SPARC#5
Closes leeworks-agents/SPARC#6

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 04:06:31 +00:00
AI-Manager 6105ba7793 Merge pull request 'chore: add ROADMAP.md for SPARC application development' (#3) from chore/add-roadmap into main 2026-03-26 02:47:54 +00:00
agent-company e8cdc089fa chore: add ROADMAP.md for SPARC application development
- Document current project state and architecture
- Identify P1 priorities: security hardening, error handling, test coverage
- Identify P2 priorities: structured logging, configurable LLM, frontend polish, CI tests
- Identify P3 priorities: export, comparison, scheduled analysis, notifications
- Reference Talos repo for infrastructure/deployment concerns

Closes leeworks-agents/SPARC#2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 00:06:56 +00:00
0xWheatyz 9c971dac72 fix(analyzer): route _analyze_company_safe through cache-aware path
Build and Push Docker Images / build-api (push) Successful in 2m19s
Build and Push Docker Images / build-frontend (push) Successful in 1m49s
_analyze_company_safe was calling SERP.query directly, bypassing the
SERP query cache in analyze_company. Now delegates fully to
analyze_company() and reads patent_count from the serp_queries cache.
2026-03-24 15:02:19 -04:00
0xWheatyz 6f0b448044 test(analyzer,serp): add tests for caching, single query, and parallel processing
- Add TestSingleQueryBugFix: verify SERP.query called once per analysis
- Add TestPatentCaching: DB cache hit/miss, SERP query cache hit/miss
- Add TestDynamicDateRange: rolling window, days_back param
- Add TestFilesystemPDFCaching: skip download, redownload empty files
- Add autouse mock_db fixture to prevent real DB connections in all tests
2026-03-24 14:39:09 -04:00
0xWheatyz 1a297eb60b feat(analyzer): integrate DB patent and SERP query caching
Before querying SERP API, check serp_queries cache (24h TTL). Before
downloading/parsing each patent, check patents table for cached
minimized_content. Store results after processing so repeated analyses
skip all network I/O and PDF parsing entirely.
2026-03-24 14:35:24 -04:00
0xWheatyz 3154f6b732 feat(database): add patent/serp caching tables and connection pooling
- Add patents table (patent_id PK, raw_sections JSONB, minimized_content)
- Add serp_queries table (query_hash unique, result_patent_ids, expires_at)
- Add cache methods: get/store_patent, get/store_serp_query
- Replace single connection with ThreadedConnectionPool (min=2, max=10)
- Add get_conn() context manager for thread-safe connection checkout
- Legacy single-connection path preserved for backwards compatibility
2026-03-24 14:34:33 -04:00
0xWheatyz b9bb3dc1cd perf(analyzer): parallelize patent download/parse/minimize with threads
Replace the sequential per-patent loop with a ThreadPoolExecutor
(workers controlled by PATENT_THREAD_WORKERS config). Each patent is
processed independently in _process_single_patent, which is thread-safe
since SERP methods are stateless and operate on separate files.
2026-03-24 14:32:23 -04:00
0xWheatyz 90f9cfc826 fix(serp): replace hardcoded date range with rolling window
The SERP query had a frozen date range (Oct-Nov 2025) that returned
stale patents. Now computes a rolling window from config
(PATENT_SEARCH_DAYS, default 90 days). Also adds filesystem-level PDF
caching to skip re-downloading existing patent PDFs, and adds
PATENT_THREAD_WORKERS config for upcoming parallel processing.
2026-03-24 14:31:43 -04:00
0xWheatyz d387bbbdf3 fix(analyzer): eliminate double SERP.query() call per company analysis
_analyze_company_safe called SERP.query() then passed the company name
to analyze_company() which called SERP.query() again — doubling API
usage. Now analyze_company() accepts an optional patents param so callers
can pass pre-fetched results through.
2026-03-24 14:16:49 -04:00
0xWheatyz fa564e5e1e chore: forcing new git commit
Build and Push Docker Images / build-frontend (push) Successful in 1m39s
Build and Push Docker Images / build-api (push) Successful in 3m22s
2026-03-23 17:45:42 -04:00
0xWheatyz 2815deb221 fix(api): configure root_path for OpenAPI docs behind reverse proxy
Build and Push Docker Images / build-api (push) Successful in 11s
Build and Push Docker Images / build-frontend (push) Successful in 29s
Add ROOT_PATH environment variable support so FastAPI generates correct
URLs for Swagger UI when served behind nginx at /api. This fixes the
"invalid version field" error when accessing /api/docs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 11:48:11 -04:00
0xWheatyz 56e8287720 fix(nginx): strip /api/ prefix when proxying to backend
Build and Push Docker Images / build-frontend (push) Successful in 21s
Build and Push Docker Images / build-api (push) Successful in 45s
Add trailing slash to proxy_pass directive so nginx strips the /api/
prefix before forwarding requests to the API container. This fixes
routes like /api/docs being passed as /api/docs instead of /docs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 02:51:28 -04:00
0xWheatyz b969423957 chore(gitlab): cleaned up old git ci files
Build and Push Docker Images / build-api (push) Successful in 9s
Build and Push Docker Images / build-frontend (push) Successful in 31s
2026-03-15 02:40:28 -04:00
0xWheatyz 0dee4c5099 feat(ci): add timestamp-based image tags with commit hash
Build and Push Docker Images / build-frontend (push) Successful in 5s
Build and Push Docker Images / build-api (push) Successful in 18s
Push images with versioned tags in format TIMESTAMP-COMMIT and
frontend-TIMESTAMP-COMMIT for better traceability and rollback support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 02:39:38 -04:00
0xWheatyz 03105a2f87 feat(ci): add timestamp-based image tags with commit hash
Build and Push Docker Images / build-frontend (push) Successful in 6s
Build and Push Docker Images / build-api (push) Successful in 18s
Push images with versioned tags in format TIMESTAMP-COMMIT and
frontend-TIMESTAMP-COMMIT for better traceability and rollback support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 02:35:33 -04:00
0xWheatyz 28e2ded501 feat(frontend): make API endpoint configurable via environment variable
Build and Push Docker Images / build-api (push) Successful in 17s
Build and Push Docker Images / build-frontend (push) Successful in 23s
Use nginx template support to allow API_URL to be passed at container
runtime, enabling the same image to be deployed to different environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 01:09:15 -04:00
0xWheatyz f87572ab7e fix(ci): changed port to 80 as 3000 does not seem to work
Build and Push Docker Images / build-frontend (push) Successful in 1m38s
Build and Push Docker Images / build-api (push) Successful in 1m51s
2026-03-15 00:43:00 -04:00
0xWheatyz 44b6c79713 fix(ci): changed port to 3000 as 80 does not seem to work
Build and Push Docker Images / build-frontend (push) Failing after 5s
Build and Push Docker Images / build-api (push) Failing after 7s
2026-03-15 00:24:13 -04:00
0xWheatyz 13fe383116 fix(ci): use explicit port 80 for insecure registry
Build and Push Docker Images / build-api (push) Successful in 2m50s
Build and Push Docker Images / build-frontend (push) Successful in 1m20s
- Remove http:// prefix from docker login (Docker ignores it)
- Add :80 to registry address so Docker uses HTTP
- Remove redundant daemon.json config (configured at daemon level)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 00:13:46 -04:00
0xWheatyz dee3cbefbd fix(ci): change internal dns name to specify http and the port
Build and Push Docker Images / build-api (push) Failing after 5s
Build and Push Docker Images / build-frontend (push) Failing after 8s
2026-03-15 00:06:42 -04:00
0xWheatyz 6acad4cff7 fix(ci): configure docker to use HTTP for internal registry
Build and Push Docker Images / build-frontend (push) Failing after 8s
Build and Push Docker Images / build-api (push) Failing after 10s
Add insecure-registries configuration to allow HTTP connections
to gitea.gitea.svc.cluster.local instead of HTTPS.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 19:37:14 -04:00
0xWheatyz 45ccd0b4e1 fix(ci): docker internal dns name does not support https
Build and Push Docker Images / build-frontend (push) Failing after 5s
Build and Push Docker Images / build-api (push) Failing after 7s
2026-03-14 19:19:20 -04:00
0xWheatyz d108d4c7ea fix(ci): internal dns name does not support https
Build and Push Docker Images / build-api (push) Failing after 6s
Build and Push Docker Images / build-frontend (push) Failing after 6s
2026-03-14 19:16:45 -04:00
0xWheatyz 068aecce61 fix(ci): moved domain to internal dns name, hopefully runner respects that and this negates the 502 error when too many requests are sent to vps
Build and Push Docker Images / build-frontend (push) Failing after 4s
Build and Push Docker Images / build-api (push) Failing after 6s
2026-03-14 19:15:15 -04:00
0xWheatyz 8790abfbf7 Merge pull request 'rewrite/frontend' (#2) from rewrite/frontend into main
Build and Push Docker Images / build-api (push) Has been cancelled
Build and Push Docker Images / build-frontend (push) Has been cancelled
Reviewed-on: http://10.0.1.10/0xWheatyz/SPARC/pulls/2
2026-03-14 22:02:12 +00:00
0xWheatyz fe0c5ca280 ci: add parallel frontend build job to workflow
Split the single build job into two parallel jobs (build-api and
build-frontend) to enable simultaneous container builds when multiple
runners are available. Frontend images are tagged with frontend- prefix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 17:37:04 -04:00
0xWheatyz ed81ae4569 docs: update documentation for React frontend and cache mode
Update all documentation to reflect recent changes:
- Replace Streamlit dashboard references with React TypeScript dashboard
- Update dashboard port from 8501 to 8080
- Add auth.py and database.py to architecture section
- Change USE_DATABASE terminology to USE_CACHE
- Add JWT_SECRET to environment variables reference
- Document default admin credentials and user seeding

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 14:30:21 -04:00
0xWheatyz ebba983a1d fix(auth): ensure JWT sub claim is RFC 7519 compliant string
- Change TokenPayload.sub type from int to str per JWT RFC 7519
- Add user_id property to TokenPayload for int conversion
- Update token creation to serialize user_id as string
- Update token consumers to use payload.user_id
- Change dashboard port from 3000 to 8080
- Add pydantic[email] for email validation
- Update default admin email to admin@sparc.dev

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 14:22:30 -04:00
0xWheatyz 258b349e98 feat(auth): seed default admin user on database init
Generate a random 16-character password and create an admin user
(admin@sparc.local) during first database initialization. Credentials
are printed to stdout so they can be captured from container logs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:49:44 -04:00
0xWheatyz fc99173028 test: update tests for cache mode terminology
Rename database mode tests to cache mode to reflect new architecture:
- Replace USE_DATABASE with USE_CACHE references
- Update test assertions for cache behavior
- Maintain backward compatibility testing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:41:05 -04:00
0xWheatyz 4405f199ba chore: remove deprecated Streamlit dashboard
Dashboard functionality replaced by React frontend in frontend/

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:41:01 -04:00
0xWheatyz 874f60f0d9 build(docker): update compose for React frontend
Replace Streamlit dashboard service with React frontend:
- Build from frontend/ directory
- Serve on port 3000 via nginx
- Remove volume mount (now using built assets)
- Add JWT_SECRET env var to api service
- Replace USE_DATABASE with USE_CACHE

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:57 -04:00
0xWheatyz cb7d7121c5 feat(frontend): add React dashboard with TypeScript
Add modern React frontend to replace Streamlit dashboard:
- Vite build system with TypeScript
- Tailwind CSS for styling
- Component structure in src/
- Production Dockerfile with nginx
- Development server on port 5173

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:52 -04:00