Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 47cddcbeaf | |||
| 6105ba7793 | |||
| e8cdc089fa |
+30
-9
@@ -1,21 +1,42 @@
|
||||
# SPARC Configuration
|
||||
|
||||
# ---- Application Environment ----
|
||||
# Set to "production" or "staging" in deployed environments.
|
||||
# The API will refuse to start with the default JWT secret unless APP_ENV=development.
|
||||
APP_ENV=development
|
||||
|
||||
# ---- API Keys ----
|
||||
|
||||
# SerpAPI key for patent search
|
||||
API_KEY=your_serpapi_key_here
|
||||
|
||||
# OpenRouter API key for LLM analysis
|
||||
OPENROUTER_API_KEY=your_openrouter_key_here
|
||||
|
||||
# Database configuration
|
||||
# All messages are stored in the database for persistence and caching
|
||||
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc
|
||||
# ---- Database ----
|
||||
|
||||
# Cache configuration
|
||||
# When USE_CACHE=true: check database for cached responses before making API calls
|
||||
# When USE_CACHE=false: always make fresh API calls (still stores results in database)
|
||||
# Default: true
|
||||
USE_CACHE=true
|
||||
# PostgreSQL credentials (used by docker-compose)
|
||||
POSTGRES_USER=postgres
|
||||
POSTGRES_PASSWORD=change-me-to-a-secure-password
|
||||
POSTGRES_DB=sparc
|
||||
|
||||
# JWT Secret for authentication
|
||||
# Full database URL (must match the credentials above)
|
||||
DATABASE_URL=postgresql://postgres:change-me-to-a-secure-password@localhost:5432/sparc
|
||||
|
||||
# ---- Authentication ----
|
||||
|
||||
# JWT Secret for signing tokens
|
||||
# IMPORTANT: Change this to a secure random string in production
|
||||
JWT_SECRET=your-secure-jwt-secret-change-in-production
|
||||
|
||||
# ---- CORS ----
|
||||
|
||||
# Comma-separated list of allowed origins for CORS
|
||||
# Defaults to http://localhost:3000,http://localhost:5173 when unset
|
||||
# CORS_ORIGINS=https://sparc.example.com,https://app.example.com
|
||||
|
||||
# ---- Cache ----
|
||||
|
||||
# When USE_CACHE=true: check database for cached responses before making API calls
|
||||
# When USE_CACHE=false: always make fresh API calls (still stores results in database)
|
||||
USE_CACHE=true
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
# SPARC Roadmap
|
||||
|
||||
Semiconductor Patent & Analytics Report Core -- development priorities.
|
||||
|
||||
## Current State
|
||||
|
||||
SPARC is a patent analysis platform with a working end-to-end pipeline:
|
||||
Python/FastAPI backend, React/TypeScript frontend, PostgreSQL for persistence
|
||||
and caching, Docker Compose for local development, and Gitea Actions CI/CD for
|
||||
image builds. Core features (patent retrieval via SerpAPI, PDF parsing, LLM
|
||||
analysis via OpenRouter/Claude, batch processing, JWT authentication, analytics
|
||||
dashboard) are all implemented and functional.
|
||||
|
||||
---
|
||||
|
||||
## P1 -- High Priority
|
||||
|
||||
These items address correctness, security, and reliability gaps that should be
|
||||
resolved before broader production use.
|
||||
|
||||
### Security hardening
|
||||
|
||||
- **Rotate default JWT secret.** `auth.py` ships a fallback
|
||||
`sparc-secret-key-change-in-production` that will be used if `JWT_SECRET` is
|
||||
unset. Add a startup check that refuses to start with the default secret in
|
||||
non-development environments.
|
||||
- **CORS allow-origins are hardcoded.** `api.py` only permits
|
||||
`localhost:3000` and `localhost:5173`. Make the allowed origins configurable
|
||||
via environment variable so the dashboard works when deployed behind a real
|
||||
domain.
|
||||
- **Database credentials in docker-compose.yml.** The compose file embeds
|
||||
`postgres:postgres` in plain text. Reference a `.env` file or Docker secrets
|
||||
instead.
|
||||
|
||||
### Error handling and resilience
|
||||
|
||||
- **`get_db_client()` in `auth.py` creates a new `DatabaseClient` on every
|
||||
call.** This bypasses the connection pool and can exhaust database
|
||||
connections under load. Refactor to share a single pooled client.
|
||||
- **`_jobs` dict is in-memory only.** Job state is lost on API restart. Persist
|
||||
job status in PostgreSQL or Redis so async batch results survive restarts.
|
||||
- **No rate limiting on auth endpoints.** `/auth/login` and `/auth/register`
|
||||
are unprotected against brute-force or abuse. Add rate limiting middleware.
|
||||
|
||||
### Test coverage for auth and admin
|
||||
|
||||
- The existing API tests (`tests/test_api.py`) bypass authentication entirely.
|
||||
Add tests that exercise the JWT flow: registration, login, protected-route
|
||||
access, token refresh, and admin-only endpoints.
|
||||
|
||||
---
|
||||
|
||||
## P2 -- Medium Priority
|
||||
|
||||
Improvements to usability, performance, and developer experience.
|
||||
|
||||
### Backend
|
||||
|
||||
- **Add structured logging.** Replace `print()` calls throughout `analyzer.py`,
|
||||
`serp_api.py`, and `llm.py` with Python `logging` so log levels and
|
||||
formatting are consistent.
|
||||
- **Make LLM model configurable.** `llm.py` hardcodes
|
||||
`anthropic/claude-3.5-sonnet`. Accept a `MODEL` environment variable to allow
|
||||
switching models without code changes.
|
||||
- **SERP cache TTL is hardcoded to 24 hours.** Expose `SERP_CACHE_TTL_HOURS`
|
||||
as an environment variable in `config.py`.
|
||||
- **Patent PDF storage.** PDFs are saved to a local `patents/` directory. For
|
||||
containerized deployments, consider object storage (S3/MinIO) or at minimum
|
||||
document the volume mount requirement more prominently.
|
||||
- **`analyze_single_patent` assumes local file path.** The method constructs
|
||||
`patents/{patent_id}.pdf` and reads from disk, but does not download the PDF
|
||||
first. Either integrate the download step or document the prerequisite.
|
||||
- **`Patent.patent_id` typed as `int` in `types.py` but used as `str`
|
||||
everywhere.** Fix the type annotation to `str`.
|
||||
|
||||
### Frontend
|
||||
|
||||
- **No loading/error states on several pages.** The Batch and Analytics pages
|
||||
would benefit from skeleton loaders and user-friendly error messages.
|
||||
- **No dark mode.** Tailwind is configured but no dark variant is applied.
|
||||
- **Missing `package-lock.json` or `pnpm-lock.yaml`.** The frontend has no
|
||||
lockfile committed, leading to non-reproducible builds.
|
||||
|
||||
### CI/CD
|
||||
|
||||
- **No test stage in the Gitea Actions workflow.** `build.yaml` builds and
|
||||
pushes images but never runs `pytest`. Add a test job that gates the build.
|
||||
- **No linting or type checking.** Add `ruff` (Python) and `tsc --noEmit`
|
||||
(TypeScript) to CI.
|
||||
|
||||
---
|
||||
|
||||
## P3 -- Nice to Have
|
||||
|
||||
Lower-urgency enhancements and future features.
|
||||
|
||||
- **Export analysis reports.** Allow users to download analysis results as PDF
|
||||
or CSV from the dashboard.
|
||||
- **Comparison view.** Side-by-side comparison of two companies' patent
|
||||
portfolios.
|
||||
- **Scheduled/recurring analysis.** Periodically re-analyze tracked companies
|
||||
and alert on significant changes.
|
||||
- **Webhook/notification support.** Send alerts (Slack, Discord, email) when
|
||||
batch jobs complete or when a company's innovation score changes
|
||||
significantly.
|
||||
- **Multi-model support.** Let users choose between LLM providers per analysis
|
||||
(e.g., GPT-4o, Gemini, Claude) and compare outputs.
|
||||
- **Patent trend charts.** Visualize patent filing frequency and technology
|
||||
category distribution over time in the Analytics page.
|
||||
- **API pagination.** The `/analyze/batch` and `/jobs` endpoints could benefit
|
||||
from cursor-based pagination for large result sets.
|
||||
- **OpenAPI client generation.** Auto-generate the TypeScript API client from
|
||||
the FastAPI OpenAPI spec to keep frontend types in sync.
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure and Deployment
|
||||
|
||||
Kubernetes manifests, Helm charts, and cluster-level concerns (MetalLB,
|
||||
storage, FluxCD sync) are tracked in the
|
||||
[Talos](https://10.0.1.10/leeworks-agents/Talos) repository. File
|
||||
infrastructure-related issues there, not here.
|
||||
+3
-1
@@ -16,6 +16,7 @@ from SPARC.analyzer import CompanyAnalyzer
|
||||
from SPARC.auth import (
|
||||
TokenResponse,
|
||||
UserResponse,
|
||||
check_jwt_secret,
|
||||
create_tokens,
|
||||
decode_token,
|
||||
get_current_admin,
|
||||
@@ -150,6 +151,7 @@ _analyzer: CompanyAnalyzer | None = None
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Initialize resources on startup."""
|
||||
global _analyzer
|
||||
check_jwt_secret()
|
||||
_analyzer = CompanyAnalyzer()
|
||||
yield
|
||||
# Cleanup if needed
|
||||
@@ -167,7 +169,7 @@ app = FastAPI(
|
||||
# Add CORS middleware for React frontend
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:3000", "http://localhost:5173"],
|
||||
allow_origins=config.cors_origins,
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
|
||||
+15
-1
@@ -13,11 +13,25 @@ from SPARC import config
|
||||
from SPARC.database import DatabaseClient
|
||||
|
||||
# JWT Configuration
|
||||
JWT_SECRET = os.getenv("JWT_SECRET", "sparc-secret-key-change-in-production")
|
||||
_DEFAULT_JWT_SECRET = "sparc-secret-key-change-in-production"
|
||||
JWT_SECRET = os.getenv("JWT_SECRET", _DEFAULT_JWT_SECRET)
|
||||
JWT_ALGORITHM = "HS256"
|
||||
ACCESS_TOKEN_EXPIRE_MINUTES = 30
|
||||
REFRESH_TOKEN_EXPIRE_DAYS = 7
|
||||
|
||||
|
||||
def check_jwt_secret() -> None:
|
||||
"""Refuse to start with the default JWT secret in non-development environments.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If JWT_SECRET is the default value and APP_ENV is not 'development'.
|
||||
"""
|
||||
if JWT_SECRET == _DEFAULT_JWT_SECRET and config.app_env != "development":
|
||||
raise RuntimeError(
|
||||
f"FATAL: JWT_SECRET is set to the default value and APP_ENV={config.app_env!r}. "
|
||||
"Set a secure JWT_SECRET environment variable before running in non-development environments."
|
||||
)
|
||||
|
||||
security = HTTPBearer()
|
||||
|
||||
|
||||
|
||||
@@ -33,3 +33,16 @@ patent_thread_workers = int(os.getenv("PATENT_THREAD_WORKERS", "5"))
|
||||
# Root path for running behind a reverse proxy (e.g., "/api" when served at /api/)
|
||||
# This ensures OpenAPI docs work correctly when accessed via the proxy
|
||||
root_path = os.getenv("ROOT_PATH", "")
|
||||
|
||||
# Application environment: "development", "staging", or "production"
|
||||
# Used for safety checks (e.g., refusing default JWT secret in production)
|
||||
app_env = os.getenv("APP_ENV", "development")
|
||||
|
||||
# CORS allowed origins (comma-separated)
|
||||
# Defaults to localhost dev origins when unset
|
||||
_cors_origins_raw = os.getenv("CORS_ORIGINS", "")
|
||||
cors_origins: list[str] = (
|
||||
[o.strip() for o in _cors_origins_raw.split(",") if o.strip()]
|
||||
if _cors_origins_raw
|
||||
else ["http://localhost:3000", "http://localhost:5173"]
|
||||
)
|
||||
|
||||
+8
-6
@@ -3,15 +3,15 @@ services:
|
||||
image: postgres:16-alpine
|
||||
container_name: sparc-postgres
|
||||
environment:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_DB: sparc
|
||||
POSTGRES_USER: ${POSTGRES_USER}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
POSTGRES_DB: ${POSTGRES_DB}
|
||||
ports:
|
||||
- "5432:5432"
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
@@ -22,7 +22,7 @@ services:
|
||||
container_name: sparc-init-db
|
||||
command: python scripts/init_database.py
|
||||
environment:
|
||||
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/sparc
|
||||
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
@@ -35,9 +35,11 @@ services:
|
||||
environment:
|
||||
API_KEY: ${API_KEY}
|
||||
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
|
||||
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/sparc
|
||||
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
|
||||
USE_CACHE: "true"
|
||||
JWT_SECRET: ${JWT_SECRET:-sparc-secret-key-change-in-production}
|
||||
CORS_ORIGINS: ${CORS_ORIGINS:-}
|
||||
APP_ENV: ${APP_ENV:-development}
|
||||
ROOT_PATH: /api
|
||||
ports:
|
||||
- "8000:8000"
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Tests for security hardening: JWT secret startup check, CORS config, credential handling."""
|
||||
|
||||
import os
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
class TestJWTSecretStartupCheck:
|
||||
"""Test the startup guard that refuses default JWT secret in non-dev environments."""
|
||||
|
||||
def test_default_secret_in_production_raises(self):
|
||||
"""Starting with default secret and APP_ENV=production must raise RuntimeError."""
|
||||
with patch.dict(os.environ, {"APP_ENV": "production"}):
|
||||
# Reload config to pick up the new APP_ENV
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
from SPARC.auth import _DEFAULT_JWT_SECRET, check_jwt_secret
|
||||
# Patch JWT_SECRET to the default
|
||||
with patch("SPARC.auth.JWT_SECRET", _DEFAULT_JWT_SECRET):
|
||||
with pytest.raises(RuntimeError, match="FATAL.*JWT_SECRET"):
|
||||
check_jwt_secret()
|
||||
|
||||
# Restore config
|
||||
with patch.dict(os.environ, {"APP_ENV": "development"}):
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
def test_default_secret_in_development_succeeds(self):
|
||||
"""Starting with default secret and APP_ENV=development must not raise."""
|
||||
with patch.dict(os.environ, {"APP_ENV": "development"}):
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
from SPARC.auth import _DEFAULT_JWT_SECRET, check_jwt_secret
|
||||
with patch("SPARC.auth.JWT_SECRET", _DEFAULT_JWT_SECRET):
|
||||
# Should not raise
|
||||
check_jwt_secret()
|
||||
|
||||
# Restore
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
def test_custom_secret_in_production_succeeds(self):
|
||||
"""Starting with a custom secret in production must not raise."""
|
||||
with patch.dict(os.environ, {"APP_ENV": "production"}):
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
from SPARC.auth import check_jwt_secret
|
||||
with patch("SPARC.auth.JWT_SECRET", "my-secure-random-secret-abc123"):
|
||||
# Should not raise
|
||||
check_jwt_secret()
|
||||
|
||||
with patch.dict(os.environ, {"APP_ENV": "development"}):
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
def test_default_secret_unset_env_succeeds(self):
|
||||
"""When APP_ENV is unset (defaults to development), default secret is allowed."""
|
||||
with patch.dict(os.environ, {}, clear=False):
|
||||
# Remove APP_ENV if present
|
||||
env = os.environ.copy()
|
||||
env.pop("APP_ENV", None)
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
from SPARC.auth import _DEFAULT_JWT_SECRET, check_jwt_secret
|
||||
with patch("SPARC.auth.JWT_SECRET", _DEFAULT_JWT_SECRET):
|
||||
# Should not raise (defaults to development)
|
||||
check_jwt_secret()
|
||||
|
||||
with patch.dict(os.environ, {"APP_ENV": "development"}):
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
|
||||
class TestCORSConfig:
|
||||
"""Test that CORS origins are configurable via environment variable."""
|
||||
|
||||
def test_default_cors_origins(self):
|
||||
"""When CORS_ORIGINS is unset, defaults to localhost origins."""
|
||||
with patch.dict(os.environ, {"CORS_ORIGINS": ""}):
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
assert SPARC.config.cors_origins == [
|
||||
"http://localhost:3000",
|
||||
"http://localhost:5173",
|
||||
]
|
||||
|
||||
def test_custom_cors_origins(self):
|
||||
"""Setting CORS_ORIGINS configures allowed origins."""
|
||||
with patch.dict(os.environ, {"CORS_ORIGINS": "https://sparc.example.com,https://app.example.com"}):
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
assert SPARC.config.cors_origins == [
|
||||
"https://sparc.example.com",
|
||||
"https://app.example.com",
|
||||
]
|
||||
# Restore
|
||||
with patch.dict(os.environ, {"CORS_ORIGINS": ""}):
|
||||
importlib.reload(SPARC.config)
|
||||
|
||||
def test_single_cors_origin(self):
|
||||
"""A single origin without comma works correctly."""
|
||||
with patch.dict(os.environ, {"CORS_ORIGINS": "https://sparc.example.com"}):
|
||||
import importlib
|
||||
import SPARC.config
|
||||
importlib.reload(SPARC.config)
|
||||
assert SPARC.config.cors_origins == ["https://sparc.example.com"]
|
||||
with patch.dict(os.environ, {"CORS_ORIGINS": ""}):
|
||||
importlib.reload(SPARC.config)
|
||||
Reference in New Issue
Block a user