Compare commits

...

35 Commits

Author SHA1 Message Date
0xWheatyz 9c971dac72 fix(analyzer): route _analyze_company_safe through cache-aware path
Build and Push Docker Images / build-api (push) Successful in 2m19s
Build and Push Docker Images / build-frontend (push) Successful in 1m49s
_analyze_company_safe was calling SERP.query directly, bypassing the
SERP query cache in analyze_company. Now delegates fully to
analyze_company() and reads patent_count from the serp_queries cache.
2026-03-24 15:02:19 -04:00
0xWheatyz 6f0b448044 test(analyzer,serp): add tests for caching, single query, and parallel processing
- Add TestSingleQueryBugFix: verify SERP.query called once per analysis
- Add TestPatentCaching: DB cache hit/miss, SERP query cache hit/miss
- Add TestDynamicDateRange: rolling window, days_back param
- Add TestFilesystemPDFCaching: skip download, redownload empty files
- Add autouse mock_db fixture to prevent real DB connections in all tests
2026-03-24 14:39:09 -04:00
0xWheatyz 1a297eb60b feat(analyzer): integrate DB patent and SERP query caching
Before querying SERP API, check serp_queries cache (24h TTL). Before
downloading/parsing each patent, check patents table for cached
minimized_content. Store results after processing so repeated analyses
skip all network I/O and PDF parsing entirely.
2026-03-24 14:35:24 -04:00
0xWheatyz 3154f6b732 feat(database): add patent/serp caching tables and connection pooling
- Add patents table (patent_id PK, raw_sections JSONB, minimized_content)
- Add serp_queries table (query_hash unique, result_patent_ids, expires_at)
- Add cache methods: get/store_patent, get/store_serp_query
- Replace single connection with ThreadedConnectionPool (min=2, max=10)
- Add get_conn() context manager for thread-safe connection checkout
- Legacy single-connection path preserved for backwards compatibility
2026-03-24 14:34:33 -04:00
0xWheatyz b9bb3dc1cd perf(analyzer): parallelize patent download/parse/minimize with threads
Replace the sequential per-patent loop with a ThreadPoolExecutor
(workers controlled by PATENT_THREAD_WORKERS config). Each patent is
processed independently in _process_single_patent, which is thread-safe
since SERP methods are stateless and operate on separate files.
2026-03-24 14:32:23 -04:00
0xWheatyz 90f9cfc826 fix(serp): replace hardcoded date range with rolling window
The SERP query had a frozen date range (Oct-Nov 2025) that returned
stale patents. Now computes a rolling window from config
(PATENT_SEARCH_DAYS, default 90 days). Also adds filesystem-level PDF
caching to skip re-downloading existing patent PDFs, and adds
PATENT_THREAD_WORKERS config for upcoming parallel processing.
2026-03-24 14:31:43 -04:00
0xWheatyz d387bbbdf3 fix(analyzer): eliminate double SERP.query() call per company analysis
_analyze_company_safe called SERP.query() then passed the company name
to analyze_company() which called SERP.query() again — doubling API
usage. Now analyze_company() accepts an optional patents param so callers
can pass pre-fetched results through.
2026-03-24 14:16:49 -04:00
0xWheatyz fa564e5e1e chore: forcing new git commit
Build and Push Docker Images / build-frontend (push) Successful in 1m39s
Build and Push Docker Images / build-api (push) Successful in 3m22s
2026-03-23 17:45:42 -04:00
0xWheatyz 2815deb221 fix(api): configure root_path for OpenAPI docs behind reverse proxy
Build and Push Docker Images / build-api (push) Successful in 11s
Build and Push Docker Images / build-frontend (push) Successful in 29s
Add ROOT_PATH environment variable support so FastAPI generates correct
URLs for Swagger UI when served behind nginx at /api. This fixes the
"invalid version field" error when accessing /api/docs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 11:48:11 -04:00
0xWheatyz 56e8287720 fix(nginx): strip /api/ prefix when proxying to backend
Build and Push Docker Images / build-frontend (push) Successful in 21s
Build and Push Docker Images / build-api (push) Successful in 45s
Add trailing slash to proxy_pass directive so nginx strips the /api/
prefix before forwarding requests to the API container. This fixes
routes like /api/docs being passed as /api/docs instead of /docs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 02:51:28 -04:00
0xWheatyz b969423957 chore(gitlab): cleaned up old git ci files
Build and Push Docker Images / build-api (push) Successful in 9s
Build and Push Docker Images / build-frontend (push) Successful in 31s
2026-03-15 02:40:28 -04:00
0xWheatyz 0dee4c5099 feat(ci): add timestamp-based image tags with commit hash
Build and Push Docker Images / build-frontend (push) Successful in 5s
Build and Push Docker Images / build-api (push) Successful in 18s
Push images with versioned tags in format TIMESTAMP-COMMIT and
frontend-TIMESTAMP-COMMIT for better traceability and rollback support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 02:39:38 -04:00
0xWheatyz 03105a2f87 feat(ci): add timestamp-based image tags with commit hash
Build and Push Docker Images / build-frontend (push) Successful in 6s
Build and Push Docker Images / build-api (push) Successful in 18s
Push images with versioned tags in format TIMESTAMP-COMMIT and
frontend-TIMESTAMP-COMMIT for better traceability and rollback support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 02:35:33 -04:00
0xWheatyz 28e2ded501 feat(frontend): make API endpoint configurable via environment variable
Build and Push Docker Images / build-api (push) Successful in 17s
Build and Push Docker Images / build-frontend (push) Successful in 23s
Use nginx template support to allow API_URL to be passed at container
runtime, enabling the same image to be deployed to different environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 01:09:15 -04:00
0xWheatyz f87572ab7e fix(ci): changed port to 80 as 3000 does not seem to work
Build and Push Docker Images / build-frontend (push) Successful in 1m38s
Build and Push Docker Images / build-api (push) Successful in 1m51s
2026-03-15 00:43:00 -04:00
0xWheatyz 44b6c79713 fix(ci): changed port to 3000 as 80 does not seem to work
Build and Push Docker Images / build-frontend (push) Failing after 5s
Build and Push Docker Images / build-api (push) Failing after 7s
2026-03-15 00:24:13 -04:00
0xWheatyz 13fe383116 fix(ci): use explicit port 80 for insecure registry
Build and Push Docker Images / build-api (push) Successful in 2m50s
Build and Push Docker Images / build-frontend (push) Successful in 1m20s
- Remove http:// prefix from docker login (Docker ignores it)
- Add :80 to registry address so Docker uses HTTP
- Remove redundant daemon.json config (configured at daemon level)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-15 00:13:46 -04:00
0xWheatyz dee3cbefbd fix(ci): change internal dns name to specify http and the port
Build and Push Docker Images / build-api (push) Failing after 5s
Build and Push Docker Images / build-frontend (push) Failing after 8s
2026-03-15 00:06:42 -04:00
0xWheatyz 6acad4cff7 fix(ci): configure docker to use HTTP for internal registry
Build and Push Docker Images / build-frontend (push) Failing after 8s
Build and Push Docker Images / build-api (push) Failing after 10s
Add insecure-registries configuration to allow HTTP connections
to gitea.gitea.svc.cluster.local instead of HTTPS.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 19:37:14 -04:00
0xWheatyz 45ccd0b4e1 fix(ci): docker internal dns name does not support https
Build and Push Docker Images / build-frontend (push) Failing after 5s
Build and Push Docker Images / build-api (push) Failing after 7s
2026-03-14 19:19:20 -04:00
0xWheatyz d108d4c7ea fix(ci): internal dns name does not support https
Build and Push Docker Images / build-api (push) Failing after 6s
Build and Push Docker Images / build-frontend (push) Failing after 6s
2026-03-14 19:16:45 -04:00
0xWheatyz 068aecce61 fix(ci): moved domain to internal dns name, hopefully runner respects that and this negates the 502 error when too many requests are sent to vps
Build and Push Docker Images / build-frontend (push) Failing after 4s
Build and Push Docker Images / build-api (push) Failing after 6s
2026-03-14 19:15:15 -04:00
0xWheatyz 8790abfbf7 Merge pull request 'rewrite/frontend' (#2) from rewrite/frontend into main
Build and Push Docker Images / build-api (push) Has been cancelled
Build and Push Docker Images / build-frontend (push) Has been cancelled
Reviewed-on: http://10.0.1.10/0xWheatyz/SPARC/pulls/2
2026-03-14 22:02:12 +00:00
0xWheatyz fe0c5ca280 ci: add parallel frontend build job to workflow
Split the single build job into two parallel jobs (build-api and
build-frontend) to enable simultaneous container builds when multiple
runners are available. Frontend images are tagged with frontend- prefix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 17:37:04 -04:00
0xWheatyz ed81ae4569 docs: update documentation for React frontend and cache mode
Update all documentation to reflect recent changes:
- Replace Streamlit dashboard references with React TypeScript dashboard
- Update dashboard port from 8501 to 8080
- Add auth.py and database.py to architecture section
- Change USE_DATABASE terminology to USE_CACHE
- Add JWT_SECRET to environment variables reference
- Document default admin credentials and user seeding

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 14:30:21 -04:00
0xWheatyz ebba983a1d fix(auth): ensure JWT sub claim is RFC 7519 compliant string
- Change TokenPayload.sub type from int to str per JWT RFC 7519
- Add user_id property to TokenPayload for int conversion
- Update token creation to serialize user_id as string
- Update token consumers to use payload.user_id
- Change dashboard port from 3000 to 8080
- Add pydantic[email] for email validation
- Update default admin email to admin@sparc.dev

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 14:22:30 -04:00
0xWheatyz 258b349e98 feat(auth): seed default admin user on database init
Generate a random 16-character password and create an admin user
(admin@sparc.local) during first database initialization. Credentials
are printed to stdout so they can be captured from container logs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:49:44 -04:00
0xWheatyz fc99173028 test: update tests for cache mode terminology
Rename database mode tests to cache mode to reflect new architecture:
- Replace USE_DATABASE with USE_CACHE references
- Update test assertions for cache behavior
- Maintain backward compatibility testing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:41:05 -04:00
0xWheatyz 4405f199ba chore: remove deprecated Streamlit dashboard
Dashboard functionality replaced by React frontend in frontend/

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:41:01 -04:00
0xWheatyz 874f60f0d9 build(docker): update compose for React frontend
Replace Streamlit dashboard service with React frontend:
- Build from frontend/ directory
- Serve on port 3000 via nginx
- Remove volume mount (now using built assets)
- Add JWT_SECRET env var to api service
- Replace USE_DATABASE with USE_CACHE

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:57 -04:00
0xWheatyz cb7d7121c5 feat(frontend): add React dashboard with TypeScript
Add modern React frontend to replace Streamlit dashboard:
- Vite build system with TypeScript
- Tailwind CSS for styling
- Component structure in src/
- Production Dockerfile with nginx
- Development server on port 5173

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:52 -04:00
0xWheatyz 9c98b948d3 feat(api): add authentication and analytics endpoints
Protect all analysis endpoints with JWT authentication:
- Require valid access token for analysis operations
- Add CORS middleware for React frontend (localhost:3000, 5173)

Add auth endpoints:
- POST /auth/register - user registration (first user becomes admin)
- POST /auth/login - JWT token issuance
- POST /auth/refresh - token refresh
- GET /auth/me - current user info

Add admin endpoints:
- GET /admin/users - list all users
- PATCH /admin/users/{id}/role - update user role
- DELETE /admin/users/{id} - delete user

Add analytics endpoint:
- GET /analytics - usage statistics by company and type

Update .env.example with USE_CACHE and JWT_SECRET config

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:48 -04:00
0xWheatyz af52107ed8 feat(backend): add response caching and user management
Replace USE_DATABASE toggle with USE_CACHE for smarter LLM response handling:
- Add prompt hashing for efficient cache lookups
- Cache API responses in database to reduce token usage
- Always store responses for analytics (cache or fresh)

Add user authentication infrastructure:
- User table with bcrypt password hashing
- CRUD operations for user management
- Role-based access control (admin/user)

Dependencies: add bcrypt and PyJWT for auth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:34 -04:00
0xWheatyz 0107691c90 feat(auth): add JWT authentication module
Add standalone auth module with JWT token handling:
- Access and refresh token generation/validation
- FastAPI dependency functions for route protection
- Admin role verification for privileged endpoints
- Secure password handling integration with database

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-14 13:40:28 -04:00
0xWheatyz 3424384088 feat: update Docker config to run API and dashboard services
Build and Push Docker Image / build-and-push (push) Has been cancelled
- Switch from Alpine to Debian slim for better package compatibility
- Add system dependencies for pdfplumber and psycopg2
- Configure separate services for API (port 8000) and dashboard (port 8501)
- Add automatic database initialization via init-db service
- Update documentation with simplified Docker setup
- Remove need for separate docker-compose.prod.yml

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-13 15:49:59 -04:00
48 changed files with 3892 additions and 1189 deletions
+11 -6
View File
@@ -6,11 +6,16 @@ API_KEY=your_serpapi_key_here
# OpenRouter API key for LLM analysis # OpenRouter API key for LLM analysis
OPENROUTER_API_KEY=your_openrouter_key_here OPENROUTER_API_KEY=your_openrouter_key_here
# Database configuration (for docker-compose setup) # Database configuration
# All messages are stored in the database for persistence and caching
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc
# Toggle between database mode and API mode # Cache configuration
# When USE_DATABASE=true: stores all messages in database instead of sending to OpenRouter # When USE_CACHE=true: check database for cached responses before making API calls
# When USE_DATABASE=false: sends messages to OpenRouter API as normal # When USE_CACHE=false: always make fresh API calls (still stores results in database)
# Default: false # Default: true
USE_DATABASE=false USE_CACHE=true
# JWT Secret for authentication
# IMPORTANT: Change this to a secure random string in production
JWT_SECRET=your-secure-jwt-secret-change-in-production
+82 -19
View File
@@ -1,4 +1,4 @@
name: Build and Push Docker Image name: Build and Push Docker Images
on: on:
push: push:
@@ -9,7 +9,7 @@ on:
workflow_dispatch: workflow_dispatch:
jobs: jobs:
build-and-push: build-api:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Install dependencies - name: Install dependencies
@@ -20,43 +20,36 @@ jobs:
- name: Checkout code - name: Checkout code
shell: sh shell: sh
run: | run: |
git clone https://gitea.leeworks.dev/${{ gitea.repository }}.git . git clone http://gitea.gitea.svc.cluster.local/${{ gitea.repository }}.git .
git checkout ${{ gitea.sha }} git checkout ${{ gitea.sha }}
- name: Determine image tags - name: Determine image tags
id: tags id: tags
shell: sh shell: sh
run: | run: |
REGISTRY="gitea.leeworks.dev" REGISTRY="gitea.gitea.svc.cluster.local:80"
REPO_OWNER="${{ gitea.repository_owner }}" REPO_OWNER="${{ gitea.repository_owner }}"
REPO_NAME="${{ gitea.repository }}" REPO_NAME="${{ gitea.repository }}"
# Extract repository name without owner
REPO_NAME_ONLY=$(echo "$REPO_NAME" | cut -d'/' -f2) REPO_NAME_ONLY=$(echo "$REPO_NAME" | cut -d'/' -f2)
# Convert to lowercase for Docker registry compatibility
REPO_OWNER_LOWER=$(echo "$REPO_OWNER" | tr '[:upper:]' '[:lower:]') REPO_OWNER_LOWER=$(echo "$REPO_OWNER" | tr '[:upper:]' '[:lower:]')
REPO_NAME_LOWER=$(echo "$REPO_NAME_ONLY" | tr '[:upper:]' '[:lower:]') REPO_NAME_LOWER=$(echo "$REPO_NAME_ONLY" | tr '[:upper:]' '[:lower:]')
# Base image path
IMAGE_BASE="${REGISTRY}/${REPO_OWNER_LOWER}/${REPO_NAME_LOWER}" IMAGE_BASE="${REGISTRY}/${REPO_OWNER_LOWER}/${REPO_NAME_LOWER}"
# Determine tag based on ref
case "${{ gitea.ref }}" in case "${{ gitea.ref }}" in
refs/tags/*) refs/tags/*)
# Tag push - use the tag name
TAG_NAME="${{ gitea.ref_name }}" TAG_NAME="${{ gitea.ref_name }}"
echo "IMAGE_TAG=${IMAGE_BASE}:${TAG_NAME}" >> $GITHUB_OUTPUT echo "IMAGE_TAG=${IMAGE_BASE}:${TAG_NAME}" >> $GITHUB_OUTPUT
echo "PUSH_LATEST=true" >> $GITHUB_OUTPUT echo "PUSH_LATEST=true" >> $GITHUB_OUTPUT
;; ;;
refs/heads/main) refs/heads/main)
# Main branch - use commit SHA (shortened to 7 chars) and latest TIMESTAMP=$(date -u +%Y%m%d%H%M%S)
SHORT_SHA=$(echo "${{ gitea.sha }}" | cut -c1-7) SHORT_SHA=$(echo "${{ gitea.sha }}" | cut -c1-7)
echo "IMAGE_TAG=${IMAGE_BASE}:${SHORT_SHA}" >> $GITHUB_OUTPUT echo "IMAGE_TAG=${IMAGE_BASE}:${TIMESTAMP}-${SHORT_SHA}" >> $GITHUB_OUTPUT
echo "PUSH_LATEST=true" >> $GITHUB_OUTPUT echo "PUSH_LATEST=true" >> $GITHUB_OUTPUT
;; ;;
*) *)
# Other branches - use branch name
BRANCH_TAG=$(echo "${{ gitea.ref_name }}" | sed 's/\//-/g') BRANCH_TAG=$(echo "${{ gitea.ref_name }}" | sed 's/\//-/g')
echo "IMAGE_TAG=${IMAGE_BASE}:${BRANCH_TAG}" >> $GITHUB_OUTPUT echo "IMAGE_TAG=${IMAGE_BASE}:${BRANCH_TAG}" >> $GITHUB_OUTPUT
echo "PUSH_LATEST=false" >> $GITHUB_OUTPUT echo "PUSH_LATEST=false" >> $GITHUB_OUTPUT
@@ -68,15 +61,15 @@ jobs:
- name: Login to registry - name: Login to registry
shell: sh shell: sh
run: | run: |
echo "${{ secrets.PERSONAL_TOKEN }}" | docker login gitea.leeworks.dev -u "${{ gitea.actor }}" --password-stdin echo "${{ secrets.PERSONAL_TOKEN }}" | docker login gitea.gitea.svc.cluster.local:80 -u "${{ gitea.actor }}" --password-stdin
- name: Build and push with Docker - name: Build and push API image
shell: sh shell: sh
run: | run: |
echo "Building image..." echo "Building API image..."
docker build -t ${{ steps.tags.outputs.IMAGE_TAG }} . docker build -t ${{ steps.tags.outputs.IMAGE_TAG }} .
echo "Pushing image..." echo "Pushing API image..."
docker push ${{ steps.tags.outputs.IMAGE_TAG }} docker push ${{ steps.tags.outputs.IMAGE_TAG }}
if [ "${{ steps.tags.outputs.PUSH_LATEST }}" = "true" ]; then if [ "${{ steps.tags.outputs.PUSH_LATEST }}" = "true" ]; then
@@ -85,5 +78,75 @@ jobs:
docker push ${{ steps.tags.outputs.IMAGE_LATEST }} docker push ${{ steps.tags.outputs.IMAGE_LATEST }}
fi fi
echo "Build and push completed successfully!" echo "API image available at ${{ steps.tags.outputs.IMAGE_TAG }}"
echo "Image available at ${{ steps.tags.outputs.IMAGE_TAG }}"
build-frontend:
runs-on: ubuntu-latest
steps:
- name: Install dependencies
shell: sh
run: |
apk add --no-cache git docker-cli
- name: Checkout code
shell: sh
run: |
git clone http://gitea.gitea.svc.cluster.local/${{ gitea.repository }}.git .
git checkout ${{ gitea.sha }}
- name: Determine image tags
id: tags
shell: sh
run: |
REGISTRY="gitea.gitea.svc.cluster.local:80"
REPO_OWNER="${{ gitea.repository_owner }}"
REPO_NAME="${{ gitea.repository }}"
REPO_NAME_ONLY=$(echo "$REPO_NAME" | cut -d'/' -f2)
REPO_OWNER_LOWER=$(echo "$REPO_OWNER" | tr '[:upper:]' '[:lower:]')
REPO_NAME_LOWER=$(echo "$REPO_NAME_ONLY" | tr '[:upper:]' '[:lower:]')
IMAGE_BASE="${REGISTRY}/${REPO_OWNER_LOWER}/${REPO_NAME_LOWER}"
case "${{ gitea.ref }}" in
refs/tags/*)
TAG_NAME="${{ gitea.ref_name }}"
echo "IMAGE_TAG=${IMAGE_BASE}:frontend-${TAG_NAME}" >> $GITHUB_OUTPUT
echo "PUSH_LATEST=true" >> $GITHUB_OUTPUT
;;
refs/heads/main)
TIMESTAMP=$(date -u +%Y%m%d%H%M%S)
SHORT_SHA=$(echo "${{ gitea.sha }}" | cut -c1-7)
echo "IMAGE_TAG=${IMAGE_BASE}:frontend-${TIMESTAMP}-${SHORT_SHA}" >> $GITHUB_OUTPUT
echo "PUSH_LATEST=true" >> $GITHUB_OUTPUT
;;
*)
BRANCH_TAG=$(echo "${{ gitea.ref_name }}" | sed 's/\//-/g')
echo "IMAGE_TAG=${IMAGE_BASE}:frontend-${BRANCH_TAG}" >> $GITHUB_OUTPUT
echo "PUSH_LATEST=false" >> $GITHUB_OUTPUT
;;
esac
echo "IMAGE_LATEST=${IMAGE_BASE}:frontend-latest" >> $GITHUB_OUTPUT
- name: Login to registry
shell: sh
run: |
echo "${{ secrets.PERSONAL_TOKEN }}" | docker login gitea.gitea.svc.cluster.local:80 -u "${{ gitea.actor }}" --password-stdin
- name: Build and push frontend image
shell: sh
run: |
echo "Building frontend image..."
docker build -t ${{ steps.tags.outputs.IMAGE_TAG }} ./frontend
echo "Pushing frontend image..."
docker push ${{ steps.tags.outputs.IMAGE_TAG }}
if [ "${{ steps.tags.outputs.PUSH_LATEST }}" = "true" ]; then
echo "Tagging and pushing frontend-latest..."
docker tag ${{ steps.tags.outputs.IMAGE_TAG }} ${{ steps.tags.outputs.IMAGE_LATEST }}
docker push ${{ steps.tags.outputs.IMAGE_LATEST }}
fi
echo "Frontend image available at ${{ steps.tags.outputs.IMAGE_TAG }}"
-33
View File
@@ -1,33 +0,0 @@
stages:
- build
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
LATEST_TAG: $CI_REGISTRY_IMAGE:latest
build-and-push:
stage: build
image: docker:24-cli
services:
- docker:24-dind
before_script:
- echo "Logging into GitLab Container Registry..."
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- echo "Building Docker image..."
- docker build -t $IMAGE_TAG -t $LATEST_TAG .
- echo "Pushing Docker image to registry..."
- docker push $IMAGE_TAG
- docker push $LATEST_TAG
- echo "Build and push completed successfully!"
- echo "Image available at $IMAGE_TAG"
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: always
- if: $CI_COMMIT_TAG
when: always
- when: manual
tags:
- docker
View File
+15 -2
View File
@@ -1,12 +1,25 @@
FROM python:3.14-alpine3.23 FROM python:3.12-slim
WORKDIR /app WORKDIR /app
# Install system dependencies for pdfplumber and psycopg2
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt . COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt RUN pip install --no-cache-dir -r requirements.txt
COPY . . COPY . .
CMD ["python3", "main.py"] # Create patents directory for PDF storage
RUN mkdir -p /app/patents
# Expose ports for API and Dashboard
EXPOSE 8000 8501
# Default command runs the API (can be overridden in docker-compose)
CMD ["uvicorn", "SPARC.api:app", "--host", "0.0.0.0", "--port", "8000"]
+32 -11
View File
@@ -17,7 +17,7 @@ SPARC automatically collects, parses, and analyzes patents from companies to pro
- **Portfolio Analysis**: Evaluates multiple patents holistically for comprehensive insights - **Portfolio Analysis**: Evaluates multiple patents holistically for comprehensive insights
- **Batch Processing**: Analyze multiple companies concurrently with progress tracking - **Batch Processing**: Analyze multiple companies concurrently with progress tracking
- **REST API**: FastAPI web service with async job support - **REST API**: FastAPI web service with async job support
- **Dashboard**: Interactive Streamlit visualization dashboard - **Dashboard**: React TypeScript web dashboard with authentication
- **Robust Testing**: 40 tests covering all major functionality - **Robust Testing**: 40 tests covering all major functionality
## Architecture ## Architecture
@@ -27,14 +27,34 @@ SPARC/
├── serp_api.py # Patent retrieval and PDF parsing ├── serp_api.py # Patent retrieval and PDF parsing
├── llm.py # Claude AI integration via OpenRouter ├── llm.py # Claude AI integration via OpenRouter
├── analyzer.py # High-level orchestration ├── analyzer.py # High-level orchestration
├── api.py # FastAPI web service ├── api.py # FastAPI web service with auth endpoints
├── auth.py # JWT authentication module
├── database.py # PostgreSQL storage with caching
├── types.py # Data models ├── types.py # Data models
└── config.py # Environment configuration └── config.py # Environment configuration
``` ```
## Installation ## Installation
### NixOS (Recommended) ### Docker (Recommended)
```bash
# Clone and configure
git clone <repository-url>
cd SPARC
cp .env.example .env
# Edit .env with your API keys
# Start all services (API, Dashboard, PostgreSQL)
docker-compose up -d
# Access the services
# - API: http://localhost:8000
# - Dashboard: http://localhost:8080
# - API Docs: http://localhost:8000/docs
```
### NixOS
```bash ```bash
nix develop nix develop
@@ -168,21 +188,22 @@ curl -X POST http://localhost:8000/analyze/batch/async \
-d '{"companies": ["nvidia", "amd", "intel", "qualcomm"]}' -d '{"companies": ["nvidia", "amd", "intel", "qualcomm"]}'
``` ```
### Visualization Dashboard ### Web Dashboard
Launch the interactive Streamlit dashboard: The React dashboard is included in Docker Compose:
```bash ```bash
streamlit run dashboard.py docker-compose up -d
``` ```
Dashboard features: Dashboard features:
- **Authentication**: User registration, login, and JWT-based sessions
- **Company Analysis**: Analyze individual companies with real-time results - **Company Analysis**: Analyze individual companies with real-time results
- **Batch Analysis**: Process multiple companies with progress tracking and charts - **Batch Analysis**: Process multiple companies with progress tracking
- **Analytics**: View historical analysis data and trends (requires database mode) - **Analytics**: View historical analysis data and trends
- **System Status**: Monitor database and analyzer health - **Admin Panel**: User management for administrators
The dashboard runs at `http://localhost:8501` by default. The dashboard runs at `http://localhost:8080` when using Docker Compose.
## Running Tests ## Running Tests
@@ -262,4 +283,4 @@ For open source projects, say how it is licensed.
Core functionality complete. Ready for production use with API keys configured. Core functionality complete. Ready for production use with API keys configured.
Next steps: API wrapper, containerization, and multi-company support. All major features implemented: REST API, React dashboard with authentication, Docker containerization, database storage with caching, and multi-company batch processing.
+86 -24
View File
@@ -4,26 +4,33 @@ This module ties together patent retrieval, parsing, and LLM analysis
to provide company performance estimation based on patent portfolios. to provide company performance estimation based on patent portfolios.
""" """
import hashlib
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
from typing import Callable from typing import Callable
from SPARC import config
from SPARC.database import DatabaseClient
from SPARC.serp_api import SERP from SPARC.serp_api import SERP
from SPARC.llm import LLMAnalyzer from SPARC.llm import LLMAnalyzer
from SPARC.types import Patent, CompanyAnalysisResult, BatchAnalysisResult from SPARC.types import Patent, Patents, CompanyAnalysisResult, BatchAnalysisResult
class CompanyAnalyzer: class CompanyAnalyzer:
"""Orchestrates end-to-end company performance analysis via patents.""" """Orchestrates end-to-end company performance analysis via patents."""
def __init__(self, openrouter_api_key: str | None = None): def __init__(self, openrouter_api_key: str | None = None, db_client: DatabaseClient | None = None):
"""Initialize the company analyzer. """Initialize the company analyzer.
Args: Args:
openrouter_api_key: Optional OpenRouter API key. If None, loads from config. openrouter_api_key: Optional OpenRouter API key. If None, loads from config.
db_client: Optional DatabaseClient for patent caching. Created automatically if None.
""" """
self.llm_analyzer = LLMAnalyzer(api_key=openrouter_api_key) self.llm_analyzer = LLMAnalyzer(api_key=openrouter_api_key)
self.db = db_client or DatabaseClient(config.database_url)
self.db.connect()
self.db.initialize_schema()
def analyze_company(self, company_name: str) -> str: def analyze_company(self, company_name: str, patents: "Patents | None" = None) -> str:
"""Analyze a company's performance based on their patent portfolio. """Analyze a company's performance based on their patent portfolio.
This is the main entry point that orchestrates the full pipeline: This is the main entry point that orchestrates the full pipeline:
@@ -35,40 +42,52 @@ class CompanyAnalyzer:
Args: Args:
company_name: Name of the company to analyze company_name: Name of the company to analyze
patents: Optional pre-fetched Patents result to avoid duplicate API calls
Returns: Returns:
Comprehensive analysis of company's innovation and performance outlook Comprehensive analysis of company's innovation and performance outlook
""" """
if patents is None:
# Check SERP query cache first
query_hash = hashlib.sha256(company_name.lower().encode()).hexdigest()
cached_ids = self.db.get_cached_serp_query(query_hash)
if cached_ids is not None:
print(f"Using cached SERP results for {company_name} ({len(cached_ids)} patents)")
patents = Patents(patents=[
Patent(patent_id=pid, pdf_link="")
for pid in cached_ids
])
else:
print(f"Retrieving patents for {company_name}...") print(f"Retrieving patents for {company_name}...")
patents = SERP.query(company_name) patents = SERP.query(company_name)
# Cache the SERP results
if patents.patents:
self.db.store_serp_query(
company_name=company_name,
query_hash=query_hash,
patent_ids=[p.patent_id for p in patents.patents],
)
if not patents.patents: if not patents.patents:
return f"No patents found for {company_name}" return f"No patents found for {company_name}"
print(f"Found {len(patents.patents)} patents. Processing...") print(f"Found {len(patents.patents)} patents. Processing...")
# Download and parse each patent # Download, parse, and minimize patents in parallel
processed_patents = [] processed_patents = []
for idx, patent in enumerate(patents.patents, 1): with ThreadPoolExecutor(max_workers=config.patent_thread_workers) as executor:
print(f"Processing patent {idx}/{len(patents.patents)}: {patent.patent_id}") future_to_patent = {
executor.submit(self._process_single_patent, patent, company_name, self.db): patent
for patent in patents.patents
}
for future in as_completed(future_to_patent):
patent = future_to_patent[future]
try: try:
# Download PDF result = future.result()
patent = SERP.save_patents(patent) if result:
processed_patents.append(result)
# Parse sections from PDF
sections = SERP.parse_patent_pdf(patent.pdf_path)
# Minimize for LLM (remove bloat)
minimized_content = SERP.minimize_patent_for_llm(sections)
processed_patents.append(
{"patent_id": patent.patent_id, "content": minimized_content}
)
except Exception as e: except Exception as e:
print(f"Warning: Failed to process {patent.patent_id}: {e}") print(f"Warning: Failed to process {patent.patent_id}: {e}")
continue
if not processed_patents: if not processed_patents:
return f"Failed to process any patents for {company_name}" return f"Failed to process any patents for {company_name}"
@@ -113,6 +132,46 @@ class CompanyAnalyzer:
except Exception as e: except Exception as e:
return f"Failed to analyze patent {patent_id}: {e}" return f"Failed to analyze patent {patent_id}: {e}"
@staticmethod
def _process_single_patent(
patent: Patent,
company_name: str = "",
db: DatabaseClient | None = None,
) -> dict | None:
"""Download, parse, and minimize a single patent. Thread-safe.
Checks DB cache before downloading. Stores results after processing.
Returns:
Dict with patent_id and minimized content, or None on failure.
"""
try:
# Check DB cache first
if db:
cached = db.get_cached_patent(patent.patent_id)
if cached and cached.get("minimized_content"):
return {"patent_id": patent.patent_id, "content": cached["minimized_content"]}
# Full processing: download, parse, minimize
patent = SERP.save_patents(patent)
sections = SERP.parse_patent_pdf(patent.pdf_path)
minimized_content = SERP.minimize_patent_for_llm(sections)
# Store in DB cache
if db:
db.store_patent(
patent_id=patent.patent_id,
company_name=company_name,
pdf_link=patent.pdf_link,
raw_sections=sections,
minimized_content=minimized_content,
)
return {"patent_id": patent.patent_id, "content": minimized_content}
except Exception as e:
print(f"Warning: Failed to process {patent.patent_id}: {e}")
return None
def _analyze_company_safe(self, company_name: str) -> CompanyAnalysisResult: def _analyze_company_safe(self, company_name: str) -> CompanyAnalysisResult:
"""Internal wrapper that catches exceptions and returns structured result. """Internal wrapper that catches exceptions and returns structured result.
@@ -123,11 +182,14 @@ class CompanyAnalyzer:
CompanyAnalysisResult with success/failure status CompanyAnalysisResult with success/failure status
""" """
try: try:
patents = SERP.query(company_name) # Delegate to analyze_company which handles SERP/patent caching
patent_count = len(patents.patents) if patents.patents else 0
analysis = self.analyze_company(company_name) analysis = self.analyze_company(company_name)
# Determine patent count from cached SERP query
query_hash = hashlib.sha256(company_name.lower().encode()).hexdigest()
cached_ids = self.db.get_cached_serp_query(query_hash)
patent_count = len(cached_ids) if cached_ids else 0
# Check if analysis indicates failure # Check if analysis indicates failure
if analysis.startswith("No patents found") or analysis.startswith( if analysis.startswith("No patents found") or analysis.startswith(
"Failed to process" "Failed to process"
+257 -7
View File
@@ -5,12 +5,23 @@ Provides REST API endpoints for analyzing company patent portfolios.
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from datetime import datetime from datetime import datetime
from typing import Annotated from typing import Annotated, List
from fastapi import BackgroundTasks, FastAPI, HTTPException, Query from fastapi import BackgroundTasks, Depends, FastAPI, HTTPException, Query
from pydantic import BaseModel, Field from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel, EmailStr, Field
from SPARC import config
from SPARC.analyzer import CompanyAnalyzer from SPARC.analyzer import CompanyAnalyzer
from SPARC.auth import (
TokenResponse,
UserResponse,
create_tokens,
decode_token,
get_current_admin,
get_current_user,
get_db_client,
)
from SPARC.types import BatchAnalysisResult, CompanyAnalysisResult from SPARC.types import BatchAnalysisResult, CompanyAnalysisResult
@@ -67,6 +78,42 @@ class HealthResponse(BaseModel):
timestamp: datetime timestamp: datetime
# Auth request/response models
class RegisterRequest(BaseModel):
"""User registration request."""
email: EmailStr
password: str = Field(..., min_length=8, description="Password (min 8 characters)")
class LoginRequest(BaseModel):
"""User login request."""
email: EmailStr
password: str
class RefreshRequest(BaseModel):
"""Token refresh request."""
refresh_token: str
class UpdateRoleRequest(BaseModel):
"""Update user role request."""
role: str = Field(..., pattern="^(admin|user)$")
class AnalyticsResponse(BaseModel):
"""Analytics response model."""
total_messages: int
by_company: List[dict]
by_type: List[dict]
period_days: int
# In-memory job storage (for demo; production would use Redis/DB) # In-memory job storage (for demo; production would use Redis/DB)
_jobs: dict[str, JobStatus] = {} _jobs: dict[str, JobStatus] = {}
_job_counter = 0 _job_counter = 0
@@ -114,8 +161,199 @@ app = FastAPI(
description="Semiconductor Patent & Analytics Report Core - Patent portfolio analysis using AI", description="Semiconductor Patent & Analytics Report Core - Patent portfolio analysis using AI",
version="1.0.0", version="1.0.0",
lifespan=lifespan, lifespan=lifespan,
root_path=config.root_path,
) )
# Add CORS middleware for React frontend
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:3000", "http://localhost:5173"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# ============== Auth Endpoints ==============
@app.post("/auth/register", response_model=UserResponse, tags=["Auth"])
async def register(request: RegisterRequest):
"""Register a new user.
The first registered user automatically becomes an admin.
"""
db = get_db_client()
# First user becomes admin
user_count = db.get_user_count()
role = "admin" if user_count == 0 else "user"
user = db.create_user(
email=request.email,
password=request.password,
role=role,
)
if not user:
raise HTTPException(
status_code=400,
detail="Email already registered",
)
return UserResponse(
id=user["id"],
email=user["email"],
role=user["role"],
created_at=user["created_at"],
)
@app.post("/auth/login", response_model=TokenResponse, tags=["Auth"])
async def login(request: LoginRequest):
"""Authenticate user and return JWT tokens."""
db = get_db_client()
user = db.authenticate_user(request.email, request.password)
if not user:
raise HTTPException(
status_code=401,
detail="Invalid email or password",
)
return create_tokens(user["id"], user["email"], user["role"])
@app.post("/auth/refresh", response_model=TokenResponse, tags=["Auth"])
async def refresh_token(request: RefreshRequest):
"""Refresh access token using refresh token."""
payload = decode_token(request.refresh_token)
if not payload or payload.type != "refresh":
raise HTTPException(
status_code=401,
detail="Invalid refresh token",
)
db = get_db_client()
user = db.get_user_by_id(payload.user_id)
if not user:
raise HTTPException(
status_code=401,
detail="User not found",
)
return create_tokens(user["id"], user["email"], user["role"])
@app.get("/auth/me", response_model=UserResponse, tags=["Auth"])
async def get_me(current_user: UserResponse = Depends(get_current_user)):
"""Get current authenticated user."""
return current_user
# ============== Admin Endpoints ==============
@app.get("/admin/users", response_model=List[UserResponse], tags=["Admin"])
async def list_users(
limit: int = Query(default=100, ge=1, le=1000),
offset: int = Query(default=0, ge=0),
_: UserResponse = Depends(get_current_admin),
):
"""List all users (admin only)."""
db = get_db_client()
users = db.get_all_users(limit=limit, offset=offset)
return [
UserResponse(
id=u["id"],
email=u["email"],
role=u["role"],
created_at=u["created_at"],
)
for u in users
]
@app.patch("/admin/users/{user_id}/role", response_model=UserResponse, tags=["Admin"])
async def update_user_role(
user_id: int,
request: UpdateRoleRequest,
current_admin: UserResponse = Depends(get_current_admin),
):
"""Update a user's role (admin only)."""
if user_id == current_admin.id:
raise HTTPException(
status_code=400,
detail="Cannot change your own role",
)
db = get_db_client()
user = db.update_user_role(user_id, request.role)
if not user:
raise HTTPException(
status_code=404,
detail="User not found",
)
return UserResponse(
id=user["id"],
email=user["email"],
role=user["role"],
created_at=user["created_at"],
)
@app.delete("/admin/users/{user_id}", tags=["Admin"])
async def delete_user(
user_id: int,
current_admin: UserResponse = Depends(get_current_admin),
):
"""Delete a user (admin only)."""
if user_id == current_admin.id:
raise HTTPException(
status_code=400,
detail="Cannot delete yourself",
)
db = get_db_client()
deleted = db.delete_user(user_id)
if not deleted:
raise HTTPException(
status_code=404,
detail="User not found",
)
return {"message": "User deleted"}
# ============== Analytics Endpoint ==============
@app.get("/analytics", response_model=AnalyticsResponse, tags=["Analytics"])
async def get_analytics(
days: int = Query(default=30, ge=1, le=365),
_: UserResponse = Depends(get_current_user),
):
"""Get analytics data (authenticated users only)."""
db = get_db_client()
analytics = db.get_analytics(days=days)
return AnalyticsResponse(
total_messages=analytics["total_messages"],
by_company=analytics["by_company"],
by_type=analytics["by_type"],
period_days=analytics["period_days"],
)
# ============== System Endpoints ==============
@app.get("/health", response_model=HealthResponse, tags=["System"]) @app.get("/health", response_model=HealthResponse, tags=["System"])
async def health_check(): async def health_check():
@@ -132,7 +370,10 @@ async def health_check():
response_model=CompanyAnalysisResponse, response_model=CompanyAnalysisResponse,
tags=["Analysis"], tags=["Analysis"],
) )
async def analyze_company(company_name: str): async def analyze_company(
company_name: str,
_: UserResponse = Depends(get_current_user),
):
"""Analyze a single company's patent portfolio. """Analyze a single company's patent portfolio.
This endpoint retrieves recent patents for the specified company, This endpoint retrieves recent patents for the specified company,
@@ -156,7 +397,10 @@ async def analyze_company(company_name: str):
response_model=BatchAnalysisResponse, response_model=BatchAnalysisResponse,
tags=["Analysis"], tags=["Analysis"],
) )
async def analyze_companies_batch(request: BatchAnalysisRequest): async def analyze_companies_batch(
request: BatchAnalysisRequest,
_: UserResponse = Depends(get_current_user),
):
"""Analyze multiple companies' patent portfolios. """Analyze multiple companies' patent portfolios.
Processes companies concurrently for improved performance. Processes companies concurrently for improved performance.
@@ -209,7 +453,9 @@ def _run_batch_job(job_id: str, companies: list[str], max_workers: int):
@app.post("/analyze/batch/async", response_model=JobStatus, tags=["Analysis"]) @app.post("/analyze/batch/async", response_model=JobStatus, tags=["Analysis"])
async def analyze_companies_async( async def analyze_companies_async(
request: BatchAnalysisRequest, background_tasks: BackgroundTasks request: BatchAnalysisRequest,
background_tasks: BackgroundTasks,
_: UserResponse = Depends(get_current_user),
): ):
"""Start an asynchronous batch analysis job. """Start an asynchronous batch analysis job.
@@ -243,7 +489,10 @@ async def analyze_companies_async(
@app.get("/jobs/{job_id}", response_model=JobStatus, tags=["Jobs"]) @app.get("/jobs/{job_id}", response_model=JobStatus, tags=["Jobs"])
async def get_job_status(job_id: str): async def get_job_status(
job_id: str,
_: UserResponse = Depends(get_current_user),
):
"""Get the status of a background analysis job. """Get the status of a background analysis job.
Args: Args:
@@ -265,6 +514,7 @@ async def list_jobs(
Query(description="Filter by status: pending, running, completed, failed"), Query(description="Filter by status: pending, running, completed, failed"),
] = None, ] = None,
limit: Annotated[int, Query(ge=1, le=100)] = 10, limit: Annotated[int, Query(ge=1, le=100)] = 10,
_: UserResponse = Depends(get_current_user),
): ):
"""List all analysis jobs. """List all analysis jobs.
+210
View File
@@ -0,0 +1,210 @@
"""JWT authentication utilities for SPARC API."""
import os
from datetime import datetime, timedelta, timezone
from typing import Optional
import jwt
from fastapi import Depends, HTTPException, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from pydantic import BaseModel
from SPARC import config
from SPARC.database import DatabaseClient
# JWT Configuration
JWT_SECRET = os.getenv("JWT_SECRET", "sparc-secret-key-change-in-production")
JWT_ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 30
REFRESH_TOKEN_EXPIRE_DAYS = 7
security = HTTPBearer()
class TokenPayload(BaseModel):
"""JWT token payload."""
sub: str # user_id as string (JWT RFC 7519 requires sub to be a string)
email: str
role: str
exp: datetime
type: str # "access" or "refresh"
@property
def user_id(self) -> int:
"""Get user_id as integer."""
return int(self.sub)
class TokenResponse(BaseModel):
"""Token response model."""
access_token: str
refresh_token: str
token_type: str = "bearer"
class UserResponse(BaseModel):
"""User response model."""
id: int
email: str
role: str
created_at: datetime
def create_access_token(user_id: int, email: str, role: str) -> str:
"""Create a JWT access token.
Args:
user_id: User ID
email: User email
role: User role
Returns:
Encoded JWT token
"""
expire = datetime.now(timezone.utc) + timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
payload = {
"sub": str(user_id),
"email": email,
"role": role,
"exp": expire,
"type": "access",
}
return jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
def create_refresh_token(user_id: int, email: str, role: str) -> str:
"""Create a JWT refresh token.
Args:
user_id: User ID
email: User email
role: User role
Returns:
Encoded JWT token
"""
expire = datetime.now(timezone.utc) + timedelta(days=REFRESH_TOKEN_EXPIRE_DAYS)
payload = {
"sub": str(user_id),
"email": email,
"role": role,
"exp": expire,
"type": "refresh",
}
return jwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
def create_tokens(user_id: int, email: str, role: str) -> TokenResponse:
"""Create both access and refresh tokens.
Args:
user_id: User ID
email: User email
role: User role
Returns:
TokenResponse with both tokens
"""
return TokenResponse(
access_token=create_access_token(user_id, email, role),
refresh_token=create_refresh_token(user_id, email, role),
)
def decode_token(token: str) -> Optional[TokenPayload]:
"""Decode and validate a JWT token.
Args:
token: JWT token string
Returns:
TokenPayload if valid, None otherwise
"""
try:
payload = jwt.decode(token, JWT_SECRET, algorithms=[JWT_ALGORITHM])
return TokenPayload(**payload)
except jwt.ExpiredSignatureError:
return None
except jwt.InvalidTokenError:
return None
def get_db_client() -> DatabaseClient:
"""Get database client for auth operations."""
client = DatabaseClient(config.database_url)
client.connect()
return client
async def get_current_user(
credentials: HTTPAuthorizationCredentials = Depends(security),
) -> UserResponse:
"""Get the current authenticated user from JWT token.
Args:
credentials: Bearer token from request
Returns:
UserResponse with user details
Raises:
HTTPException: If token is invalid or expired
"""
token = credentials.credentials
payload = decode_token(token)
if not payload:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired token",
headers={"WWW-Authenticate": "Bearer"},
)
if payload.type != "access":
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token type",
headers={"WWW-Authenticate": "Bearer"},
)
db = get_db_client()
user = db.get_user_by_id(payload.user_id)
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="User not found",
headers={"WWW-Authenticate": "Bearer"},
)
return UserResponse(
id=user["id"],
email=user["email"],
role=user["role"],
created_at=user["created_at"],
)
async def get_current_admin(
current_user: UserResponse = Depends(get_current_user),
) -> UserResponse:
"""Require admin role for the current user.
Args:
current_user: Current authenticated user
Returns:
UserResponse if admin
Raises:
HTTPException: If user is not admin
"""
if current_user.role != "admin":
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Admin access required",
)
return current_user
+17 -4
View File
@@ -13,10 +13,23 @@ api_key = os.getenv("API_KEY")
# OpenRouter API key for LLM analysis # OpenRouter API key for LLM analysis
openrouter_api_key = os.getenv("OPENROUTER_API_KEY") openrouter_api_key = os.getenv("OPENROUTER_API_KEY")
# Database configuration # Database configuration - all messages are stored in the database
# The database serves as both a persistent store and a cache layer
database_url = os.getenv("DATABASE_URL", "postgresql://postgres:postgres@localhost:5432/sparc") database_url = os.getenv("DATABASE_URL", "postgresql://postgres:postgres@localhost:5432/sparc")
# Toggle between database mode and API mode # Cache configuration
# When True: stores all messages in database instead of sending to OpenRouter # When enabled (default), the system checks the database for cached responses
# When False: sends messages to OpenRouter API as normal # before making API calls, saving tokens and reducing latency
use_cache = os.getenv("USE_CACHE", "true").lower() in ("true", "1", "yes")
# Legacy compatibility - USE_DATABASE is deprecated, database is always used
# This variable is kept for backwards compatibility but has no effect
use_database = os.getenv("USE_DATABASE", "false").lower() in ("true", "1", "yes") use_database = os.getenv("USE_DATABASE", "false").lower() in ("true", "1", "yes")
# Patent search configuration
patent_search_days = int(os.getenv("PATENT_SEARCH_DAYS", "90"))
patent_thread_workers = int(os.getenv("PATENT_THREAD_WORKERS", "5"))
# Root path for running behind a reverse proxy (e.g., "/api" when served at /api/)
# This ensures OpenAPI docs work correctly when accessed via the proxy
root_path = os.getenv("ROOT_PATH", "")
+470 -8
View File
@@ -1,33 +1,62 @@
"""Database client for storing and retrieving LLM messages.""" """Database client for storing and retrieving LLM messages and user authentication."""
import contextlib
import psycopg2 import psycopg2
from psycopg2.pool import ThreadedConnectionPool
from psycopg2.extras import RealDictCursor from psycopg2.extras import RealDictCursor
from typing import Dict, List, Optional from typing import Dict, List, Optional
from datetime import datetime from datetime import datetime, timedelta
import json import json
import hashlib
import bcrypt
class DatabaseClient: class DatabaseClient:
"""Handles database operations for message storage and retrieval.""" """Handles database operations for message storage and retrieval."""
def __init__(self, database_url: str): def __init__(self, database_url: str, minconn: int = 2, maxconn: int = 10):
"""Initialize the database client. """Initialize the database client.
Args: Args:
database_url: PostgreSQL connection string database_url: PostgreSQL connection string
minconn: Minimum connections in the pool
maxconn: Maximum connections in the pool
""" """
self.database_url = database_url self.database_url = database_url
self._pool: ThreadedConnectionPool | None = None
self._minconn = minconn
self._maxconn = maxconn
# Legacy single connection kept for backwards compatibility
self.conn = None self.conn = None
def _ensure_pool(self):
"""Create the connection pool if it doesn't exist yet."""
if self._pool is None or self._pool.closed:
self._pool = ThreadedConnectionPool(
self._minconn, self._maxconn, self.database_url
)
@contextlib.contextmanager
def get_conn(self):
"""Check out a connection from the pool. Returns it on exit."""
self._ensure_pool()
conn = self._pool.getconn()
try:
yield conn
finally:
self._pool.putconn(conn)
def connect(self): def connect(self):
"""Establish database connection.""" """Establish database connection (legacy single-connection path)."""
if not self.conn or self.conn.closed: if not self.conn or self.conn.closed:
self.conn = psycopg2.connect(self.database_url) self.conn = psycopg2.connect(self.database_url)
def close(self): def close(self):
"""Close database connection.""" """Close database connection and pool."""
if self.conn and not self.conn.closed: if self.conn and not self.conn.closed:
self.conn.close() self.conn.close()
if self._pool and not self._pool.closed:
self._pool.closeall()
def initialize_schema(self): def initialize_schema(self):
"""Create database tables if they don't exist.""" """Create database tables if they don't exist."""
@@ -43,10 +72,12 @@ class DatabaseClient:
analysis_type VARCHAR(50), analysis_type VARCHAR(50),
model VARCHAR(100), model VARCHAR(100),
prompt TEXT NOT NULL, prompt TEXT NOT NULL,
prompt_hash VARCHAR(64),
response TEXT, response TEXT,
metadata JSONB, metadata JSONB,
token_usage JSONB, token_usage JSONB,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
is_cached BOOLEAN DEFAULT FALSE
) )
""") """)
@@ -62,8 +93,143 @@ class DatabaseClient:
ON llm_messages(company_name) ON llm_messages(company_name)
""") """)
# Add prompt_hash and is_cached columns if they don't exist (for existing tables)
# This must run BEFORE creating the index on prompt_hash
cursor.execute("""
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'llm_messages' AND column_name = 'prompt_hash'
) THEN
ALTER TABLE llm_messages ADD COLUMN prompt_hash VARCHAR(64);
END IF;
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'llm_messages' AND column_name = 'is_cached'
) THEN
ALTER TABLE llm_messages ADD COLUMN is_cached BOOLEAN DEFAULT FALSE;
END IF;
END $$;
""")
# Create index on prompt_hash for cache lookups
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_messages_prompt_hash
ON llm_messages(prompt_hash)
""")
# Create users table for authentication
cursor.execute("""
CREATE TABLE IF NOT EXISTS users (
id SERIAL PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
role VARCHAR(20) DEFAULT 'user' CHECK (role IN ('admin', 'user')),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
# Create index on email for fast lookups
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_users_email
ON users(email)
""")
# Create patents cache table
cursor.execute("""
CREATE TABLE IF NOT EXISTS patents (
patent_id VARCHAR(64) PRIMARY KEY,
company_name VARCHAR(255),
pdf_link TEXT,
raw_sections JSONB,
minimized_content TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_patents_company
ON patents(company_name)
""")
# Create SERP query cache table
cursor.execute("""
CREATE TABLE IF NOT EXISTS serp_queries (
id SERIAL PRIMARY KEY,
company_name VARCHAR(255),
query_hash VARCHAR(64) UNIQUE,
result_patent_ids TEXT[],
expires_at TIMESTAMP NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
cursor.execute("""
CREATE INDEX IF NOT EXISTS idx_serp_queries_hash
ON serp_queries(query_hash)
""")
self.conn.commit() self.conn.commit()
@staticmethod
def hash_prompt(prompt: str) -> str:
"""Generate a hash of the prompt for cache lookups.
Args:
prompt: The prompt text to hash
Returns:
SHA-256 hash of the prompt
"""
return hashlib.sha256(prompt.encode()).hexdigest()
def get_cached_response(
self,
prompt: str,
company_name: Optional[str] = None,
analysis_type: Optional[str] = None,
) -> Optional[Dict]:
"""Look up a cached response for a given prompt.
Args:
prompt: The prompt to look up
company_name: Optional company name filter
analysis_type: Optional analysis type filter
Returns:
Cached message dict if found, None otherwise
"""
self.connect()
prompt_hash = self.hash_prompt(prompt)
query = """
SELECT * FROM llm_messages
WHERE prompt_hash = %s
AND response IS NOT NULL
AND response NOT LIKE '[DATABASE MODE]%%'
AND response NOT LIKE '[TEST MODE]%%'
AND response NOT LIKE '[NO API]%%'
"""
params = [prompt_hash]
if company_name:
query += " AND company_name = %s"
params.append(company_name)
if analysis_type:
query += " AND analysis_type = %s"
params.append(analysis_type)
query += " ORDER BY timestamp DESC LIMIT 1"
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(query, params)
result = cursor.fetchone()
return dict(result) if result else None
def store_message( def store_message(
self, self,
prompt: str, prompt: str,
@@ -73,6 +239,7 @@ class DatabaseClient:
model: Optional[str] = None, model: Optional[str] = None,
metadata: Optional[Dict] = None, metadata: Optional[Dict] = None,
token_usage: Optional[Dict] = None, token_usage: Optional[Dict] = None,
is_cached: bool = False,
) -> int: ) -> int:
"""Store an LLM message exchange in the database. """Store an LLM message exchange in the database.
@@ -84,28 +251,33 @@ class DatabaseClient:
model: Model identifier used model: Model identifier used
metadata: Additional metadata as dict metadata: Additional metadata as dict
token_usage: Token usage information token_usage: Token usage information
is_cached: Whether this response was served from cache
Returns: Returns:
The ID of the inserted record The ID of the inserted record
""" """
self.connect() self.connect()
prompt_hash = self.hash_prompt(prompt)
with self.conn.cursor() as cursor: with self.conn.cursor() as cursor:
cursor.execute( cursor.execute(
""" """
INSERT INTO llm_messages INSERT INTO llm_messages
(prompt, response, company_name, analysis_type, model, metadata, token_usage) (prompt, prompt_hash, response, company_name, analysis_type, model, metadata, token_usage, is_cached)
VALUES (%s, %s, %s, %s, %s, %s, %s) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)
RETURNING id RETURNING id
""", """,
( (
prompt, prompt,
prompt_hash,
response, response,
company_name, company_name,
analysis_type, analysis_type,
model, model,
json.dumps(metadata) if metadata else None, json.dumps(metadata) if metadata else None,
json.dumps(token_usage) if token_usage else None, json.dumps(token_usage) if token_usage else None,
is_cached,
), ),
) )
@@ -208,3 +380,293 @@ class DatabaseClient:
"by_type": [dict(row) for row in by_type], "by_type": [dict(row) for row in by_type],
"period_days": days, "period_days": days,
} }
# Patent Cache Methods
def get_cached_patent(self, patent_id: str) -> Optional[Dict]:
"""Look up a cached patent by ID.
Returns:
Dict with raw_sections and minimized_content, or None.
"""
with self.get_conn() as conn:
with conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"SELECT * FROM patents WHERE patent_id = %s",
(patent_id,),
)
row = cursor.fetchone()
return dict(row) if row else None
def store_patent(
self,
patent_id: str,
company_name: str,
pdf_link: str,
raw_sections: Dict,
minimized_content: str,
) -> None:
"""Store a processed patent in the cache."""
with self.get_conn() as conn:
with conn.cursor() as cursor:
cursor.execute(
"""
INSERT INTO patents (patent_id, company_name, pdf_link, raw_sections, minimized_content)
VALUES (%s, %s, %s, %s, %s)
ON CONFLICT (patent_id) DO UPDATE SET
raw_sections = EXCLUDED.raw_sections,
minimized_content = EXCLUDED.minimized_content
""",
(patent_id, company_name, pdf_link, json.dumps(raw_sections), minimized_content),
)
conn.commit()
def get_cached_serp_query(self, query_hash: str) -> Optional[List[str]]:
"""Look up cached SERP query results.
Returns:
List of patent IDs if cache hit and not expired, None otherwise.
"""
with self.get_conn() as conn:
with conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"""
SELECT result_patent_ids FROM serp_queries
WHERE query_hash = %s AND expires_at > NOW()
""",
(query_hash,),
)
row = cursor.fetchone()
return row["result_patent_ids"] if row else None
def store_serp_query(
self,
company_name: str,
query_hash: str,
patent_ids: List[str],
ttl_hours: int = 24,
) -> None:
"""Store SERP query results in the cache."""
expires_at = datetime.now() + timedelta(hours=ttl_hours)
with self.get_conn() as conn:
with conn.cursor() as cursor:
cursor.execute(
"""
INSERT INTO serp_queries (company_name, query_hash, result_patent_ids, expires_at)
VALUES (%s, %s, %s, %s)
ON CONFLICT (query_hash) DO UPDATE SET
result_patent_ids = EXCLUDED.result_patent_ids,
expires_at = EXCLUDED.expires_at
""",
(company_name, query_hash, patent_ids, expires_at),
)
conn.commit()
# User Authentication Methods
@staticmethod
def hash_password(password: str) -> str:
"""Hash a password using bcrypt.
Args:
password: Plain text password
Returns:
Hashed password string
"""
return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
@staticmethod
def verify_password(password: str, password_hash: str) -> bool:
"""Verify a password against its hash.
Args:
password: Plain text password
password_hash: Stored hash
Returns:
True if password matches
"""
return bcrypt.checkpw(password.encode(), password_hash.encode())
def create_user(
self,
email: str,
password: str,
role: str = "user",
) -> Optional[Dict]:
"""Create a new user.
Args:
email: User email
password: Plain text password
role: User role ('admin' or 'user')
Returns:
Created user dict or None if email exists
"""
self.connect()
password_hash = self.hash_password(password)
try:
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"""
INSERT INTO users (email, password_hash, role)
VALUES (%s, %s, %s)
RETURNING id, email, role, created_at
""",
(email, password_hash, role),
)
user = cursor.fetchone()
self.conn.commit()
return dict(user) if user else None
except psycopg2.errors.UniqueViolation:
self.conn.rollback()
return None
def authenticate_user(self, email: str, password: str) -> Optional[Dict]:
"""Authenticate a user by email and password.
Args:
email: User email
password: Plain text password
Returns:
User dict if authenticated, None otherwise
"""
self.connect()
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"SELECT * FROM users WHERE email = %s",
(email,),
)
user = cursor.fetchone()
if user and self.verify_password(password, user["password_hash"]):
return {
"id": user["id"],
"email": user["email"],
"role": user["role"],
"created_at": user["created_at"],
}
return None
def get_user_by_id(self, user_id: int) -> Optional[Dict]:
"""Get a user by ID.
Args:
user_id: User ID
Returns:
User dict or None
"""
self.connect()
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"SELECT id, email, role, created_at FROM users WHERE id = %s",
(user_id,),
)
user = cursor.fetchone()
return dict(user) if user else None
def get_user_by_email(self, email: str) -> Optional[Dict]:
"""Get a user by email.
Args:
email: User email
Returns:
User dict or None
"""
self.connect()
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"SELECT id, email, role, created_at FROM users WHERE email = %s",
(email,),
)
user = cursor.fetchone()
return dict(user) if user else None
def get_all_users(self, limit: int = 100, offset: int = 0) -> List[Dict]:
"""Get all users (admin only).
Args:
limit: Maximum number of users
offset: Offset for pagination
Returns:
List of user dicts
"""
self.connect()
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"""
SELECT id, email, role, created_at
FROM users
ORDER BY created_at DESC
LIMIT %s OFFSET %s
""",
(limit, offset),
)
return [dict(row) for row in cursor.fetchall()]
def update_user_role(self, user_id: int, role: str) -> Optional[Dict]:
"""Update a user's role (admin only).
Args:
user_id: User ID
role: New role ('admin' or 'user')
Returns:
Updated user dict or None
"""
self.connect()
with self.conn.cursor(cursor_factory=RealDictCursor) as cursor:
cursor.execute(
"""
UPDATE users
SET role = %s, updated_at = CURRENT_TIMESTAMP
WHERE id = %s
RETURNING id, email, role, created_at
""",
(role, user_id),
)
user = cursor.fetchone()
self.conn.commit()
return dict(user) if user else None
def delete_user(self, user_id: int) -> bool:
"""Delete a user (admin only).
Args:
user_id: User ID
Returns:
True if deleted
"""
self.connect()
with self.conn.cursor() as cursor:
cursor.execute("DELETE FROM users WHERE id = %s", (user_id,))
deleted = cursor.rowcount > 0
self.conn.commit()
return deleted
def get_user_count(self) -> int:
"""Get total user count.
Returns:
Number of users
"""
self.connect()
with self.conn.cursor() as cursor:
cursor.execute("SELECT COUNT(*) FROM users")
return cursor.fetchone()[0]
+79 -41
View File
@@ -9,31 +9,29 @@ from typing import Dict
class LLMAnalyzer: class LLMAnalyzer:
"""Handles LLM-based analysis of patent content.""" """Handles LLM-based analysis of patent content."""
def __init__(self, api_key: str | None = None, test_mode: bool = False, use_database: bool | None = None): def __init__(self, api_key: str | None = None, test_mode: bool = False, use_cache: bool | None = None):
"""Initialize the LLM analyzer. """Initialize the LLM analyzer.
Args: Args:
api_key: OpenRouter API key. If None, will attempt to load from config. api_key: OpenRouter API key. If None, will attempt to load from config.
test_mode: If True, print prompts instead of making API calls test_mode: If True, print prompts instead of making API calls
use_database: If True, store messages in database instead of calling API. use_cache: If True, check database cache before making API calls.
If None, will use config.use_database If None, uses config.use_cache (default: True)
""" """
self.test_mode = test_mode self.test_mode = test_mode
self.use_database = use_database if use_database is not None else config.use_database self.use_cache = use_cache if use_cache is not None else config.use_cache
self.db_client = None self.model = "anthropic/claude-3.5-sonnet"
# Initialize database client if in database mode # Always initialize database client for storage and caching
if self.use_database:
self.db_client = DatabaseClient(config.database_url) self.db_client = DatabaseClient(config.database_url)
self.db_client.initialize_schema() self.db_client.initialize_schema()
# Initialize OpenRouter client if not in database mode # Initialize OpenRouter client if API key is available
if (api_key or config.openrouter_api_key) and not test_mode and not self.use_database: if (api_key or config.openrouter_api_key) and not test_mode:
self.client = OpenAI( self.client = OpenAI(
api_key=api_key or config.openrouter_api_key, api_key=api_key or config.openrouter_api_key,
base_url="https://openrouter.ai/api/v1" base_url="https://openrouter.ai/api/v1"
) )
self.model = "anthropic/claude-3.5-sonnet"
else: else:
self.client = None self.client = None
@@ -68,22 +66,31 @@ Provide a concise analysis (2-3 paragraphs) focusing on what this patent reveals
print("=" * 80) print("=" * 80)
return "[TEST MODE - No API call made]" return "[TEST MODE - No API call made]"
# Database mode: store the prompt and return a placeholder response # Check cache first
if self.use_database: if self.use_cache:
response_text = "[DATABASE MODE] Message stored for testing/analytics. Enable API mode to get actual analysis." cached = self.db_client.get_cached_response(
prompt=prompt,
company_name=company_name,
analysis_type="single_patent"
)
if cached:
# Log the cache hit
self.db_client.store_message( self.db_client.store_message(
prompt=prompt, prompt=prompt,
response=response_text, response=cached["response"],
company_name=company_name, company_name=company_name,
analysis_type="single_patent", analysis_type="single_patent",
model=self.model if hasattr(self, 'model') else None, model=self.model,
metadata={"patent_content_length": len(patent_content)} metadata={
"patent_content_length": len(patent_content),
"cache_hit": True,
"original_message_id": cached["id"]
},
is_cached=True
) )
return cached["response"]
return response_text # Call API if no cache hit and client is available
# API mode: send to OpenRouter
if self.client: if self.client:
response = self.client.chat.completions.create( response = self.client.chat.completions.create(
model=self.model, model=self.model,
@@ -92,8 +99,7 @@ Provide a concise analysis (2-3 paragraphs) focusing on what this patent reveals
) )
response_text = response.choices[0].message.content response_text = response.choices[0].message.content
# Store in database if db_client is available (for logging even in API mode) # Store in database for future cache lookups
if self.db_client:
self.db_client.store_message( self.db_client.store_message(
prompt=prompt, prompt=prompt,
response=response_text, response=response_text,
@@ -110,6 +116,18 @@ Provide a concise analysis (2-3 paragraphs) focusing on what this patent reveals
return response_text return response_text
# No API client available - store prompt for later processing
placeholder = "[NO API] Prompt stored in database. Configure OPENROUTER_API_KEY to enable analysis."
self.db_client.store_message(
prompt=prompt,
response=placeholder,
company_name=company_name,
analysis_type="single_patent",
model=self.model,
metadata={"patent_content_length": len(patent_content), "pending": True}
)
return placeholder
def analyze_patent_portfolio( def analyze_patent_portfolio(
self, patents_data: list[Dict[str, str]], company_name: str self, patents_data: list[Dict[str, str]], company_name: str
) -> str: ) -> str:
@@ -150,25 +168,37 @@ Provide a comprehensive analysis (4-5 paragraphs) with a final verdict on the co
print(prompt) print(prompt)
return "[TEST MODE]" return "[TEST MODE]"
# Database mode: store the prompt and return a placeholder response
if self.use_database:
response_text = "[DATABASE MODE] Message stored for testing/analytics. Enable API mode to get actual analysis."
self.db_client.store_message(
prompt=prompt,
response=response_text,
company_name=company_name,
analysis_type="portfolio",
model=self.model if hasattr(self, 'model') else None,
metadata = { metadata = {
"patent_count": len(patents_data), "patent_count": len(patents_data),
"patent_ids": [p['patent_id'] for p in patents_data] "patent_ids": [p['patent_id'] for p in patents_data]
} }
# Check cache first
if self.use_cache:
cached = self.db_client.get_cached_response(
prompt=prompt,
company_name=company_name,
analysis_type="portfolio"
) )
if cached:
# Log the cache hit
self.db_client.store_message(
prompt=prompt,
response=cached["response"],
company_name=company_name,
analysis_type="portfolio",
model=self.model,
metadata={
**metadata,
"cache_hit": True,
"original_message_id": cached["id"]
},
is_cached=True
)
return cached["response"]
return response_text # Call API if no cache hit and client is available
if self.client:
# API mode: send to OpenRouter
try: try:
response = self.client.chat.completions.create( response = self.client.chat.completions.create(
model=self.model, model=self.model,
@@ -178,18 +208,14 @@ Provide a comprehensive analysis (4-5 paragraphs) with a final verdict on the co
response_text = response.choices[0].message.content response_text = response.choices[0].message.content
# Store in database if db_client is available (for logging even in API mode) # Store in database for future cache lookups
if self.db_client:
self.db_client.store_message( self.db_client.store_message(
prompt=prompt, prompt=prompt,
response=response_text, response=response_text,
company_name=company_name, company_name=company_name,
analysis_type="portfolio", analysis_type="portfolio",
model=self.model, model=self.model,
metadata={ metadata=metadata,
"patent_count": len(patents_data),
"patent_ids": [p['patent_id'] for p in patents_data]
},
token_usage={ token_usage={
"prompt_tokens": response.usage.prompt_tokens, "prompt_tokens": response.usage.prompt_tokens,
"completion_tokens": response.usage.completion_tokens, "completion_tokens": response.usage.completion_tokens,
@@ -201,3 +227,15 @@ Provide a comprehensive analysis (4-5 paragraphs) with a final verdict on the co
except AttributeError: except AttributeError:
return prompt return prompt
# No API client available - store prompt for later processing
placeholder = "[NO API] Prompt stored in database. Configure OPENROUTER_API_KEY to enable analysis."
self.db_client.store_message(
prompt=prompt,
response=placeholder,
company_name=company_name,
analysis_type="portfolio",
model=self.model,
metadata={**metadata, "pending": True}
)
return placeholder
+18 -6
View File
@@ -1,17 +1,20 @@
import os
import serpapi import serpapi
from SPARC import config from SPARC import config
import re import re
import pdfplumber # pip install pdfplumber import pdfplumber # pip install pdfplumber
import requests import requests
from datetime import datetime, timedelta
from typing import Dict from typing import Dict
from SPARC.types import Patents, Patent from SPARC.types import Patents, Patent
class SERP: class SERP:
def query(company: str) -> Patents: def query(company: str, days_back: int = None) -> Patents:
"""Query Google Patents for a company's recent patents. """Query Google Patents for a company's recent patents.
Args: Args:
company: Name of the company to search for company: Name of the company to search for
days_back: Number of days to look back for patents (default from config)
Returns: Returns:
Patents object containing list of patents with PDF links Patents object containing list of patents with PDF links
@@ -23,13 +26,19 @@ class SERP:
patents with restricted access). The returned count may be lower patents with restricted access). The returned count may be lower
than the requested number of results. than the requested number of results.
""" """
if days_back is None:
days_back = config.patent_search_days
end_date = datetime.now()
start_date = end_date - timedelta(days=days_back)
date_filter = f"cdr:1,cd_min:{start_date.strftime('%-m/%-d/%Y')},cd_max:{end_date.strftime('%-m/%-d/%Y')}"
# Make API call # Make API call
params = { params = {
"engine": "google_patents", "engine": "google_patents",
"q": company, "q": company,
"num": 10, "num": 10,
"filter": 1, "filter": 1,
"tbs": "cdr:1,cd_min:10/28/2025,cd_max:11/4/2025", "tbs": date_filter,
"api_key": config.api_key, "api_key": config.api_key,
} }
search = serpapi.search(params) search = serpapi.search(params)
@@ -46,7 +55,7 @@ class SERP:
def save_patents(patent: Patent) -> Patent: def save_patents(patent: Patent) -> Patent:
""" """
Save the patent PDF to the patents folder Save the patent PDF to the patents folder, skipping download if already cached.
Args: Args:
patent: Patent object patent: Patent object
@@ -54,12 +63,15 @@ class SERP:
Returns: Returns:
Patent object with updated PDF path Patent object with updated PDF path
""" """
pdf_path = f"patents/{patent.patent_id}.pdf"
os.makedirs("patents", exist_ok=True)
if not (os.path.exists(pdf_path) and os.path.getsize(pdf_path) > 0):
response = requests.get(patent.pdf_link) response = requests.get(patent.pdf_link)
print(patent.pdf_link) with open(pdf_path, "wb") as f:
with open(f"patents/{patent.patent_id}.pdf", "wb") as f:
f.write(response.content) f.write(response.content)
patent.pdf_path = f"patents/{patent.patent_id}.pdf" patent.pdf_path = pdf_path
return patent return patent
def parse_patent_pdf(pdf_path: str) -> Dict: def parse_patent_pdf(pdf_path: str) -> Dict:
-778
View File
@@ -1,778 +0,0 @@
"""SPARC Visualization Dashboard.
A Streamlit-based dashboard for visualizing patent analysis results.
Run with: streamlit run dashboard.py
"""
import streamlit as st
import plotly.express as px
import plotly.graph_objects as go
import pandas as pd
from datetime import datetime, timedelta
from SPARC.analyzer import CompanyAnalyzer
from SPARC.database import DatabaseClient
from SPARC import config
st.set_page_config(
page_title="SPARC Dashboard",
page_icon="",
layout="wide",
initial_sidebar_state="collapsed",
)
# Modern CSS styling
st.markdown("""
<style>
/* Hide default Streamlit elements */
#MainMenu {visibility: hidden;}
footer {visibility: hidden;}
header {visibility: hidden;}
/* Root variables for theming */
:root {
--primary: #6366f1;
--primary-dark: #4f46e5;
--secondary: #0ea5e9;
--success: #10b981;
--warning: #f59e0b;
--error: #ef4444;
--bg-dark: #0f172a;
--bg-card: #1e293b;
--bg-card-hover: #334155;
--text-primary: #f8fafc;
--text-secondary: #94a3b8;
--border: #334155;
}
/* Main app background */
.stApp {
background: linear-gradient(135deg, #0f172a 0%, #1e1b4b 100%);
}
/* Top navigation bar */
.nav-container {
background: rgba(30, 41, 59, 0.8);
backdrop-filter: blur(12px);
border-bottom: 1px solid rgba(99, 102, 241, 0.2);
padding: 1rem 2rem;
margin: -1rem -1rem 2rem -1rem;
display: flex;
align-items: center;
justify-content: space-between;
}
.nav-brand {
display: flex;
align-items: center;
gap: 0.75rem;
}
.nav-brand h1 {
font-size: 1.5rem;
font-weight: 700;
background: linear-gradient(135deg, #6366f1, #0ea5e9);
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
margin: 0;
}
.nav-brand span {
font-size: 0.75rem;
color: var(--text-secondary);
text-transform: uppercase;
letter-spacing: 0.1em;
}
/* Card styling */
.modern-card {
background: rgba(30, 41, 59, 0.6);
backdrop-filter: blur(8px);
border: 1px solid rgba(99, 102, 241, 0.15);
border-radius: 16px;
padding: 1.5rem;
margin-bottom: 1rem;
transition: all 0.3s ease;
}
.modern-card:hover {
border-color: rgba(99, 102, 241, 0.4);
box-shadow: 0 8px 32px rgba(99, 102, 241, 0.15);
}
/* Metric cards */
.metric-card {
background: linear-gradient(135deg, rgba(99, 102, 241, 0.1), rgba(14, 165, 233, 0.1));
border: 1px solid rgba(99, 102, 241, 0.2);
border-radius: 12px;
padding: 1.25rem;
text-align: center;
}
.metric-value {
font-size: 2rem;
font-weight: 700;
background: linear-gradient(135deg, #6366f1, #0ea5e9);
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
}
.metric-label {
font-size: 0.875rem;
color: var(--text-secondary);
text-transform: uppercase;
letter-spacing: 0.05em;
margin-top: 0.25rem;
}
/* Section headers */
.section-header {
font-size: 1.25rem;
font-weight: 600;
color: var(--text-primary);
margin-bottom: 1rem;
padding-bottom: 0.5rem;
border-bottom: 2px solid rgba(99, 102, 241, 0.3);
}
/* Input fields */
.stTextInput > div > div > input,
.stTextArea > div > div > textarea {
background: rgba(30, 41, 59, 0.8) !important;
border: 1px solid rgba(99, 102, 241, 0.3) !important;
border-radius: 10px !important;
color: var(--text-primary) !important;
padding: 0.75rem 1rem !important;
}
.stTextInput > div > div > input:focus,
.stTextArea > div > div > textarea:focus {
border-color: var(--primary) !important;
box-shadow: 0 0 0 2px rgba(99, 102, 241, 0.2) !important;
}
/* Buttons */
.stButton > button {
background: linear-gradient(135deg, #6366f1, #4f46e5) !important;
color: white !important;
border: none !important;
border-radius: 10px !important;
padding: 0.75rem 1.5rem !important;
font-weight: 600 !important;
transition: all 0.3s ease !important;
box-shadow: 0 4px 14px rgba(99, 102, 241, 0.3) !important;
}
.stButton > button:hover {
transform: translateY(-2px) !important;
box-shadow: 0 6px 20px rgba(99, 102, 241, 0.4) !important;
}
/* Tabs styling */
.stTabs [data-baseweb="tab-list"] {
background: rgba(30, 41, 59, 0.6);
border-radius: 12px;
padding: 0.5rem;
gap: 0.5rem;
border: 1px solid rgba(99, 102, 241, 0.15);
}
.stTabs [data-baseweb="tab"] {
background: transparent;
border-radius: 8px;
color: var(--text-secondary);
padding: 0.75rem 1.5rem;
font-weight: 500;
}
.stTabs [aria-selected="true"] {
background: linear-gradient(135deg, #6366f1, #4f46e5) !important;
color: white !important;
}
.stTabs [data-baseweb="tab-border"] {
display: none;
}
.stTabs [data-baseweb="tab-highlight"] {
display: none;
}
/* Expander styling */
.streamlit-expanderHeader {
background: rgba(30, 41, 59, 0.6) !important;
border: 1px solid rgba(99, 102, 241, 0.15) !important;
border-radius: 10px !important;
color: var(--text-primary) !important;
}
.streamlit-expanderContent {
background: rgba(30, 41, 59, 0.4) !important;
border: 1px solid rgba(99, 102, 241, 0.1) !important;
border-top: none !important;
border-radius: 0 0 10px 10px !important;
}
/* Slider */
.stSlider > div > div > div {
background: var(--primary) !important;
}
/* Select box */
.stSelectbox > div > div {
background: rgba(30, 41, 59, 0.8) !important;
border: 1px solid rgba(99, 102, 241, 0.3) !important;
border-radius: 10px !important;
}
/* Progress bar */
.stProgress > div > div > div {
background: linear-gradient(90deg, #6366f1, #0ea5e9) !important;
}
/* Alerts */
.stAlert {
border-radius: 10px !important;
border: none !important;
}
/* Metrics */
[data-testid="stMetricValue"] {
background: linear-gradient(135deg, #6366f1, #0ea5e9);
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
font-weight: 700;
}
[data-testid="stMetricLabel"] {
color: var(--text-secondary) !important;
}
/* Plotly charts */
.js-plotly-plot {
border-radius: 12px;
overflow: hidden;
}
/* Status badges */
.status-badge {
display: inline-block;
padding: 0.25rem 0.75rem;
border-radius: 9999px;
font-size: 0.75rem;
font-weight: 600;
text-transform: uppercase;
}
.status-success {
background: rgba(16, 185, 129, 0.2);
color: #10b981;
border: 1px solid rgba(16, 185, 129, 0.3);
}
.status-warning {
background: rgba(245, 158, 11, 0.2);
color: #f59e0b;
border: 1px solid rgba(245, 158, 11, 0.3);
}
.status-error {
background: rgba(239, 68, 68, 0.2);
color: #ef4444;
border: 1px solid rgba(239, 68, 68, 0.3);
}
/* Dividers */
hr {
border: none;
border-top: 1px solid rgba(99, 102, 241, 0.2);
margin: 1.5rem 0;
}
/* Info boxes */
.info-box {
background: linear-gradient(135deg, rgba(99, 102, 241, 0.1), rgba(14, 165, 233, 0.05));
border: 1px solid rgba(99, 102, 241, 0.2);
border-radius: 12px;
padding: 1rem 1.25rem;
margin: 1rem 0;
}
/* Feature list */
.feature-item {
display: flex;
align-items: flex-start;
gap: 0.75rem;
padding: 0.75rem 0;
border-bottom: 1px solid rgba(99, 102, 241, 0.1);
}
.feature-icon {
color: var(--primary);
font-size: 1.25rem;
}
</style>
""", unsafe_allow_html=True)
@st.cache_resource
def get_analyzer():
"""Get or create the CompanyAnalyzer instance."""
return CompanyAnalyzer()
@st.cache_resource
def get_db_client():
"""Get database client if available."""
if config.use_database:
try:
client = DatabaseClient()
client.connect()
return client
except Exception:
return None
return None
def render_header():
"""Render the modern dashboard header."""
st.markdown("""
<div class="nav-container">
<div class="nav-brand">
<h1>⚡ SPARC</h1>
<span>Semiconductor Patent Analytics</span>
</div>
</div>
""", unsafe_allow_html=True)
def render_navigation():
"""Render horizontal tab navigation at the top."""
tabs = st.tabs(["🔍 Company Analysis", "📦 Batch Analysis", "📊 Analytics", "️ About"])
return tabs
def render_company_analysis():
"""Render single company analysis page."""
st.markdown('<p class="section-header">Single Company Analysis</p>', unsafe_allow_html=True)
st.markdown("Analyze a company's patent portfolio using AI-powered insights.")
st.markdown("")
# Search card
with st.container():
col1, col2 = st.columns([3, 1])
with col1:
company_name = st.text_input(
"Company Name",
placeholder="Enter company name (e.g., nvidia, intel, amd)",
help="Enter the company name to analyze their patent portfolio",
label_visibility="collapsed",
)
with col2:
analyze_btn = st.button("🔍 Analyze", type="primary", use_container_width=True)
if analyze_btn and company_name:
with st.spinner(f"Analyzing {company_name}..."):
analyzer = get_analyzer()
result = analyzer._analyze_company_safe(company_name)
if result.success:
st.success(f"✓ Analysis complete for {company_name.upper()}")
st.markdown("")
# Metrics row with custom styling
col1, col2, col3 = st.columns(3)
with col1:
st.metric("Patents Found", result.patent_count)
with col2:
st.metric("Analysis Status", "Complete")
with col3:
st.metric("Timestamp", result.timestamp.strftime("%H:%M:%S"))
st.markdown("")
# Analysis content in a styled container
st.markdown('<p class="section-header">AI Analysis Results</p>', unsafe_allow_html=True)
with st.container():
st.markdown(result.analysis)
else:
st.error(f"Analysis failed: {result.error}")
elif not company_name and analyze_btn:
st.warning("Please enter a company name to analyze.")
def render_batch_analysis():
"""Render batch analysis page."""
st.markdown('<p class="section-header">Batch Company Analysis</p>', unsafe_allow_html=True)
st.markdown("Analyze multiple companies simultaneously for comparative insights.")
st.markdown("")
# Input section
col1, col2 = st.columns([2, 1])
with col1:
companies_input = st.text_area(
"Company Names",
placeholder="Enter company names (one per line or comma-separated):\nnvidia\namd\nintel\nqualcomm",
height=150,
label_visibility="collapsed",
)
with col2:
st.markdown("**Configuration**")
max_workers = st.slider("Concurrent Workers", 1, 5, 3, help="Number of parallel analysis threads")
st.markdown("")
analyze_btn = st.button(
"🚀 Run Batch Analysis", type="primary", use_container_width=True
)
if analyze_btn and companies_input:
# Parse company names
companies = [
c.strip()
for c in companies_input.replace(",", "\n").split("\n")
if c.strip()
]
if not companies:
st.warning("Please enter at least one company name")
return
st.info(f"🔄 Starting analysis of {len(companies)} companies...")
# Progress tracking
progress_bar = st.progress(0)
status_text = st.empty()
analyzer = get_analyzer()
def update_progress(company: str, completed: int, total: int):
progress = completed / total
progress_bar.progress(progress)
status_text.text(f"Analyzing {company}... ({completed}/{total})")
result = analyzer.analyze_companies(
companies=companies,
max_workers=max_workers,
progress_callback=update_progress,
)
progress_bar.progress(1.0)
status_text.text("✓ Analysis complete!")
st.markdown("")
# Summary metrics
st.markdown('<p class="section-header">Results Summary</p>', unsafe_allow_html=True)
col1, col2, col3, col4 = st.columns(4)
with col1:
st.metric("Total Companies", result.total_companies)
with col2:
st.metric("Successful", result.successful)
with col3:
st.metric("Failed", result.failed)
with col4:
success_rate = (
(result.successful / result.total_companies * 100)
if result.total_companies > 0
else 0
)
st.metric("Success Rate", f"{success_rate:.1f}%")
# Results chart
if result.results:
df = pd.DataFrame(
[
{
"Company": r.company_name.upper(),
"Patents": r.patent_count,
"Status": "Success" if r.success else "Failed",
}
for r in result.results
]
)
fig = px.bar(
df,
x="Company",
y="Patents",
color="Status",
color_discrete_map={"Success": "#10b981", "Failed": "#ef4444"},
title="",
)
fig.update_layout(
plot_bgcolor="rgba(0,0,0,0)",
paper_bgcolor="rgba(0,0,0,0)",
font_color="#94a3b8",
legend=dict(
orientation="h",
yanchor="bottom",
y=1.02,
xanchor="right",
x=1
),
xaxis=dict(showgrid=False),
yaxis=dict(showgrid=True, gridcolor="rgba(99, 102, 241, 0.1)"),
)
st.plotly_chart(fig, use_container_width=True)
st.markdown("")
# Individual results
st.markdown('<p class="section-header">Detailed Results</p>', unsafe_allow_html=True)
for r in result.results:
status_icon = "" if r.success else ""
status_class = "status-success" if r.success else "status-error"
with st.expander(
f"{status_icon} {r.company_name.upper()}{r.patent_count} patents"
):
if r.success:
st.markdown(r.analysis)
else:
st.error(r.error)
def render_analytics():
"""Render analytics page with database insights."""
st.markdown('<p class="section-header">Analytics Dashboard</p>', unsafe_allow_html=True)
st.markdown("Track historical analysis data and view insights.")
db_client = get_db_client()
if not db_client:
st.markdown("")
st.markdown("""
<div class="info-box">
<strong>⚠️ Database Not Connected</strong><br>
<span style="color: #94a3b8;">Set <code>USE_DATABASE=true</code> in your .env file to enable analytics tracking.</span>
</div>
""", unsafe_allow_html=True)
st.info("Analytics features require storing analysis results in PostgreSQL for historical tracking.")
return
st.markdown("")
# Time range selector
col1, col2, col3 = st.columns([1, 2, 1])
with col1:
days = st.selectbox("Time Range", [7, 14, 30, 90], index=0, format_func=lambda x: f"Last {x} days")
try:
analytics = db_client.get_analytics(days=days)
if not analytics:
st.info("No analytics data available yet. Run some analyses first!")
return
st.markdown("")
# Summary metrics
col1, col2, col3 = st.columns(3)
with col1:
total = analytics.get("total_messages", 0)
st.metric("Total Analyses", total)
with col2:
companies = len(analytics.get("by_company", {}))
st.metric("Companies Analyzed", companies)
with col3:
types = len(analytics.get("by_type", {}))
st.metric("Analysis Types", types)
st.markdown("")
# Charts
col1, col2 = st.columns(2)
with col1:
by_company = analytics.get("by_company", {})
if by_company:
df = pd.DataFrame(
[{"Company": k.upper(), "Count": v} for k, v in by_company.items()]
)
fig = px.pie(
df, values="Count", names="Company", title="Distribution by Company",
hole=0.4,
color_discrete_sequence=px.colors.sequential.Purp_r,
)
fig.update_layout(
plot_bgcolor="rgba(0,0,0,0)",
paper_bgcolor="rgba(0,0,0,0)",
font_color="#94a3b8",
)
st.plotly_chart(fig, use_container_width=True)
with col2:
by_type = analytics.get("by_type", {})
if by_type:
df = pd.DataFrame(
[{"Type": k, "Count": v} for k, v in by_type.items()]
)
fig = px.bar(df, x="Type", y="Count", title="Analysis Types",
color_discrete_sequence=["#6366f1"])
fig.update_layout(
plot_bgcolor="rgba(0,0,0,0)",
paper_bgcolor="rgba(0,0,0,0)",
font_color="#94a3b8",
xaxis=dict(showgrid=False),
yaxis=dict(showgrid=True, gridcolor="rgba(99, 102, 241, 0.1)"),
)
st.plotly_chart(fig, use_container_width=True)
st.markdown("")
# Recent messages
st.markdown('<p class="section-header">Recent Analyses</p>', unsafe_allow_html=True)
messages = db_client.get_messages(limit=10)
if messages:
for msg in messages:
with st.expander(
f"📄 {msg.get('company_name', 'Unknown').upper()}{msg.get('analysis_type', 'N/A')} ({msg.get('timestamp', 'N/A')})"
):
st.markdown(f"**Model:** `{msg.get('model', 'N/A')}`")
if msg.get("response"):
st.markdown(msg["response"][:500] + "...")
except Exception as e:
st.error(f"Error fetching analytics: {e}")
def render_about():
"""Render about page."""
st.markdown('<p class="section-header">About SPARC</p>', unsafe_allow_html=True)
col1, col2 = st.columns([2, 1])
with col1:
st.markdown("""
**SPARC** (Semiconductor Patent & Analytics Report Core) is an AI-powered patent analysis
platform that evaluates company performance by analyzing their patent portfolios
with cutting-edge language models.
""")
st.markdown("")
st.markdown("**Key Features**")
features = [
("🔍", "Patent Retrieval", "Automated collection via SerpAPI's Google Patents"),
("📄", "Intelligent Parsing", "Extracts key sections from patent documents"),
("🤖", "AI Analysis", "Deep analysis powered by Claude 3.5 Sonnet"),
("", "Batch Processing", "Analyze multiple companies concurrently"),
("🌐", "REST API", "FastAPI web service for seamless integration"),
("📊", "Analytics", "Track and visualize historical analysis data"),
]
for icon, title, desc in features:
st.markdown(f"""
<div class="feature-item">
<span class="feature-icon">{icon}</span>
<div>
<strong>{title}</strong><br>
<span style="color: #94a3b8; font-size: 0.875rem;">{desc}</span>
</div>
</div>
""", unsafe_allow_html=True)
with col2:
st.markdown("**Technology Stack**")
st.markdown("""
<div class="info-box">
<div style="display: grid; gap: 0.5rem;">
<div><span style="color: #6366f1;">Backend</span><br><span style="color: #94a3b8;">Python, FastAPI</span></div>
<div><span style="color: #6366f1;">AI Model</span><br><span style="color: #94a3b8;">Claude 3.5 Sonnet</span></div>
<div><span style="color: #6366f1;">Database</span><br><span style="color: #94a3b8;">PostgreSQL</span></div>
<div><span style="color: #6366f1;">Dashboard</span><br><span style="color: #94a3b8;">Streamlit, Plotly</span></div>
<div><span style="color: #6366f1;">Data Source</span><br><span style="color: #94a3b8;">SerpAPI Patents</span></div>
</div>
</div>
""", unsafe_allow_html=True)
st.markdown("")
st.markdown("**API Endpoints**")
st.code("http://localhost:8000/docs", language=None)
st.code("http://localhost:8000/health", language=None)
st.markdown("")
st.markdown("")
# System status
st.markdown('<p class="section-header">System Status</p>', unsafe_allow_html=True)
col1, col2, col3 = st.columns(3)
with col1:
db_client = get_db_client()
if db_client:
st.markdown("""
<div class="metric-card">
<div style="color: #10b981; font-size: 1.5rem;">●</div>
<div class="metric-label">Database</div>
<div style="color: #10b981; font-weight: 600;">Connected</div>
</div>
""", unsafe_allow_html=True)
else:
st.markdown("""
<div class="metric-card">
<div style="color: #f59e0b; font-size: 1.5rem;">●</div>
<div class="metric-label">Database</div>
<div style="color: #f59e0b; font-weight: 600;">Not Configured</div>
</div>
""", unsafe_allow_html=True)
with col2:
analyzer = get_analyzer()
if analyzer:
st.markdown("""
<div class="metric-card">
<div style="color: #10b981; font-size: 1.5rem;">●</div>
<div class="metric-label">Analyzer</div>
<div style="color: #10b981; font-weight: 600;">Ready</div>
</div>
""", unsafe_allow_html=True)
else:
st.markdown("""
<div class="metric-card">
<div style="color: #ef4444; font-size: 1.5rem;">●</div>
<div class="metric-label">Analyzer</div>
<div style="color: #ef4444; font-weight: 600;">Not Initialized</div>
</div>
""", unsafe_allow_html=True)
with col3:
st.markdown("""
<div class="metric-card">
<div style="color: #10b981; font-size: 1.5rem;">●</div>
<div class="metric-label">Dashboard</div>
<div style="color: #10b981; font-weight: 600;">Online</div>
</div>
""", unsafe_allow_html=True)
def main():
"""Main dashboard entry point."""
render_header()
tabs = render_navigation()
with tabs[0]:
render_company_analysis()
with tabs[1]:
render_batch_analysis()
with tabs[2]:
render_analytics()
with tabs[3]:
render_about()
if __name__ == "__main__":
main()
+35 -8
View File
@@ -12,25 +12,52 @@ services:
- postgres_data:/var/lib/postgresql/data - postgres_data:/var/lib/postgresql/data
healthcheck: healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"] test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s interval: 5s
timeout: 5s timeout: 5s
retries: 5 retries: 5
restart: unless-stopped
app: init-db:
build: build: .
context: . container_name: sparc-init-db
dockerfile: Dockerfile command: python scripts/init_database.py
container_name: sparc-app environment:
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/sparc
depends_on: depends_on:
postgres: postgres:
condition: service_healthy condition: service_healthy
restart: "no"
api:
build: .
container_name: sparc-api
command: uvicorn SPARC.api:app --host 0.0.0.0 --port 8000
environment: environment:
USE_DATABASE: true API_KEY: ${API_KEY}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
DATABASE_URL: postgresql://postgres:postgres@postgres:5432/sparc DATABASE_URL: postgresql://postgres:postgres@postgres:5432/sparc
USE_CACHE: "true"
JWT_SECRET: ${JWT_SECRET:-sparc-secret-key-change-in-production}
ROOT_PATH: /api
ports: ports:
- "8000:8000" - "8000:8000"
depends_on:
postgres:
condition: service_healthy
init-db:
condition: service_completed_successfully
volumes: volumes:
- .:/app - ./patents:/app/patents
restart: unless-stopped
dashboard:
build: ./frontend
container_name: sparc-dashboard
ports:
- "8080:80"
depends_on:
- api
restart: unless-stopped
volumes: volumes:
postgres_data: postgres_data:
+58 -45
View File
@@ -1,16 +1,19 @@
# Database Mode for Testing and Analytics # Database Storage and Caching
This document explains how to use SPARC's database mode for storing LLM messages for testing and analytics purposes. This document explains how SPARC uses PostgreSQL for storing LLM messages, enabling response caching and analytics.
## Overview ## Overview
SPARC supports two modes of operation: SPARC stores all LLM interactions in PostgreSQL, providing:
1. **API Mode** (default): Messages are sent to OpenRouter's API and you receive real LLM responses - **Response Caching**: Avoid redundant API calls for previously analyzed patents
2. **Database Mode**: Messages are stored in a PostgreSQL database without making API calls, useful for: - **Analytics**: Track usage patterns, token consumption, and analysis history
- Testing the application without consuming API credits - **Persistence**: Maintain analysis history across sessions
- Collecting analytics on message patterns and usage
- Development and debugging SPARC supports two cache modes:
1. **Cache Mode** (default, `USE_CACHE=true`): Check database for cached responses before making API calls
2. **Fresh Mode** (`USE_CACHE=false`): Always make fresh API calls (still stores results in database)
## Setup ## Setup
@@ -45,43 +48,43 @@ cp .env.example .env
Edit `.env` and set: Edit `.env` and set:
```env ```env
# For database mode (testing/analytics) # Database connection (required)
USE_DATABASE=true
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc
# For API mode (production) # Cache mode: use cached responses when available
USE_DATABASE=false USE_CACHE=true
# API key for fresh LLM calls
OPENROUTER_API_KEY=your_openrouter_key_here OPENROUTER_API_KEY=your_openrouter_key_here
``` ```
## Usage ## Usage
### Running in Database Mode ### Running with Cache Mode (Default)
Set `USE_DATABASE=true` in your `.env` file, then run the application normally: Set `USE_CACHE=true` in your `.env` file, then run the application normally:
```bash ```bash
python main.py python main.py
``` ```
Instead of sending messages to OpenRouter, the application will: The application will:
- Store all prompts in the database - Check the database for cached responses matching the request
- Return a placeholder response - If found, return the cached response (no API call)
- Log metadata (company name, analysis type, timestamps) - If not found, make an API call and store the response for future use
### Running in API Mode ### Running with Fresh Mode
Set `USE_DATABASE=false` in your `.env` file, then run the application normally: Set `USE_CACHE=false` in your `.env` file to always get fresh responses:
```bash ```bash
python main.py python main.py
``` ```
The application will send messages to OpenRouter and return real LLM responses. The application will:
- Always send messages to OpenRouter for real LLM responses
### Hybrid Mode (Optional) - Store all responses in the database
- Useful when you need the latest analysis or want to refresh cached data
You can also enable database logging while still using the API by initializing the database client in your code. The `LLMAnalyzer` will automatically log all API calls to the database if a database client is available.
## Viewing Analytics ## Viewing Analytics
@@ -195,16 +198,16 @@ docker-compose down -v
## Toggling Between Modes ## Toggling Between Modes
You can easily switch between modes by changing the `USE_DATABASE` environment variable: You can easily switch between modes by changing the `USE_CACHE` environment variable:
### Quick Toggle (temporary, for testing) ### Quick Toggle (temporary)
```bash ```bash
# Run in database mode # Run with caching enabled
USE_DATABASE=true python main.py USE_CACHE=true python main.py
# Run in API mode # Run with fresh API calls
USE_DATABASE=false python main.py USE_CACHE=false python main.py
``` ```
### Persistent Toggle ### Persistent Toggle
@@ -212,38 +215,48 @@ USE_DATABASE=false python main.py
Edit your `.env` file: Edit your `.env` file:
```env ```env
# For testing/analytics # Use cached responses when available (recommended for most use)
USE_DATABASE=true USE_CACHE=true
# For production use # Always make fresh API calls
USE_DATABASE=false USE_CACHE=false
``` ```
## Use Cases ## Use Cases
### Testing Without API Costs ### Cost Optimization with Caching
During development, enable database mode to test the full application flow without consuming API credits: Cache mode reduces API costs by reusing previous analysis results:
```bash ```bash
USE_DATABASE=true python main.py USE_CACHE=true python main.py
```
If the same company/patent combination was analyzed before, the cached response is returned instantly.
### Fresh Analysis
When you need the latest LLM analysis (e.g., after model updates):
```bash
USE_CACHE=false python main.py
``` ```
### Collecting Usage Analytics ### Collecting Usage Analytics
Enable database mode in a test environment to collect analytics on: The database stores all interactions, enabling analytics on:
- Which companies are analyzed most frequently - Which companies are analyzed most frequently
- Types of analyses performed - Types of analyses performed
- Prompt patterns and lengths - Token usage and costs over time
- Usage over time - Response caching hit rates
### Development and Debugging ### Development and Debugging
Database mode is useful for: Database storage is useful for:
- Testing patent parsing logic without API calls - Reviewing actual prompts sent to the LLM
- Analyzing response patterns
- Debugging the full pipeline end-to-end - Debugging the full pipeline end-to-end
- Collecting sample prompts for optimization - Understanding token usage patterns
- Understanding token usage patterns (when in API mode with logging)
## Troubleshooting ## Troubleshooting
+78 -120
View File
@@ -55,28 +55,25 @@ USE_DATABASE=true
## Step 2: Start Services with Docker Compose ## Step 2: Start Services with Docker Compose
```bash ```bash
# Start PostgreSQL database # Start all services (PostgreSQL, API, and Dashboard)
docker-compose up -d postgres docker-compose up -d
# Wait for postgres to be healthy (check with) # Check status
docker-compose ps docker-compose ps
# You should see sparc-postgres with status "healthy" # You should see:
# - sparc-postgres (healthy)
# - sparc-api (running on port 8000)
# - sparc-dashboard (running on port 8080)
``` ```
The database is automatically initialized by the `init-db` service.
--- ---
## Step 3: Initialize the Database ## Step 3: Database Schema
```bash The `init-db` service automatically creates the `llm_messages` table with the following schema:
# Option A: If running locally with Python
python scripts/init_database.py
# Option B: If using Docker, run inside container
docker-compose run --rm sparc-app python scripts/init_database.py
```
This creates the `llm_messages` table with the following schema:
| Column | Type | Purpose | | Column | Type | Purpose |
|--------|------|---------| |--------|------|---------|
@@ -95,22 +92,37 @@ This creates the `llm_messages` table with the following schema:
## Step 4: Run the Services ## Step 4: Run the Services
### Option A: Run Locally (Development) ### Option A: Run with Docker Compose (Recommended)
All services are started automatically with `docker-compose up -d` from Step 2.
```bash ```bash
# Terminal 1: Start FastAPI backend # View logs
uvicorn SPARC.api:app --host 0.0.0.0 --port 8000 --reload docker-compose logs -f
# Terminal 2: Start Streamlit dashboard # View specific service logs
streamlit run dashboard.py --server.port 8501 --server.address 0.0.0.0 docker-compose logs -f api
docker-compose logs -f dashboard
``` ```
### Option B: Run with Docker (Production) ### Option B: Run Locally (Development)
See [Production Docker Compose](#production-docker-compose) section below for a complete `docker-compose.prod.yml` configuration. If you prefer running services locally without Docker:
```bash ```bash
docker-compose -f docker-compose.prod.yml up -d # Start PostgreSQL with Docker
docker-compose up -d postgres
# Wait for database to be healthy, then initialize
python scripts/init_database.py
# Start FastAPI backend
uvicorn SPARC.api:app --host 0.0.0.0 --port 8000 --reload
# For the React frontend (separate terminal)
cd frontend
npm install
npm run dev
``` ```
--- ---
@@ -131,7 +143,7 @@ Access the services:
|---------|-----| |---------|-----|
| REST API | http://localhost:8000 | | REST API | http://localhost:8000 |
| API Documentation (Swagger) | http://localhost:8000/docs | | API Documentation (Swagger) | http://localhost:8000/docs |
| Dashboard (Web UI) | http://localhost:8501 | | Dashboard (Web UI) | http://localhost:8080 |
--- ---
@@ -139,16 +151,17 @@ Access the services:
### Via Dashboard (Web UI) ### Via Dashboard (Web UI)
1. Open http://localhost:8501 1. Open http://localhost:8080
2. Select **"Company Analysis"** from the sidebar 2. Register a new account or login (default admin: `admin` / `admin`)
3. Enter a company name (e.g., "Intel") 3. Navigate to **"Analysis"** from the sidebar
4. Click **"Analyze"** 4. Enter a company name (e.g., "Intel")
5. Click **"Analyze"**
This will: This will:
- Query SerpAPI for recent patents - Query SerpAPI for recent patents
- Download and parse patent PDFs - Download and parse patent PDFs
- Send patent content to Claude for analysis - Send patent content to Claude for analysis
- Store prompt/response in PostgreSQL - Store prompt/response in PostgreSQL (with caching)
- Display results in the dashboard - Display results in the dashboard
### Via REST API ### Via REST API
@@ -223,12 +236,12 @@ docker exec -it sparc-postgres psql -U postgres -d sparc -c \
| Component | Purpose | | Component | Purpose |
|-----------|---------| |-----------|---------|
| **Dashboard** | Streamlit web UI for interactive analysis | | **Dashboard** | React TypeScript web UI with authentication |
| **FastAPI** | REST API for programmatic access | | **FastAPI** | REST API with JWT authentication |
| **Analyzer** | Orchestrates patent retrieval and LLM analysis | | **Analyzer** | Orchestrates patent retrieval and LLM analysis |
| **SerpAPI** | Retrieves patent data from Google Patents | | **SerpAPI** | Retrieves patent data from Google Patents |
| **OpenRouter** | Routes requests to Claude for AI analysis | | **OpenRouter** | Routes requests to Claude for AI analysis |
| **PostgreSQL** | Stores prompts, responses, and analytics | | **PostgreSQL** | Stores prompts, responses, users, and cached results |
--- ---
@@ -238,10 +251,9 @@ docker exec -it sparc-postgres psql -U postgres -d sparc -c \
|----------|----------|---------|-------------| |----------|----------|---------|-------------|
| `API_KEY` | Yes | - | SerpAPI key for patent search | | `API_KEY` | Yes | - | SerpAPI key for patent search |
| `OPENROUTER_API_KEY` | Yes | - | OpenRouter API key for Claude access | | `OPENROUTER_API_KEY` | Yes | - | OpenRouter API key for Claude access |
| `DATABASE_URL` | Yes* | - | PostgreSQL connection string | | `DATABASE_URL` | Yes | - | PostgreSQL connection string |
| `USE_DATABASE` | No | `false` | Set to `true` to enable database storage | | `USE_CACHE` | No | `true` | Check database for cached responses before API calls |
| `JWT_SECRET` | Yes | - | Secret key for JWT authentication (change in production!) |
*Required when `USE_DATABASE=true`
### Database URL Format ### Database URL Format
@@ -256,97 +268,41 @@ postgresql://postgres:postgres@localhost:5432/sparc
--- ---
## Production Docker Compose ## Docker Compose Services
Create a `docker-compose.prod.yml` file for full production deployment: The `docker-compose.yml` includes all services needed for production:
```yaml | Service | Container | Port | Description |
version: '3.8' |---------|-----------|------|-------------|
| `postgres` | sparc-postgres | 5432 | PostgreSQL database |
| `init-db` | sparc-init-db | - | One-time database initialization (seeds admin user) |
| `api` | sparc-api | 8000 | FastAPI REST API with JWT auth |
| `dashboard` | sparc-dashboard | 8080 | React TypeScript web UI |
services: ### Common Docker Compose Commands
postgres:
image: postgres:16-alpine
container_name: sparc-postgres
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sparc
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 5
restart: unless-stopped
api:
build: .
container_name: sparc-api
command: uvicorn SPARC.api:app --host 0.0.0.0 --port 8000
environment:
- API_KEY=${API_KEY}
- OPENROUTER_API_KEY=${OPENROUTER_API_KEY}
- DATABASE_URL=postgresql://postgres:postgres@postgres:5432/sparc
- USE_DATABASE=true
ports:
- "8000:8000"
depends_on:
postgres:
condition: service_healthy
volumes:
- ./patents:/app/patents
restart: unless-stopped
dashboard:
build: .
container_name: sparc-dashboard
command: streamlit run dashboard.py --server.port 8501 --server.address 0.0.0.0
environment:
- API_KEY=${API_KEY}
- OPENROUTER_API_KEY=${OPENROUTER_API_KEY}
- DATABASE_URL=postgresql://postgres:postgres@postgres:5432/sparc
- USE_DATABASE=true
ports:
- "8501:8501"
depends_on:
- api
volumes:
- ./patents:/app/patents
restart: unless-stopped
init-db:
build: .
container_name: sparc-init-db
command: python scripts/init_database.py
environment:
- DATABASE_URL=postgresql://postgres:postgres@postgres:5432/sparc
- USE_DATABASE=true
depends_on:
postgres:
condition: service_healthy
restart: "no"
volumes:
postgres_data:
```
### Deploy with Production Compose
```bash ```bash
# Start all services # Start all services
docker-compose -f docker-compose.prod.yml up -d docker-compose up -d
# Start with rebuild (after code changes)
docker-compose up -d --build
# View logs # View logs
docker-compose -f docker-compose.prod.yml logs -f docker-compose logs -f
# View specific service logs
docker-compose logs -f api
docker-compose logs -f dashboard
# Stop all services # Stop all services
docker-compose -f docker-compose.prod.yml down docker-compose down
# Stop and remove volumes (WARNING: deletes data) # Stop and remove volumes (WARNING: deletes data)
docker-compose -f docker-compose.prod.yml down -v docker-compose down -v
# Restart a specific service
docker-compose restart api
``` ```
--- ---
@@ -417,20 +373,22 @@ docker-compose logs -f dashboard
## Quick Reference ## Quick Reference
```bash ```bash
# Development setup # Docker setup (recommended)
cp .env.example .env
# Edit .env with API keys
docker-compose up -d
# Local development setup
cp .env.example .env cp .env.example .env
# Edit .env with API keys # Edit .env with API keys
docker-compose up -d postgres docker-compose up -d postgres
python scripts/init_database.py python scripts/init_database.py
uvicorn SPARC.api:app --reload & uvicorn SPARC.api:app --reload &
streamlit run dashboard.py cd frontend && npm install && npm run dev &
# Production setup
docker-compose -f docker-compose.prod.yml up -d
# Check status # Check status
curl http://localhost:8000/health curl http://localhost:8000/health
open http://localhost:8501 open http://localhost:8080
# View data # View data
python scripts/view_analytics.py python scripts/view_analytics.py
+22
View File
@@ -0,0 +1,22 @@
# Dependencies
node_modules/
# Build output
dist/
# Local env files
.env.local
.env.*.local
# Editor directories
.vscode/
.idea/
# OS files
.DS_Store
Thumbs.db
# Debug logs
npm-debug.log*
yarn-debug.log*
yarn-error.log*
+32
View File
@@ -0,0 +1,32 @@
# Build stage
FROM node:20-alpine AS build
WORKDIR /app
# Copy package files
COPY package.json package-lock.json* ./
# Install dependencies
RUN npm install
# Copy source files
COPY . .
# Build the application
RUN npm run build
# Production stage
FROM nginx:alpine
# Copy built files
COPY --from=build /app/dist /usr/share/nginx/html
# Copy nginx template (processed at startup with envsubst)
COPY nginx.conf.template /etc/nginx/templates/default.conf.template
# Default API URL (override with -e API_URL=...)
ENV API_URL=http://api:8000/
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
+13
View File
@@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>SPARC Dashboard</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
+34
View File
@@ -0,0 +1,34 @@
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
# Gzip compression
gzip on;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
# Handle React Router (SPA)
location / {
try_files $uri $uri/ /index.html;
}
# Proxy API requests to backend
location /api/ {
proxy_pass ${API_URL}/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_cache_bypass $http_upgrade;
}
# Cache static assets
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2)$ {
expires 1y;
add_header Cache-Control "public, immutable";
}
}
+37
View File
@@ -0,0 +1,37 @@
{
"name": "sparc-dashboard",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"lint": "eslint .",
"preview": "vite preview"
},
"dependencies": {
"@tanstack/react-query": "^5.51.0",
"axios": "^1.7.2",
"lucide-react": "^0.400.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.24.0",
"recharts": "^2.12.7"
},
"devDependencies": {
"@eslint/js": "^9.6.0",
"@types/react": "^18.3.3",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.1",
"autoprefixer": "^10.4.19",
"eslint": "^9.6.0",
"eslint-plugin-react-hooks": "^5.1.0",
"eslint-plugin-react-refresh": "^0.4.7",
"globals": "^15.8.0",
"postcss": "^8.4.39",
"tailwindcss": "^3.4.4",
"typescript": "~5.5.3",
"typescript-eslint": "^8.0.0",
"vite": "^5.3.3"
}
}
+6
View File
@@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}
+67
View File
@@ -0,0 +1,67 @@
import { BrowserRouter, Routes, Route, Navigate } from 'react-router-dom';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { AuthProvider } from './context/AuthContext';
import { Layout } from './components/Layout';
import { ProtectedRoute } from './components/ProtectedRoute';
import { Login } from './pages/Login';
import { Register } from './pages/Register';
import { Analysis } from './pages/Analysis';
import { Batch } from './pages/Batch';
import { AnalyticsPage } from './pages/Analytics';
import { About } from './pages/About';
import { AdminUsers } from './pages/AdminUsers';
const queryClient = new QueryClient({
defaultOptions: {
queries: {
staleTime: 1000 * 60 * 5, // 5 minutes
retry: 1,
},
},
});
function App() {
return (
<QueryClientProvider client={queryClient}>
<AuthProvider>
<BrowserRouter>
<Routes>
{/* Public routes */}
<Route path="/login" element={<Login />} />
<Route path="/register" element={<Register />} />
{/* Protected routes */}
<Route
element={
<ProtectedRoute>
<Layout />
</ProtectedRoute>
}
>
<Route path="/analysis" element={<Analysis />} />
<Route path="/batch" element={<Batch />} />
<Route path="/analytics" element={<AnalyticsPage />} />
<Route path="/about" element={<About />} />
{/* Admin routes */}
<Route
path="/admin/users"
element={
<ProtectedRoute requireAdmin>
<AdminUsers />
</ProtectedRoute>
}
/>
</Route>
{/* Default redirect */}
<Route path="/" element={<Navigate to="/analysis" replace />} />
<Route path="*" element={<Navigate to="/analysis" replace />} />
</Routes>
</BrowserRouter>
</AuthProvider>
</QueryClientProvider>
);
}
export default App;
+154
View File
@@ -0,0 +1,154 @@
import axios, { AxiosError, InternalAxiosRequestConfig } from 'axios';
import type { TokenResponse, User, CompanyAnalysis, BatchAnalysisResult, JobStatus, Analytics } from '../types';
const API_BASE_URL = import.meta.env.VITE_API_URL || '/api';
const api = axios.create({
baseURL: API_BASE_URL,
headers: {
'Content-Type': 'application/json',
},
});
// Token management
let accessToken: string | null = localStorage.getItem('access_token');
let refreshToken: string | null = localStorage.getItem('refresh_token');
export const setTokens = (tokens: TokenResponse) => {
accessToken = tokens.access_token;
refreshToken = tokens.refresh_token;
localStorage.setItem('access_token', tokens.access_token);
localStorage.setItem('refresh_token', tokens.refresh_token);
};
export const clearTokens = () => {
accessToken = null;
refreshToken = null;
localStorage.removeItem('access_token');
localStorage.removeItem('refresh_token');
};
export const getAccessToken = () => accessToken;
// Request interceptor to add auth header
api.interceptors.request.use((config: InternalAxiosRequestConfig) => {
if (accessToken) {
config.headers.Authorization = `Bearer ${accessToken}`;
}
return config;
});
// Response interceptor to handle token refresh
api.interceptors.response.use(
(response) => response,
async (error: AxiosError) => {
const originalRequest = error.config as InternalAxiosRequestConfig & { _retry?: boolean };
if (error.response?.status === 401 && !originalRequest._retry && refreshToken) {
originalRequest._retry = true;
try {
const response = await axios.post<TokenResponse>(`${API_BASE_URL}/auth/refresh`, {
refresh_token: refreshToken,
});
setTokens(response.data);
originalRequest.headers.Authorization = `Bearer ${response.data.access_token}`;
return api(originalRequest);
} catch {
clearTokens();
window.location.href = '/login';
}
}
return Promise.reject(error);
}
);
// Auth API
export const authApi = {
register: async (email: string, password: string): Promise<User> => {
const response = await api.post<User>('/auth/register', { email, password });
return response.data;
},
login: async (email: string, password: string): Promise<TokenResponse> => {
const response = await api.post<TokenResponse>('/auth/login', { email, password });
setTokens(response.data);
return response.data;
},
getMe: async (): Promise<User> => {
const response = await api.get<User>('/auth/me');
return response.data;
},
logout: () => {
clearTokens();
},
};
// Analysis API
export const analysisApi = {
analyzeCompany: async (companyName: string): Promise<CompanyAnalysis> => {
const response = await api.get<CompanyAnalysis>(`/analyze/${encodeURIComponent(companyName)}`);
return response.data;
},
analyzeBatch: async (companies: string[], maxWorkers = 3): Promise<BatchAnalysisResult> => {
const response = await api.post<BatchAnalysisResult>('/analyze/batch', {
companies,
max_workers: maxWorkers,
});
return response.data;
},
analyzeBatchAsync: async (companies: string[], maxWorkers = 3): Promise<JobStatus> => {
const response = await api.post<JobStatus>('/analyze/batch/async', {
companies,
max_workers: maxWorkers,
});
return response.data;
},
getJobStatus: async (jobId: string): Promise<JobStatus> => {
const response = await api.get<JobStatus>(`/jobs/${jobId}`);
return response.data;
},
listJobs: async (status?: string, limit = 10): Promise<JobStatus[]> => {
const params = new URLSearchParams();
if (status) params.append('status', status);
params.append('limit', limit.toString());
const response = await api.get<JobStatus[]>(`/jobs?${params}`);
return response.data;
},
};
// Analytics API
export const analyticsApi = {
getAnalytics: async (days = 30): Promise<Analytics> => {
const response = await api.get<Analytics>(`/analytics?days=${days}`);
return response.data;
},
};
// Admin API
export const adminApi = {
listUsers: async (limit = 100, offset = 0): Promise<User[]> => {
const response = await api.get<User[]>(`/admin/users?limit=${limit}&offset=${offset}`);
return response.data;
},
updateUserRole: async (userId: number, role: 'admin' | 'user'): Promise<User> => {
const response = await api.patch<User>(`/admin/users/${userId}/role`, { role });
return response.data;
},
deleteUser: async (userId: number): Promise<void> => {
await api.delete(`/admin/users/${userId}`);
},
};
export default api;
+108
View File
@@ -0,0 +1,108 @@
import { Outlet, NavLink, useNavigate } from 'react-router-dom';
import { useAuth } from '../context/AuthContext';
import { Search, Layers, BarChart3, Info, Users, LogOut } from 'lucide-react';
export function Layout() {
const { user, isAdmin, logout } = useAuth();
const navigate = useNavigate();
const handleLogout = () => {
logout();
navigate('/login');
};
const navItems = [
{ to: '/analysis', icon: Search, label: 'Analysis' },
{ to: '/batch', icon: Layers, label: 'Batch' },
{ to: '/analytics', icon: BarChart3, label: 'Analytics' },
{ to: '/about', icon: Info, label: 'About' },
];
if (isAdmin) {
navItems.push({ to: '/admin/users', icon: Users, label: 'Users' });
}
return (
<div className="min-h-screen bg-gradient-to-br from-bg-dark to-indigo-950">
{/* Header */}
<header className="bg-bg-card/80 backdrop-blur-lg border-b border-primary/20">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
<div className="flex items-center justify-between h-16">
{/* Brand */}
<div className="flex items-center gap-3">
<span className="text-2xl"></span>
<div>
<h1 className="text-xl font-bold bg-gradient-to-r from-primary to-secondary bg-clip-text text-transparent">
SPARC
</h1>
<span className="text-xs text-text-secondary uppercase tracking-wider">
Semiconductor Patent Analytics
</span>
</div>
</div>
{/* Navigation */}
<nav className="hidden md:flex items-center gap-1 bg-bg-card/60 rounded-xl p-1 border border-primary/15">
{navItems.map(({ to, icon: Icon, label }) => (
<NavLink
key={to}
to={to}
className={({ isActive }) =>
`flex items-center gap-2 px-4 py-2 rounded-lg text-sm font-medium transition-all ${
isActive
? 'bg-gradient-to-r from-primary to-primary-dark text-white'
: 'text-text-secondary hover:text-text-primary hover:bg-bg-card-hover'
}`
}
>
<Icon size={16} />
{label}
</NavLink>
))}
</nav>
{/* User menu */}
<div className="flex items-center gap-4">
<div className="text-right hidden sm:block">
<div className="text-sm font-medium text-text-primary">{user?.email}</div>
<div className="text-xs text-text-secondary capitalize">{user?.role}</div>
</div>
<button
onClick={handleLogout}
className="flex items-center gap-2 px-3 py-2 rounded-lg text-text-secondary hover:text-error hover:bg-error/10 transition-all"
>
<LogOut size={18} />
<span className="hidden sm:inline">Logout</span>
</button>
</div>
</div>
</div>
</header>
{/* Mobile Navigation */}
<nav className="md:hidden fixed bottom-0 left-0 right-0 bg-bg-card/95 backdrop-blur-lg border-t border-primary/20 z-50">
<div className="flex justify-around py-2">
{navItems.map(({ to, icon: Icon, label }) => (
<NavLink
key={to}
to={to}
className={({ isActive }) =>
`flex flex-col items-center gap-1 px-3 py-2 rounded-lg text-xs font-medium transition-all ${
isActive ? 'text-primary' : 'text-text-secondary'
}`
}
>
<Icon size={20} />
{label}
</NavLink>
))}
</div>
</nav>
{/* Main content */}
<main className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8 pb-24 md:pb-8">
<Outlet />
</main>
</div>
);
}
@@ -0,0 +1,30 @@
import { Navigate, useLocation } from 'react-router-dom';
import { useAuth } from '../context/AuthContext';
interface ProtectedRouteProps {
children: React.ReactNode;
requireAdmin?: boolean;
}
export function ProtectedRoute({ children, requireAdmin = false }: ProtectedRouteProps) {
const { isAuthenticated, isAdmin, isLoading } = useAuth();
const location = useLocation();
if (isLoading) {
return (
<div className="min-h-screen bg-gradient-to-br from-bg-dark to-indigo-950 flex items-center justify-center">
<div className="animate-spin rounded-full h-12 w-12 border-t-2 border-b-2 border-primary"></div>
</div>
);
}
if (!isAuthenticated) {
return <Navigate to="/login" state={{ from: location }} replace />;
}
if (requireAdmin && !isAdmin) {
return <Navigate to="/analysis" replace />;
}
return <>{children}</>;
}
+81
View File
@@ -0,0 +1,81 @@
import { createContext, useContext, useState, useEffect, ReactNode } from 'react';
import { authApi, getAccessToken } from '../api/client';
import type { User } from '../types';
interface AuthContextType {
user: User | null;
isLoading: boolean;
isAuthenticated: boolean;
isAdmin: boolean;
login: (email: string, password: string) => Promise<void>;
register: (email: string, password: string) => Promise<void>;
logout: () => void;
refreshUser: () => Promise<void>;
}
const AuthContext = createContext<AuthContextType | undefined>(undefined);
export function AuthProvider({ children }: { children: ReactNode }) {
const [user, setUser] = useState<User | null>(null);
const [isLoading, setIsLoading] = useState(true);
const refreshUser = async () => {
try {
const userData = await authApi.getMe();
setUser(userData);
} catch {
setUser(null);
}
};
useEffect(() => {
const initAuth = async () => {
if (getAccessToken()) {
await refreshUser();
}
setIsLoading(false);
};
initAuth();
}, []);
const login = async (email: string, password: string) => {
await authApi.login(email, password);
await refreshUser();
};
const register = async (email: string, password: string) => {
await authApi.register(email, password);
await authApi.login(email, password);
await refreshUser();
};
const logout = () => {
authApi.logout();
setUser(null);
};
return (
<AuthContext.Provider
value={{
user,
isLoading,
isAuthenticated: !!user,
isAdmin: user?.role === 'admin',
login,
register,
logout,
refreshUser,
}}
>
{children}
</AuthContext.Provider>
);
}
export function useAuth() {
const context = useContext(AuthContext);
if (context === undefined) {
throw new Error('useAuth must be used within an AuthProvider');
}
return context;
}
+34
View File
@@ -0,0 +1,34 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}
/* Custom scrollbar */
::-webkit-scrollbar {
width: 8px;
height: 8px;
}
::-webkit-scrollbar-track {
background: #1e293b;
}
::-webkit-scrollbar-thumb {
background: #6366f1;
border-radius: 4px;
}
::-webkit-scrollbar-thumb:hover {
background: #4f46e5;
}
/* Selection */
::selection {
background: rgba(99, 102, 241, 0.3);
color: #f8fafc;
}
+10
View File
@@ -0,0 +1,10 @@
import { StrictMode } from 'react';
import { createRoot } from 'react-dom/client';
import App from './App';
import './index.css';
createRoot(document.getElementById('root')!).render(
<StrictMode>
<App />
</StrictMode>
);
+171
View File
@@ -0,0 +1,171 @@
import { useQuery } from '@tanstack/react-query';
import axios from 'axios';
import { Search, FileText, Bot, Zap, Globe, BarChart3, CheckCircle, AlertTriangle, XCircle } from 'lucide-react';
const API_BASE_URL = import.meta.env.VITE_API_URL || '/api';
export function About() {
const { data: health } = useQuery({
queryKey: ['health'],
queryFn: async () => {
const response = await axios.get(`${API_BASE_URL}/health`);
return response.data;
},
refetchInterval: 30000,
});
const features = [
{
icon: Search,
title: 'Patent Retrieval',
description: 'Automated collection via SerpAPI\'s Google Patents',
},
{
icon: FileText,
title: 'Intelligent Parsing',
description: 'Extracts key sections from patent documents',
},
{
icon: Bot,
title: 'AI Analysis',
description: 'Deep analysis powered by Claude 3.5 Sonnet',
},
{
icon: Zap,
title: 'Batch Processing',
description: 'Analyze multiple companies concurrently',
},
{
icon: Globe,
title: 'REST API',
description: 'FastAPI web service for seamless integration',
},
{
icon: BarChart3,
title: 'Analytics',
description: 'Track and visualize historical analysis data',
},
];
const techStack = [
{ label: 'Backend', value: 'Python, FastAPI' },
{ label: 'AI Model', value: 'Claude 3.5 Sonnet' },
{ label: 'Database', value: 'PostgreSQL' },
{ label: 'Frontend', value: 'React, TailwindCSS' },
{ label: 'Data Source', value: 'SerpAPI Patents' },
];
return (
<div className="space-y-8">
{/* Header */}
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
About SPARC
</h2>
</div>
<div className="grid grid-cols-1 lg:grid-cols-3 gap-8">
{/* Main Content */}
<div className="lg:col-span-2 space-y-6">
{/* Description */}
<p className="text-text-secondary leading-relaxed">
<strong className="text-text-primary">SPARC</strong> (Semiconductor Patent & Analytics Report Core)
is an AI-powered patent analysis platform that evaluates company performance by analyzing their
patent portfolios with cutting-edge language models.
</p>
{/* Features */}
<div>
<h3 className="text-lg font-semibold text-text-primary mb-4">Key Features</h3>
<div className="space-y-3">
{features.map(({ icon: Icon, title, description }) => (
<div
key={title}
className="flex items-start gap-4 py-3 border-b border-primary/10 last:border-0"
>
<div className="flex-shrink-0">
<Icon className="text-primary" size={20} />
</div>
<div>
<div className="font-medium text-text-primary">{title}</div>
<div className="text-sm text-text-secondary">{description}</div>
</div>
</div>
))}
</div>
</div>
</div>
{/* Sidebar */}
<div className="space-y-6">
{/* Tech Stack */}
<div className="bg-gradient-to-br from-primary/10 to-secondary/5 border border-primary/20 rounded-xl p-5">
<h3 className="font-semibold text-text-primary mb-4">Technology Stack</h3>
<div className="space-y-3">
{techStack.map(({ label, value }) => (
<div key={label}>
<div className="text-primary text-sm">{label}</div>
<div className="text-text-secondary text-sm">{value}</div>
</div>
))}
</div>
</div>
{/* API Endpoints */}
<div className="bg-bg-card/60 border border-primary/15 rounded-xl p-5">
<h3 className="font-semibold text-text-primary mb-4">API Endpoints</h3>
<div className="space-y-2">
<code className="block bg-bg-dark px-3 py-2 rounded text-sm text-text-secondary">
http://localhost:8000/docs
</code>
<code className="block bg-bg-dark px-3 py-2 rounded text-sm text-text-secondary">
http://localhost:8000/health
</code>
</div>
</div>
</div>
</div>
{/* System Status */}
<div>
<h3 className="text-lg font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-4">
System Status
</h3>
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
<StatusCard
label="API"
status={health ? 'online' : 'offline'}
/>
<StatusCard
label="Database"
status="configured"
/>
<StatusCard
label="Dashboard"
status="online"
/>
</div>
</div>
</div>
);
}
function StatusCard({ label, status }: { label: string; status: 'online' | 'offline' | 'configured' }) {
const statusConfig = {
online: { icon: CheckCircle, color: 'text-success', bg: 'bg-success' },
offline: { icon: XCircle, color: 'text-error', bg: 'bg-error' },
configured: { icon: AlertTriangle, color: 'text-warning', bg: 'bg-warning' },
};
const { icon: Icon, color, bg } = statusConfig[status];
return (
<div className="bg-gradient-to-br from-primary/10 to-secondary/10 border border-primary/20 rounded-xl p-5 text-center">
<div className={`inline-flex items-center justify-center w-8 h-8 rounded-full ${bg}/20 mb-2`}>
<Icon className={color} size={20} />
</div>
<div className="text-sm text-text-secondary uppercase tracking-wide">{label}</div>
<div className={`font-semibold ${color} capitalize`}>{status}</div>
</div>
);
}
+183
View File
@@ -0,0 +1,183 @@
import { useState } from 'react';
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { adminApi } from '../api/client';
import { useAuth } from '../context/AuthContext';
import { Users, Shield, User, Trash2, AlertCircle } from 'lucide-react';
import type { User as UserType } from '../types';
export function AdminUsers() {
const { user: currentUser } = useAuth();
const queryClient = useQueryClient();
const [deleteConfirm, setDeleteConfirm] = useState<number | null>(null);
const { data: users, isLoading, isError } = useQuery({
queryKey: ['admin-users'],
queryFn: () => adminApi.listUsers(),
});
const updateRoleMutation = useMutation({
mutationFn: ({ userId, role }: { userId: number; role: 'admin' | 'user' }) =>
adminApi.updateUserRole(userId, role),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['admin-users'] });
},
});
const deleteMutation = useMutation({
mutationFn: (userId: number) => adminApi.deleteUser(userId),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['admin-users'] });
setDeleteConfirm(null);
},
});
const handleRoleChange = (user: UserType) => {
const newRole = user.role === 'admin' ? 'user' : 'admin';
updateRoleMutation.mutate({ userId: user.id, role: newRole });
};
const handleDelete = (userId: number) => {
deleteMutation.mutate(userId);
};
if (isLoading) {
return (
<div className="flex items-center justify-center min-h-[400px]">
<div className="animate-spin rounded-full h-12 w-12 border-t-2 border-b-2 border-primary"></div>
</div>
);
}
if (isError) {
return (
<div className="flex items-center gap-2 bg-error/10 border border-error/20 text-error rounded-xl px-4 py-3">
<AlertCircle size={18} />
<span>Failed to load users.</span>
</div>
);
}
return (
<div className="space-y-6">
{/* Header */}
<div className="flex items-center justify-between">
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
User Management
</h2>
<p className="text-text-secondary">Manage user accounts and permissions.</p>
</div>
<div className="flex items-center gap-2 bg-primary/10 border border-primary/20 rounded-xl px-4 py-2">
<Users size={18} className="text-primary" />
<span className="text-text-primary font-semibold">{users?.length || 0} Users</span>
</div>
</div>
{/* Users Table */}
<div className="bg-bg-card/60 border border-primary/15 rounded-2xl overflow-hidden">
<div className="overflow-x-auto">
<table className="w-full">
<thead>
<tr className="border-b border-primary/10">
<th className="text-left px-6 py-4 text-sm font-semibold text-text-secondary uppercase tracking-wider">
User
</th>
<th className="text-left px-6 py-4 text-sm font-semibold text-text-secondary uppercase tracking-wider">
Role
</th>
<th className="text-left px-6 py-4 text-sm font-semibold text-text-secondary uppercase tracking-wider">
Created
</th>
<th className="text-right px-6 py-4 text-sm font-semibold text-text-secondary uppercase tracking-wider">
Actions
</th>
</tr>
</thead>
<tbody className="divide-y divide-primary/10">
{users?.map((user) => (
<tr key={user.id} className="hover:bg-bg-card-hover/50 transition-colors">
<td className="px-6 py-4">
<div className="flex items-center gap-3">
<div className="w-10 h-10 rounded-full bg-gradient-to-br from-primary/20 to-secondary/20 flex items-center justify-center">
{user.role === 'admin' ? (
<Shield className="text-primary" size={18} />
) : (
<User className="text-secondary" size={18} />
)}
</div>
<div>
<div className="font-medium text-text-primary">{user.email}</div>
{user.id === currentUser?.id && (
<span className="text-xs text-primary">(You)</span>
)}
</div>
</div>
</td>
<td className="px-6 py-4">
<span
className={`inline-flex items-center gap-1 px-3 py-1 rounded-full text-xs font-semibold uppercase ${
user.role === 'admin'
? 'bg-primary/20 text-primary border border-primary/30'
: 'bg-secondary/20 text-secondary border border-secondary/30'
}`}
>
{user.role === 'admin' ? <Shield size={12} /> : <User size={12} />}
{user.role}
</span>
</td>
<td className="px-6 py-4 text-text-secondary">
{new Date(user.created_at).toLocaleDateString()}
</td>
<td className="px-6 py-4">
<div className="flex items-center justify-end gap-2">
{user.id !== currentUser?.id && (
<>
<button
onClick={() => handleRoleChange(user)}
disabled={updateRoleMutation.isPending}
className={`px-3 py-1.5 rounded-lg text-sm font-medium transition-all ${
user.role === 'admin'
? 'bg-secondary/10 text-secondary hover:bg-secondary/20 border border-secondary/30'
: 'bg-primary/10 text-primary hover:bg-primary/20 border border-primary/30'
} disabled:opacity-50`}
>
{user.role === 'admin' ? 'Demote' : 'Promote'}
</button>
{deleteConfirm === user.id ? (
<div className="flex items-center gap-1">
<button
onClick={() => handleDelete(user.id)}
disabled={deleteMutation.isPending}
className="px-3 py-1.5 rounded-lg text-sm font-medium bg-error text-white hover:bg-error/80 transition-all disabled:opacity-50"
>
Confirm
</button>
<button
onClick={() => setDeleteConfirm(null)}
className="px-3 py-1.5 rounded-lg text-sm font-medium bg-bg-card-hover text-text-secondary hover:text-text-primary transition-all"
>
Cancel
</button>
</div>
) : (
<button
onClick={() => setDeleteConfirm(user.id)}
className="p-1.5 rounded-lg text-error/70 hover:text-error hover:bg-error/10 transition-all"
>
<Trash2 size={18} />
</button>
)}
</>
)}
</div>
</td>
</tr>
))}
</tbody>
</table>
</div>
</div>
</div>
);
}
+135
View File
@@ -0,0 +1,135 @@
import { useState } from 'react';
import { useMutation } from '@tanstack/react-query';
import { analysisApi } from '../api/client';
import { Search, CheckCircle, AlertCircle, Clock, FileText } from 'lucide-react';
import type { CompanyAnalysis } from '../types';
export function Analysis() {
const [companyName, setCompanyName] = useState('');
const [result, setResult] = useState<CompanyAnalysis | null>(null);
const mutation = useMutation({
mutationFn: (name: string) => analysisApi.analyzeCompany(name),
onSuccess: (data) => setResult(data),
});
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault();
if (companyName.trim()) {
mutation.mutate(companyName.trim());
}
};
return (
<div className="space-y-6">
{/* Header */}
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
Single Company Analysis
</h2>
<p className="text-text-secondary">
Analyze a company's patent portfolio using AI-powered insights.
</p>
</div>
{/* Search Form */}
<form onSubmit={handleSubmit} className="flex gap-4">
<div className="flex-1 relative">
<Search className="absolute left-4 top-1/2 -translate-y-1/2 text-text-secondary" size={18} />
<input
type="text"
value={companyName}
onChange={(e) => setCompanyName(e.target.value)}
placeholder="Enter company name (e.g., nvidia, intel, amd)"
className="w-full bg-bg-card/80 border border-primary/30 rounded-xl pl-12 pr-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all"
/>
</div>
<button
type="submit"
disabled={mutation.isPending || !companyName.trim()}
className="bg-gradient-to-r from-primary to-primary-dark text-white font-semibold py-3 px-6 rounded-xl hover:shadow-lg hover:shadow-primary/30 transition-all disabled:opacity-50 disabled:cursor-not-allowed flex items-center gap-2"
>
{mutation.isPending ? (
<div className="animate-spin rounded-full h-5 w-5 border-t-2 border-b-2 border-white"></div>
) : (
<>
<Search size={18} />
Analyze
</>
)}
</button>
</form>
{/* Error */}
{mutation.isError && (
<div className="flex items-center gap-2 bg-error/10 border border-error/20 text-error rounded-xl px-4 py-3">
<AlertCircle size={18} />
<span>Analysis failed. Please try again.</span>
</div>
)}
{/* Results */}
{result && (
<div className="space-y-6">
{/* Success/Failure Status */}
{result.success ? (
<div className="flex items-center gap-2 bg-success/10 border border-success/20 text-success rounded-xl px-4 py-3">
<CheckCircle size={18} />
<span>Analysis complete for {result.company_name.toUpperCase()}</span>
</div>
) : (
<div className="flex items-center gap-2 bg-error/10 border border-error/20 text-error rounded-xl px-4 py-3">
<AlertCircle size={18} />
<span>Analysis failed: {result.error}</span>
</div>
)}
{/* Metrics */}
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
<MetricCard
icon={FileText}
label="Patents Found"
value={result.patent_count.toString()}
/>
<MetricCard
icon={CheckCircle}
label="Analysis Status"
value={result.success ? 'Complete' : 'Failed'}
/>
<MetricCard
icon={Clock}
label="Timestamp"
value={new Date(result.timestamp).toLocaleTimeString()}
/>
</div>
{/* Analysis Content */}
{result.success && result.analysis && (
<div className="bg-bg-card/60 backdrop-blur-lg border border-primary/15 rounded-2xl p-6">
<h3 className="text-lg font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-4">
AI Analysis Results
</h3>
<div className="prose prose-invert max-w-none">
<div className="text-text-primary whitespace-pre-wrap leading-relaxed">
{result.analysis}
</div>
</div>
</div>
)}
</div>
)}
</div>
);
}
function MetricCard({ icon: Icon, label, value }: { icon: typeof FileText; label: string; value: string }) {
return (
<div className="bg-gradient-to-br from-primary/10 to-secondary/10 border border-primary/20 rounded-xl p-5 text-center">
<Icon className="mx-auto mb-2 text-primary" size={24} />
<div className="text-2xl font-bold bg-gradient-to-r from-primary to-secondary bg-clip-text text-transparent">
{value}
</div>
<div className="text-sm text-text-secondary uppercase tracking-wide mt-1">{label}</div>
</div>
);
}
+179
View File
@@ -0,0 +1,179 @@
import { useState } from 'react';
import { useQuery } from '@tanstack/react-query';
import { analyticsApi } from '../api/client';
import { AlertCircle, Database } from 'lucide-react';
import { PieChart, Pie, Cell, BarChart, Bar, XAxis, YAxis, Tooltip, ResponsiveContainer, Legend } from 'recharts';
const COLORS = ['#6366f1', '#0ea5e9', '#10b981', '#f59e0b', '#ef4444', '#8b5cf6', '#ec4899', '#14b8a6'];
export function AnalyticsPage() {
const [days, setDays] = useState(30);
const { data, isLoading, isError } = useQuery({
queryKey: ['analytics', days],
queryFn: () => analyticsApi.getAnalytics(days),
});
if (isLoading) {
return (
<div className="flex items-center justify-center min-h-[400px]">
<div className="animate-spin rounded-full h-12 w-12 border-t-2 border-b-2 border-primary"></div>
</div>
);
}
if (isError) {
return (
<div className="space-y-6">
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
Analytics Dashboard
</h2>
</div>
<div className="bg-gradient-to-br from-primary/10 to-secondary/5 border border-primary/20 rounded-xl p-6">
<div className="flex items-center gap-3 text-warning mb-2">
<Database size={24} />
<span className="font-semibold">Database Not Connected</span>
</div>
<p className="text-text-secondary">
Set <code className="bg-bg-card px-2 py-1 rounded">USE_DATABASE=true</code> in your .env file to enable analytics tracking.
</p>
</div>
<div className="flex items-center gap-2 bg-secondary/10 border border-secondary/20 text-secondary rounded-xl px-4 py-3">
<AlertCircle size={18} />
<span>Analytics features require storing analysis results in PostgreSQL for historical tracking.</span>
</div>
</div>
);
}
if (!data || (data.total_messages === 0 && data.by_company.length === 0)) {
return (
<div className="space-y-6">
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
Analytics Dashboard
</h2>
<p className="text-text-secondary">Track historical analysis data and view insights.</p>
</div>
<div className="flex items-center gap-2 bg-secondary/10 border border-secondary/20 text-secondary rounded-xl px-4 py-3">
<AlertCircle size={18} />
<span>No analytics data available yet. Run some analyses first!</span>
</div>
</div>
);
}
const companyData = data.by_company.map((c) => ({
name: (c.company_name || 'Unknown').toUpperCase(),
value: c.count,
}));
const typeData = data.by_type.map((t) => ({
name: t.analysis_type || 'Unknown',
count: t.count,
}));
return (
<div className="space-y-6">
{/* Header */}
<div className="flex flex-wrap items-center justify-between gap-4">
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
Analytics Dashboard
</h2>
<p className="text-text-secondary">Track historical analysis data and view insights.</p>
</div>
{/* Time Range Selector */}
<select
value={days}
onChange={(e) => setDays(Number(e.target.value))}
className="bg-bg-card/80 border border-primary/30 rounded-xl px-4 py-2 text-text-primary focus:outline-none focus:border-primary"
>
<option value={7}>Last 7 days</option>
<option value={14}>Last 14 days</option>
<option value={30}>Last 30 days</option>
<option value={90}>Last 90 days</option>
</select>
</div>
{/* Summary Metrics */}
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
<MetricCard label="Total Analyses" value={data.total_messages} />
<MetricCard label="Companies Analyzed" value={data.by_company.length} />
<MetricCard label="Analysis Types" value={data.by_type.length} />
</div>
{/* Charts */}
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6">
{/* Pie Chart - Distribution by Company */}
{companyData.length > 0 && (
<div className="bg-bg-card/60 border border-primary/15 rounded-2xl p-6">
<h3 className="text-lg font-semibold text-text-primary mb-4">Distribution by Company</h3>
<ResponsiveContainer width="100%" height={300}>
<PieChart>
<Pie
data={companyData}
cx="50%"
cy="50%"
innerRadius={60}
outerRadius={100}
paddingAngle={2}
dataKey="value"
label={({ name, percent }) => `${name} ${(percent * 100).toFixed(0)}%`}
labelLine={false}
>
{companyData.map((_, index) => (
<Cell key={`cell-${index}`} fill={COLORS[index % COLORS.length]} />
))}
</Pie>
<Tooltip
contentStyle={{
backgroundColor: '#1e293b',
border: '1px solid rgba(99, 102, 241, 0.3)',
borderRadius: '8px',
}}
/>
<Legend />
</PieChart>
</ResponsiveContainer>
</div>
)}
{/* Bar Chart - Analysis Types */}
{typeData.length > 0 && (
<div className="bg-bg-card/60 border border-primary/15 rounded-2xl p-6">
<h3 className="text-lg font-semibold text-text-primary mb-4">Analysis Types</h3>
<ResponsiveContainer width="100%" height={300}>
<BarChart data={typeData}>
<XAxis dataKey="name" stroke="#94a3b8" fontSize={12} />
<YAxis stroke="#94a3b8" fontSize={12} />
<Tooltip
contentStyle={{
backgroundColor: '#1e293b',
border: '1px solid rgba(99, 102, 241, 0.3)',
borderRadius: '8px',
}}
labelStyle={{ color: '#f8fafc' }}
/>
<Bar dataKey="count" fill="#6366f1" radius={[4, 4, 0, 0]} />
</BarChart>
</ResponsiveContainer>
</div>
)}
</div>
</div>
);
}
function MetricCard({ label, value }: { label: string; value: number }) {
return (
<div className="bg-gradient-to-br from-primary/10 to-secondary/10 border border-primary/20 rounded-xl p-5 text-center">
<div className="text-3xl font-bold bg-gradient-to-r from-primary to-secondary bg-clip-text text-transparent">
{value}
</div>
<div className="text-sm text-text-secondary uppercase tracking-wide mt-1">{label}</div>
</div>
);
}
+248
View File
@@ -0,0 +1,248 @@
import { useState } from 'react';
import { useMutation } from '@tanstack/react-query';
import { analysisApi } from '../api/client';
import { Rocket, CheckCircle, AlertCircle, ChevronDown, ChevronUp } from 'lucide-react';
import { BarChart, Bar, XAxis, YAxis, Tooltip, ResponsiveContainer, Cell } from 'recharts';
import type { BatchAnalysisResult } from '../types';
export function Batch() {
const [companiesInput, setCompaniesInput] = useState('');
const [maxWorkers, setMaxWorkers] = useState(3);
const [result, setResult] = useState<BatchAnalysisResult | null>(null);
const [expandedItems, setExpandedItems] = useState<Set<string>>(new Set());
const mutation = useMutation({
mutationFn: ({ companies, workers }: { companies: string[]; workers: number }) =>
analysisApi.analyzeBatch(companies, workers),
onSuccess: (data) => setResult(data),
});
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault();
const companies = companiesInput
.split(/[,\n]/)
.map((c) => c.trim())
.filter((c) => c.length > 0);
if (companies.length > 0) {
mutation.mutate({ companies, workers: maxWorkers });
}
};
const toggleExpand = (company: string) => {
const newExpanded = new Set(expandedItems);
if (newExpanded.has(company)) {
newExpanded.delete(company);
} else {
newExpanded.add(company);
}
setExpandedItems(newExpanded);
};
const chartData = result?.results.map((r) => ({
name: r.company_name.toUpperCase(),
patents: r.patent_count,
success: r.success,
}));
return (
<div className="space-y-6">
{/* Header */}
<div>
<h2 className="text-xl font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-2">
Batch Company Analysis
</h2>
<p className="text-text-secondary">
Analyze multiple companies simultaneously for comparative insights.
</p>
</div>
{/* Input Form */}
<form onSubmit={handleSubmit} className="grid grid-cols-1 md:grid-cols-3 gap-4">
<div className="md:col-span-2">
<textarea
value={companiesInput}
onChange={(e) => setCompaniesInput(e.target.value)}
placeholder="Enter company names (one per line or comma-separated):&#10;nvidia&#10;amd&#10;intel&#10;qualcomm"
rows={6}
className="w-full bg-bg-card/80 border border-primary/30 rounded-xl px-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all resize-none"
/>
</div>
<div className="space-y-4">
<div>
<label className="block text-sm font-medium text-text-secondary mb-2">
Concurrent Workers
</label>
<input
type="range"
min={1}
max={5}
value={maxWorkers}
onChange={(e) => setMaxWorkers(Number(e.target.value))}
className="w-full accent-primary"
/>
<div className="text-center text-text-primary font-semibold">{maxWorkers}</div>
</div>
<button
type="submit"
disabled={mutation.isPending || !companiesInput.trim()}
className="w-full bg-gradient-to-r from-primary to-primary-dark text-white font-semibold py-3 px-6 rounded-xl hover:shadow-lg hover:shadow-primary/30 transition-all disabled:opacity-50 disabled:cursor-not-allowed flex items-center justify-center gap-2"
>
{mutation.isPending ? (
<div className="animate-spin rounded-full h-5 w-5 border-t-2 border-b-2 border-white"></div>
) : (
<>
<Rocket size={18} />
Run Batch Analysis
</>
)}
</button>
</div>
</form>
{/* Progress */}
{mutation.isPending && (
<div className="bg-bg-card/60 border border-primary/15 rounded-xl p-4">
<div className="flex items-center gap-2 text-secondary">
<div className="animate-spin rounded-full h-4 w-4 border-t-2 border-b-2 border-secondary"></div>
<span>Analyzing companies...</span>
</div>
</div>
)}
{/* Error */}
{mutation.isError && (
<div className="flex items-center gap-2 bg-error/10 border border-error/20 text-error rounded-xl px-4 py-3">
<AlertCircle size={18} />
<span>Batch analysis failed. Please try again.</span>
</div>
)}
{/* Results */}
{result && (
<div className="space-y-6">
{/* Summary Metrics */}
<div>
<h3 className="text-lg font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-4">
Results Summary
</h3>
<div className="grid grid-cols-2 md:grid-cols-4 gap-4">
<SummaryCard label="Total Companies" value={result.total_companies} />
<SummaryCard label="Successful" value={result.successful} color="success" />
<SummaryCard label="Failed" value={result.failed} color="error" />
<SummaryCard
label="Success Rate"
value={`${Math.round((result.successful / result.total_companies) * 100)}%`}
/>
</div>
</div>
{/* Chart */}
{chartData && chartData.length > 0 && (
<div className="bg-bg-card/60 border border-primary/15 rounded-2xl p-6">
<ResponsiveContainer width="100%" height={300}>
<BarChart data={chartData}>
<XAxis dataKey="name" stroke="#94a3b8" fontSize={12} />
<YAxis stroke="#94a3b8" fontSize={12} />
<Tooltip
contentStyle={{
backgroundColor: '#1e293b',
border: '1px solid rgba(99, 102, 241, 0.3)',
borderRadius: '8px',
}}
labelStyle={{ color: '#f8fafc' }}
/>
<Bar dataKey="patents" radius={[4, 4, 0, 0]}>
{chartData.map((entry, index) => (
<Cell
key={`cell-${index}`}
fill={entry.success ? '#10b981' : '#ef4444'}
/>
))}
</Bar>
</BarChart>
</ResponsiveContainer>
</div>
)}
{/* Detailed Results */}
<div>
<h3 className="text-lg font-semibold text-text-primary border-b-2 border-primary/30 pb-2 mb-4">
Detailed Results
</h3>
<div className="space-y-3">
{result.results.map((r) => (
<div
key={r.company_name}
className="bg-bg-card/60 border border-primary/15 rounded-xl overflow-hidden"
>
<button
onClick={() => toggleExpand(r.company_name)}
className="w-full flex items-center justify-between p-4 hover:bg-bg-card-hover transition-colors"
>
<div className="flex items-center gap-3">
{r.success ? (
<CheckCircle className="text-success" size={20} />
) : (
<AlertCircle className="text-error" size={20} />
)}
<span className="font-semibold text-text-primary">
{r.company_name.toUpperCase()}
</span>
<span className="text-text-secondary">
{r.patent_count} patents
</span>
</div>
{expandedItems.has(r.company_name) ? (
<ChevronUp className="text-text-secondary" size={20} />
) : (
<ChevronDown className="text-text-secondary" size={20} />
)}
</button>
{expandedItems.has(r.company_name) && (
<div className="border-t border-primary/10 p-4 bg-bg-dark/40">
{r.success ? (
<div className="text-text-primary whitespace-pre-wrap leading-relaxed">
{r.analysis}
</div>
) : (
<div className="text-error">{r.error}</div>
)}
</div>
)}
</div>
))}
</div>
</div>
</div>
)}
</div>
);
}
function SummaryCard({
label,
value,
color,
}: {
label: string;
value: number | string;
color?: 'success' | 'error';
}) {
const colorClass = color === 'success' ? 'text-success' : color === 'error' ? 'text-error' : '';
return (
<div className="bg-gradient-to-br from-primary/10 to-secondary/10 border border-primary/20 rounded-xl p-4 text-center">
<div
className={`text-2xl font-bold ${
colorClass || 'bg-gradient-to-r from-primary to-secondary bg-clip-text text-transparent'
}`}
>
{value}
</div>
<div className="text-sm text-text-secondary uppercase tracking-wide mt-1">{label}</div>
</div>
);
}
+121
View File
@@ -0,0 +1,121 @@
import { useState } from 'react';
import { Link, useNavigate, useLocation } from 'react-router-dom';
import { useAuth } from '../context/AuthContext';
import { LogIn, Mail, Lock, AlertCircle } from 'lucide-react';
export function Login() {
const [email, setEmail] = useState('');
const [password, setPassword] = useState('');
const [error, setError] = useState('');
const [isLoading, setIsLoading] = useState(false);
const { login } = useAuth();
const navigate = useNavigate();
const location = useLocation();
const from = (location.state as { from?: { pathname: string } })?.from?.pathname || '/analysis';
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError('');
setIsLoading(true);
try {
await login(email, password);
navigate(from, { replace: true });
} catch (err) {
setError(err instanceof Error ? err.message : 'Invalid email or password');
} finally {
setIsLoading(false);
}
};
return (
<div className="min-h-screen bg-gradient-to-br from-bg-dark to-indigo-950 flex items-center justify-center px-4">
<div className="w-full max-w-md">
{/* Brand */}
<div className="text-center mb-8">
<div className="flex items-center justify-center gap-3 mb-4">
<span className="text-4xl"></span>
<h1 className="text-3xl font-bold bg-gradient-to-r from-primary to-secondary bg-clip-text text-transparent">
SPARC
</h1>
</div>
<p className="text-text-secondary">Semiconductor Patent Analytics Dashboard</p>
</div>
{/* Login Card */}
<div className="bg-bg-card/60 backdrop-blur-lg border border-primary/15 rounded-2xl p-8">
<h2 className="text-xl font-semibold text-text-primary mb-6">Sign in to your account</h2>
{error && (
<div className="flex items-center gap-2 bg-error/10 border border-error/20 text-error rounded-lg px-4 py-3 mb-6">
<AlertCircle size={18} />
<span className="text-sm">{error}</span>
</div>
)}
<form onSubmit={handleSubmit} className="space-y-5">
<div>
<label htmlFor="email" className="block text-sm font-medium text-text-secondary mb-2">
Email
</label>
<div className="relative">
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 text-text-secondary" size={18} />
<input
id="email"
type="email"
value={email}
onChange={(e) => setEmail(e.target.value)}
required
className="w-full bg-bg-dark/80 border border-primary/30 rounded-xl pl-10 pr-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all"
placeholder="you@example.com"
/>
</div>
</div>
<div>
<label htmlFor="password" className="block text-sm font-medium text-text-secondary mb-2">
Password
</label>
<div className="relative">
<Lock className="absolute left-3 top-1/2 -translate-y-1/2 text-text-secondary" size={18} />
<input
id="password"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
required
className="w-full bg-bg-dark/80 border border-primary/30 rounded-xl pl-10 pr-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all"
placeholder="Enter your password"
/>
</div>
</div>
<button
type="submit"
disabled={isLoading}
className="w-full bg-gradient-to-r from-primary to-primary-dark text-white font-semibold py-3 px-4 rounded-xl hover:shadow-lg hover:shadow-primary/30 transition-all disabled:opacity-50 disabled:cursor-not-allowed flex items-center justify-center gap-2"
>
{isLoading ? (
<div className="animate-spin rounded-full h-5 w-5 border-t-2 border-b-2 border-white"></div>
) : (
<>
<LogIn size={18} />
Sign In
</>
)}
</button>
</form>
<div className="mt-6 text-center">
<span className="text-text-secondary text-sm">Don't have an account? </span>
<Link to="/register" className="text-primary hover:text-primary-dark font-medium text-sm">
Sign up
</Link>
</div>
</div>
</div>
</div>
);
}
+153
View File
@@ -0,0 +1,153 @@
import { useState } from 'react';
import { Link, useNavigate } from 'react-router-dom';
import { useAuth } from '../context/AuthContext';
import { UserPlus, Mail, Lock, AlertCircle } from 'lucide-react';
export function Register() {
const [email, setEmail] = useState('');
const [password, setPassword] = useState('');
const [confirmPassword, setConfirmPassword] = useState('');
const [error, setError] = useState('');
const [isLoading, setIsLoading] = useState(false);
const { register } = useAuth();
const navigate = useNavigate();
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
setError('');
if (password !== confirmPassword) {
setError('Passwords do not match');
return;
}
if (password.length < 8) {
setError('Password must be at least 8 characters');
return;
}
setIsLoading(true);
try {
await register(email, password);
navigate('/analysis', { replace: true });
} catch (err) {
setError(err instanceof Error ? err.message : 'Registration failed');
} finally {
setIsLoading(false);
}
};
return (
<div className="min-h-screen bg-gradient-to-br from-bg-dark to-indigo-950 flex items-center justify-center px-4">
<div className="w-full max-w-md">
{/* Brand */}
<div className="text-center mb-8">
<div className="flex items-center justify-center gap-3 mb-4">
<span className="text-4xl"></span>
<h1 className="text-3xl font-bold bg-gradient-to-r from-primary to-secondary bg-clip-text text-transparent">
SPARC
</h1>
</div>
<p className="text-text-secondary">Semiconductor Patent Analytics Dashboard</p>
</div>
{/* Register Card */}
<div className="bg-bg-card/60 backdrop-blur-lg border border-primary/15 rounded-2xl p-8">
<h2 className="text-xl font-semibold text-text-primary mb-6">Create your account</h2>
{error && (
<div className="flex items-center gap-2 bg-error/10 border border-error/20 text-error rounded-lg px-4 py-3 mb-6">
<AlertCircle size={18} />
<span className="text-sm">{error}</span>
</div>
)}
<form onSubmit={handleSubmit} className="space-y-5">
<div>
<label htmlFor="email" className="block text-sm font-medium text-text-secondary mb-2">
Email
</label>
<div className="relative">
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 text-text-secondary" size={18} />
<input
id="email"
type="email"
value={email}
onChange={(e) => setEmail(e.target.value)}
required
className="w-full bg-bg-dark/80 border border-primary/30 rounded-xl pl-10 pr-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all"
placeholder="you@example.com"
/>
</div>
</div>
<div>
<label htmlFor="password" className="block text-sm font-medium text-text-secondary mb-2">
Password
</label>
<div className="relative">
<Lock className="absolute left-3 top-1/2 -translate-y-1/2 text-text-secondary" size={18} />
<input
id="password"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
required
minLength={8}
className="w-full bg-bg-dark/80 border border-primary/30 rounded-xl pl-10 pr-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all"
placeholder="At least 8 characters"
/>
</div>
</div>
<div>
<label htmlFor="confirmPassword" className="block text-sm font-medium text-text-secondary mb-2">
Confirm Password
</label>
<div className="relative">
<Lock className="absolute left-3 top-1/2 -translate-y-1/2 text-text-secondary" size={18} />
<input
id="confirmPassword"
type="password"
value={confirmPassword}
onChange={(e) => setConfirmPassword(e.target.value)}
required
className="w-full bg-bg-dark/80 border border-primary/30 rounded-xl pl-10 pr-4 py-3 text-text-primary placeholder-text-secondary/50 focus:outline-none focus:border-primary focus:ring-2 focus:ring-primary/20 transition-all"
placeholder="Confirm your password"
/>
</div>
</div>
<button
type="submit"
disabled={isLoading}
className="w-full bg-gradient-to-r from-primary to-primary-dark text-white font-semibold py-3 px-4 rounded-xl hover:shadow-lg hover:shadow-primary/30 transition-all disabled:opacity-50 disabled:cursor-not-allowed flex items-center justify-center gap-2"
>
{isLoading ? (
<div className="animate-spin rounded-full h-5 w-5 border-t-2 border-b-2 border-white"></div>
) : (
<>
<UserPlus size={18} />
Create Account
</>
)}
</button>
</form>
<div className="mt-6 text-center">
<span className="text-text-secondary text-sm">Already have an account? </span>
<Link to="/login" className="text-primary hover:text-primary-dark font-medium text-sm">
Sign in
</Link>
</div>
</div>
<p className="mt-6 text-center text-xs text-text-secondary">
The first registered user will automatically become an admin.
</p>
</div>
</div>
);
}
+46
View File
@@ -0,0 +1,46 @@
export interface User {
id: number;
email: string;
role: 'admin' | 'user';
created_at: string;
}
export interface TokenResponse {
access_token: string;
refresh_token: string;
token_type: string;
}
export interface CompanyAnalysis {
company_name: string;
analysis: string;
patent_count: number;
success: boolean;
error: string | null;
timestamp: string;
}
export interface BatchAnalysisResult {
results: CompanyAnalysis[];
total_companies: number;
successful: number;
failed: number;
timestamp: string;
}
export interface JobStatus {
job_id: string;
status: 'pending' | 'running' | 'completed' | 'failed';
progress: number;
total_companies: number;
completed_companies: number;
result: BatchAnalysisResult | null;
error: string | null;
}
export interface Analytics {
total_messages: number;
by_company: Array<{ company_name: string; count: number }>;
by_type: Array<{ analysis_type: string; count: number }>;
period_days: number;
}
+9
View File
@@ -0,0 +1,9 @@
/// <reference types="vite/client" />
interface ImportMetaEnv {
readonly VITE_API_URL: string;
}
interface ImportMeta {
readonly env: ImportMetaEnv;
}
+32
View File
@@ -0,0 +1,32 @@
/** @type {import('tailwindcss').Config} */
export default {
content: [
"./index.html",
"./src/**/*.{js,ts,jsx,tsx}",
],
theme: {
extend: {
colors: {
primary: {
DEFAULT: '#6366f1',
dark: '#4f46e5',
},
secondary: '#0ea5e9',
success: '#10b981',
warning: '#f59e0b',
error: '#ef4444',
bg: {
dark: '#0f172a',
card: '#1e293b',
'card-hover': '#334155',
},
text: {
primary: '#f8fafc',
secondary: '#94a3b8',
},
border: '#334155',
},
},
},
plugins: [],
}
+24
View File
@@ -0,0 +1,24 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"isolatedModules": true,
"moduleDetection": "force",
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true,
"baseUrl": ".",
"paths": {
"@/*": ["src/*"]
}
},
"include": ["src"]
}
+16
View File
@@ -0,0 +1,16 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
export default defineConfig({
plugins: [react()],
server: {
port: 3000,
proxy: {
'/api': {
target: 'http://localhost:8000',
changeOrigin: true,
rewrite: (path) => path.replace(/^\/api/, ''),
},
},
},
})
+3 -2
View File
@@ -8,8 +8,9 @@ openai
psycopg2-binary psycopg2-binary
fastapi fastapi
uvicorn[standard] uvicorn[standard]
pydantic[email]
httpx httpx
streamlit
plotly
numpy numpy
pandas pandas
bcrypt
PyJWT
+33
View File
@@ -8,6 +8,8 @@ Usage:
python scripts/init_database.py python scripts/init_database.py
""" """
import secrets
import string
import sys import sys
import os import os
@@ -17,6 +19,14 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from SPARC import config from SPARC import config
from SPARC.database import DatabaseClient from SPARC.database import DatabaseClient
DEFAULT_ADMIN_EMAIL = "admin@sparc.dev"
def generate_password(length: int = 16) -> str:
"""Generate a secure random password."""
alphabet = string.ascii_letters + string.digits
return "".join(secrets.choice(alphabet) for _ in range(length))
def main(): def main():
"""Initialize the database schema.""" """Initialize the database schema."""
@@ -29,9 +39,32 @@ def main():
print("Database schema initialized successfully!") print("Database schema initialized successfully!")
print("\nTables created:") print("\nTables created:")
print(" - llm_messages: Stores all LLM prompts and responses") print(" - llm_messages: Stores all LLM prompts and responses")
print(" - users: Stores user accounts")
print("\nIndexes created:") print("\nIndexes created:")
print(" - idx_messages_timestamp: For time-based queries") print(" - idx_messages_timestamp: For time-based queries")
print(" - idx_messages_company: For company-specific queries") print(" - idx_messages_company: For company-specific queries")
print(" - idx_users_email: For user lookups")
# Create default admin user if not exists
existing_admin = db_client.get_user_by_email(DEFAULT_ADMIN_EMAIL)
if existing_admin:
print(f"\nDefault admin user already exists: {DEFAULT_ADMIN_EMAIL}")
else:
password = generate_password()
admin_user = db_client.create_user(
email=DEFAULT_ADMIN_EMAIL,
password=password,
role="admin",
)
if admin_user:
print("\n" + "=" * 50)
print("DEFAULT ADMIN CREDENTIALS")
print("=" * 50)
print(f"Email: {DEFAULT_ADMIN_EMAIL}")
print(f"Password: {password}")
print("=" * 50)
print("Please save these credentials securely!")
print("=" * 50)
except Exception as e: except Exception as e:
print(f"Error initializing database: {e}") print(f"Error initializing database: {e}")
+79 -25
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Test script to verify database mode functionality. """Test script to verify database caching functionality.
This script tests the LLMAnalyzer in database mode without requiring This script tests the LLMAnalyzer with database caching without requiring
actual API keys or patent downloads. actual API keys or patent downloads.
""" """
@@ -9,28 +9,29 @@ from SPARC.llm import LLMAnalyzer
from SPARC.database import DatabaseClient from SPARC.database import DatabaseClient
from SPARC import config from SPARC import config
def test_database_mode(): def test_database_storage():
"""Test that database mode stores messages correctly.""" """Test that messages are always stored in database."""
print("Testing Database Mode") print("Testing Database Storage & Caching")
print("=" * 70) print("=" * 70)
# Initialize analyzer in database mode # Initialize analyzer (database is always used)
print("\n1. Initializing LLMAnalyzer in database mode...") print("\n1. Initializing LLMAnalyzer...")
analyzer = LLMAnalyzer(use_database=True) analyzer = LLMAnalyzer(use_cache=True)
print(f" - use_database: {analyzer.use_database}") print(f" - use_cache: {analyzer.use_cache}")
print(f" - db_client: {analyzer.db_client is not None}") print(f" - db_client: {analyzer.db_client is not None}")
print(f" - client (API): {analyzer.client is not None}")
# Test single patent analysis # Test single patent analysis (without API key, stores placeholder)
print("\n2. Testing single patent analysis (database mode)...") print("\n2. Testing single patent analysis (no API key)...")
result = analyzer.analyze_patent_content( result = analyzer.analyze_patent_content(
patent_content="Test patent content about semiconductor innovation", patent_content="Test patent content about semiconductor innovation",
company_name="TestCorp" company_name="TestCorp"
) )
print(f" Result: {result}") print(f" Result: {result[:80]}...")
# Test portfolio analysis # Test portfolio analysis
print("\n3. Testing portfolio analysis (database mode)...") print("\n3. Testing portfolio analysis (no API key)...")
test_patents = [ test_patents = [
{"patent_id": "US001", "content": "First test patent"}, {"patent_id": "US001", "content": "First test patent"},
{"patent_id": "US002", "content": "Second test patent"}, {"patent_id": "US002", "content": "Second test patent"},
@@ -39,7 +40,7 @@ def test_database_mode():
patents_data=test_patents, patents_data=test_patents,
company_name="TestCorp" company_name="TestCorp"
) )
print(f" Result: {result}") print(f" Result: {result[:80]}...")
# Verify messages were stored # Verify messages were stored
print("\n4. Verifying messages were stored...") print("\n4. Verifying messages were stored...")
@@ -48,7 +49,8 @@ def test_database_mode():
print(f" Found {len(messages)} stored messages") print(f" Found {len(messages)} stored messages")
for msg in messages: for msg in messages:
print(f" - ID: {msg['id']}, Type: {msg['analysis_type']}, Timestamp: {msg['timestamp']}") cached_status = "CACHED" if msg.get('is_cached') else "NEW"
print(f" - ID: {msg['id']}, Type: {msg['analysis_type']}, Status: {cached_status}")
# Get analytics # Get analytics
print("\n5. Getting analytics...") print("\n5. Getting analytics...")
@@ -58,18 +60,68 @@ def test_database_mode():
print(f" By type: {analytics['by_type']}") print(f" By type: {analytics['by_type']}")
print("\n" + "=" * 70) print("\n" + "=" * 70)
print("Database mode test completed successfully!") print("Database storage test completed successfully!")
def test_api_mode(): def test_caching():
"""Test that API mode initializes correctly.""" """Test that caching works correctly."""
print("\nTesting API Mode") print("\nTesting Cache Functionality")
print("=" * 70) print("=" * 70)
print("\n1. Initializing LLMAnalyzer in API mode...") db_client = DatabaseClient(config.database_url)
analyzer = LLMAnalyzer(use_database=False, test_mode=True) db_client.initialize_schema()
# Store a fake cached response
print("\n1. Storing a test response in database...")
test_prompt = "Test prompt for caching"
test_response = "This is a cached response from previous API call"
db_client.store_message(
prompt=test_prompt,
response=test_response,
company_name="CacheTest",
analysis_type="test",
model="test-model"
)
# Try to retrieve from cache
print("\n2. Testing cache retrieval...")
cached = db_client.get_cached_response(
prompt=test_prompt,
company_name="CacheTest",
analysis_type="test"
)
if cached:
print(f" Cache hit! Response: {cached['response']}")
else:
print(" Cache miss (unexpected)")
# Test cache miss
print("\n3. Testing cache miss...")
cached = db_client.get_cached_response(
prompt="Different prompt",
company_name="CacheTest",
analysis_type="test"
)
if cached:
print(" Unexpected cache hit")
else:
print(" Cache miss as expected")
print("\n" + "=" * 70)
print("Cache test completed successfully!")
def test_test_mode():
"""Test that test mode works correctly."""
print("\nTesting Test Mode")
print("=" * 70)
print("\n1. Initializing LLMAnalyzer in test mode...")
analyzer = LLMAnalyzer(test_mode=True)
print(f" - use_database: {analyzer.use_database}")
print(f" - test_mode: {analyzer.test_mode}") print(f" - test_mode: {analyzer.test_mode}")
print(f" - db_client: {analyzer.db_client is not None}")
print("\n2. Testing single patent analysis (test mode)...") print("\n2. Testing single patent analysis (test mode)...")
result = analyzer.analyze_patent_content( result = analyzer.analyze_patent_content(
@@ -79,9 +131,11 @@ def test_api_mode():
print(f" Result: {result}") print(f" Result: {result}")
print("\n" + "=" * 70) print("\n" + "=" * 70)
print("API mode test completed successfully!") print("Test mode test completed successfully!")
if __name__ == "__main__": if __name__ == "__main__":
test_database_mode() test_database_storage()
print("\n") print("\n")
test_api_mode() test_caching()
print("\n")
test_test_mode()
+191 -3
View File
@@ -1,11 +1,22 @@
"""Tests for the high-level company analyzer orchestration.""" """Tests for the high-level company analyzer orchestration."""
import pytest import pytest
from unittest.mock import Mock, patch, call from unittest.mock import Mock, patch, call, MagicMock
from SPARC.analyzer import CompanyAnalyzer from SPARC.analyzer import CompanyAnalyzer
from SPARC.types import Patent, Patents, CompanyAnalysisResult, BatchAnalysisResult from SPARC.types import Patent, Patents, CompanyAnalysisResult, BatchAnalysisResult
@pytest.fixture(autouse=True)
def mock_db(mocker):
"""Mock DatabaseClient for all tests so no real DB connection is needed."""
mock_db_cls = mocker.patch("SPARC.analyzer.DatabaseClient")
mock_db_instance = MagicMock()
mock_db_instance.get_cached_patent.return_value = None
mock_db_instance.get_cached_serp_query.return_value = None
mock_db_cls.return_value = mock_db_instance
return mock_db_instance
class TestCompanyAnalyzer: class TestCompanyAnalyzer:
"""Test the CompanyAnalyzer orchestration logic.""" """Test the CompanyAnalyzer orchestration logic."""
@@ -17,7 +28,7 @@ class TestCompanyAnalyzer:
mock_llm.assert_called_once_with(api_key="test-key") mock_llm.assert_called_once_with(api_key="test-key")
def test_analyze_company_full_pipeline(self, mocker): def test_analyze_company_full_pipeline(self, mocker, mock_db):
"""Test complete company analysis pipeline.""" """Test complete company analysis pipeline."""
# Mock all the dependencies # Mock all the dependencies
mock_query = mocker.patch("SPARC.analyzer.SERP.query") mock_query = mocker.patch("SPARC.analyzer.SERP.query")
@@ -178,6 +189,180 @@ class TestCompanyAnalyzer:
assert "PDF not found" in result assert "PDF not found" in result
class TestSingleQueryBugFix:
"""Test that SERP.query is only called once per company analysis."""
def test_analyze_company_safe_calls_query_once(self, mocker, mock_db):
"""_analyze_company_safe should call SERP.query exactly once."""
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
mock_query.return_value = Patents(patents=[patent])
def save_side_effect(p):
p.pdf_path = "patents/US123.pdf"
return p
mock_save.side_effect = save_side_effect
mock_parse.return_value = {"abstract": "Test"}
mock_minimize.return_value = "Content"
mock_llm_instance = Mock()
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
mock_llm.return_value = mock_llm_instance
analyzer = CompanyAnalyzer()
analyzer._analyze_company_safe("TestCorp")
# The key assertion: SERP.query called exactly once, not twice
mock_query.assert_called_once_with("TestCorp")
def test_analyze_company_with_prefetched_patents_skips_query(self, mocker):
"""analyze_company should not call SERP.query when patents are provided."""
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
prefetched = Patents(patents=[patent])
def save_side_effect(p):
p.pdf_path = "patents/US123.pdf"
return p
mock_save.side_effect = save_side_effect
mock_parse.return_value = {"abstract": "Test"}
mock_minimize.return_value = "Content"
mock_llm_instance = Mock()
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
mock_llm.return_value = mock_llm_instance
analyzer = CompanyAnalyzer()
analyzer.analyze_company("TestCorp", patents=prefetched)
# SERP.query should never be called
mock_query.assert_not_called()
class TestPatentCaching:
"""Test patent-level DB caching in the pipeline."""
def test_process_single_patent_uses_db_cache(self, mocker, mock_db):
"""_process_single_patent returns cached content when available."""
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
mock_db.get_cached_patent.return_value = {
"patent_id": "US123",
"minimized_content": "Cached minimized content",
}
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
result = CompanyAnalyzer._process_single_patent(patent, "TestCorp", mock_db)
assert result == {"patent_id": "US123", "content": "Cached minimized content"}
# Should NOT download since cache hit
mock_save.assert_not_called()
def test_process_single_patent_stores_to_db_cache(self, mocker, mock_db):
"""_process_single_patent stores result in DB after processing."""
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
# No cache hit
mock_db.get_cached_patent.return_value = None
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
def save_side_effect(p):
p.pdf_path = "patents/US123.pdf"
return p
mock_save.side_effect = save_side_effect
mock_parse.return_value = {"abstract": "Test abstract"}
mock_minimize.return_value = "Minimized content"
result = CompanyAnalyzer._process_single_patent(patent, "TestCorp", mock_db)
assert result == {"patent_id": "US123", "content": "Minimized content"}
mock_db.store_patent.assert_called_once_with(
patent_id="US123",
company_name="TestCorp",
pdf_link="http://example.com/test.pdf",
raw_sections={"abstract": "Test abstract"},
minimized_content="Minimized content",
)
def test_serp_query_cache_hit_skips_api(self, mocker, mock_db):
"""When SERP query is cached, API call is skipped."""
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
# Simulate SERP cache hit
mock_db.get_cached_serp_query.return_value = ["US123"]
# Simulate patent cache hit too
mock_db.get_cached_patent.return_value = {
"patent_id": "US123",
"minimized_content": "Cached content",
}
mock_llm_instance = Mock()
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
mock_llm.return_value = mock_llm_instance
analyzer = CompanyAnalyzer()
result = analyzer.analyze_company("TestCorp")
assert result == "Analysis"
# SERP.query should NOT be called
mock_query.assert_not_called()
# No downloads should happen
mock_save.assert_not_called()
def test_serp_query_cache_miss_stores_result(self, mocker, mock_db):
"""When SERP query cache misses, result is stored after API call."""
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
mock_db.get_cached_serp_query.return_value = None
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
mock_query.return_value = Patents(patents=[patent])
def save_side_effect(p):
p.pdf_path = "patents/US123.pdf"
return p
mock_save.side_effect = save_side_effect
mock_parse.return_value = {"abstract": "Test"}
mock_minimize.return_value = "Content"
mock_llm_instance = Mock()
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
mock_llm.return_value = mock_llm_instance
analyzer = CompanyAnalyzer()
analyzer.analyze_company("TestCorp")
mock_db.store_serp_query.assert_called_once()
call_kwargs = mock_db.store_serp_query.call_args[1]
assert call_kwargs["company_name"] == "TestCorp"
assert call_kwargs["patent_ids"] == ["US123"]
class TestBatchProcessing: class TestBatchProcessing:
"""Test multi-company batch processing functionality.""" """Test multi-company batch processing functionality."""
@@ -316,7 +501,7 @@ class TestBatchProcessing:
assert callback.call_count == 2 assert callback.call_count == 2
def test_company_analysis_result_structure(self, mocker): def test_company_analysis_result_structure(self, mocker, mock_db):
"""Test CompanyAnalysisResult has correct structure.""" """Test CompanyAnalysisResult has correct structure."""
mock_query = mocker.patch("SPARC.analyzer.SERP.query") mock_query = mocker.patch("SPARC.analyzer.SERP.query")
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents") mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
@@ -327,6 +512,9 @@ class TestBatchProcessing:
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf") patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
mock_query.return_value = Patents(patents=[patent]) mock_query.return_value = Patents(patents=[patent])
# Simulate DB caching: after store, subsequent get returns the IDs
mock_db.get_cached_serp_query.side_effect = [None, ["US123"]]
def save_side_effect(p): def save_side_effect(p):
p.pdf_path = "patents/US123.pdf" p.pdf_path = "patents/US123.pdf"
return p return p
+66 -10
View File
@@ -1,13 +1,22 @@
"""Tests for LLM analysis functionality.""" """Tests for LLM analysis functionality."""
import pytest import pytest
from unittest.mock import Mock, MagicMock from unittest.mock import Mock, MagicMock, patch
from SPARC.llm import LLMAnalyzer from SPARC.llm import LLMAnalyzer
class TestLLMAnalyzer: class TestLLMAnalyzer:
"""Test LLM analyzer initialization and API interaction.""" """Test LLM analyzer initialization and API interaction."""
@pytest.fixture(autouse=True)
def mock_database(self, mocker):
"""Mock the database client for all tests."""
mock_db_client = Mock()
mock_db_client.get_cached_response.return_value = None # No cache hit by default
mock_db_client.store_message.return_value = 1
mocker.patch("SPARC.llm.DatabaseClient", return_value=mock_db_client)
return mock_db_client
def test_analyzer_initialization_with_api_key(self, mocker): def test_analyzer_initialization_with_api_key(self, mocker):
"""Test that analyzer initializes with provided API key.""" """Test that analyzer initializes with provided API key."""
mock_openai = mocker.patch("SPARC.llm.OpenAI") mock_openai = mocker.patch("SPARC.llm.OpenAI")
@@ -25,7 +34,7 @@ class TestLLMAnalyzer:
mock_openai = mocker.patch("SPARC.llm.OpenAI") mock_openai = mocker.patch("SPARC.llm.OpenAI")
mock_config = mocker.patch("SPARC.llm.config") mock_config = mocker.patch("SPARC.llm.config")
mock_config.openrouter_api_key = "config-key-456" mock_config.openrouter_api_key = "config-key-456"
mock_config.use_database = False mock_config.use_cache = True
mock_config.database_url = "postgresql://localhost/test" mock_config.database_url = "postgresql://localhost/test"
analyzer = LLMAnalyzer() analyzer = LLMAnalyzer()
@@ -35,7 +44,7 @@ class TestLLMAnalyzer:
base_url="https://openrouter.ai/api/v1" base_url="https://openrouter.ai/api/v1"
) )
def test_analyze_patent_content(self, mocker): def test_analyze_patent_content(self, mocker, mock_database):
"""Test single patent content analysis.""" """Test single patent content analysis."""
mock_openai = mocker.patch("SPARC.llm.OpenAI") mock_openai = mocker.patch("SPARC.llm.OpenAI")
mock_client = Mock() mock_client = Mock()
@@ -44,9 +53,10 @@ class TestLLMAnalyzer:
# Mock the API response # Mock the API response
mock_response = Mock() mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content="Innovative GPU architecture."))] mock_response.choices = [Mock(message=Mock(content="Innovative GPU architecture."))]
mock_response.usage = Mock(prompt_tokens=100, completion_tokens=50, total_tokens=150)
mock_client.chat.completions.create.return_value = mock_response mock_client.chat.completions.create.return_value = mock_response
analyzer = LLMAnalyzer(api_key="test-key") analyzer = LLMAnalyzer(api_key="test-key", use_cache=False)
result = analyzer.analyze_patent_content( result = analyzer.analyze_patent_content(
patent_content="ABSTRACT: GPU with new cache design...", patent_content="ABSTRACT: GPU with new cache design...",
company_name="NVIDIA", company_name="NVIDIA",
@@ -61,7 +71,32 @@ class TestLLMAnalyzer:
assert "NVIDIA" in prompt_text assert "NVIDIA" in prompt_text
assert "GPU with new cache design" in prompt_text assert "GPU with new cache design" in prompt_text
def test_analyze_patent_portfolio(self, mocker): # Verify message was stored in database
mock_database.store_message.assert_called_once()
def test_analyze_patent_content_cache_hit(self, mocker, mock_database):
"""Test that cached responses are returned without API call."""
mock_openai = mocker.patch("SPARC.llm.OpenAI")
mock_client = Mock()
mock_openai.return_value = mock_client
# Set up cache hit
mock_database.get_cached_response.return_value = {
"id": 1,
"response": "Cached analysis result"
}
analyzer = LLMAnalyzer(api_key="test-key", use_cache=True)
result = analyzer.analyze_patent_content(
patent_content="ABSTRACT: GPU with new cache design...",
company_name="NVIDIA",
)
assert result == "Cached analysis result"
# API should NOT be called on cache hit
mock_client.chat.completions.create.assert_not_called()
def test_analyze_patent_portfolio(self, mocker, mock_database):
"""Test portfolio analysis with multiple patents.""" """Test portfolio analysis with multiple patents."""
mock_openai = mocker.patch("SPARC.llm.OpenAI") mock_openai = mocker.patch("SPARC.llm.OpenAI")
mock_client = Mock() mock_client = Mock()
@@ -72,9 +107,10 @@ class TestLLMAnalyzer:
mock_response.choices = [ mock_response.choices = [
Mock(message=Mock(content="Strong portfolio in AI and graphics.")) Mock(message=Mock(content="Strong portfolio in AI and graphics."))
] ]
mock_response.usage = Mock(prompt_tokens=200, completion_tokens=100, total_tokens=300)
mock_client.chat.completions.create.return_value = mock_response mock_client.chat.completions.create.return_value = mock_response
analyzer = LLMAnalyzer(api_key="test-key") analyzer = LLMAnalyzer(api_key="test-key", use_cache=False)
patents_data = [ patents_data = [
{"patent_id": "US123", "content": "AI acceleration patent"}, {"patent_id": "US123", "content": "AI acceleration patent"},
{"patent_id": "US456", "content": "Graphics rendering patent"}, {"patent_id": "US456", "content": "Graphics rendering patent"},
@@ -95,7 +131,7 @@ class TestLLMAnalyzer:
assert "AI acceleration patent" in prompt_text assert "AI acceleration patent" in prompt_text
assert "Graphics rendering patent" in prompt_text assert "Graphics rendering patent" in prompt_text
def test_analyze_patent_portfolio_with_correct_token_limit(self, mocker): def test_analyze_patent_portfolio_with_correct_token_limit(self, mocker, mock_database):
"""Test that portfolio analysis uses higher token limit.""" """Test that portfolio analysis uses higher token limit."""
mock_openai = mocker.patch("SPARC.llm.OpenAI") mock_openai = mocker.patch("SPARC.llm.OpenAI")
mock_client = Mock() mock_client = Mock()
@@ -103,9 +139,10 @@ class TestLLMAnalyzer:
mock_response = Mock() mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content="Analysis result."))] mock_response.choices = [Mock(message=Mock(content="Analysis result."))]
mock_response.usage = Mock(prompt_tokens=100, completion_tokens=50, total_tokens=150)
mock_client.chat.completions.create.return_value = mock_response mock_client.chat.completions.create.return_value = mock_response
analyzer = LLMAnalyzer(api_key="test-key") analyzer = LLMAnalyzer(api_key="test-key", use_cache=False)
patents_data = [{"patent_id": "US123", "content": "Test content"}] patents_data = [{"patent_id": "US123", "content": "Test content"}]
analyzer.analyze_patent_portfolio(patents_data, "TestCo") analyzer.analyze_patent_portfolio(patents_data, "TestCo")
@@ -114,7 +151,7 @@ class TestLLMAnalyzer:
# Portfolio analysis should use 2048 tokens # Portfolio analysis should use 2048 tokens
assert call_args[1]["max_tokens"] == 2048 assert call_args[1]["max_tokens"] == 2048
def test_analyze_single_patent_with_correct_token_limit(self, mocker): def test_analyze_single_patent_with_correct_token_limit(self, mocker, mock_database):
"""Test that single patent analysis uses lower token limit.""" """Test that single patent analysis uses lower token limit."""
mock_openai = mocker.patch("SPARC.llm.OpenAI") mock_openai = mocker.patch("SPARC.llm.OpenAI")
mock_client = Mock() mock_client = Mock()
@@ -122,11 +159,30 @@ class TestLLMAnalyzer:
mock_response = Mock() mock_response = Mock()
mock_response.choices = [Mock(message=Mock(content="Analysis result."))] mock_response.choices = [Mock(message=Mock(content="Analysis result."))]
mock_response.usage = Mock(prompt_tokens=100, completion_tokens=50, total_tokens=150)
mock_client.chat.completions.create.return_value = mock_response mock_client.chat.completions.create.return_value = mock_response
analyzer = LLMAnalyzer(api_key="test-key") analyzer = LLMAnalyzer(api_key="test-key", use_cache=False)
analyzer.analyze_patent_content("Test content", "TestCo") analyzer.analyze_patent_content("Test content", "TestCo")
call_args = mock_client.chat.completions.create.call_args call_args = mock_client.chat.completions.create.call_args
# Single patent should use 1024 tokens # Single patent should use 1024 tokens
assert call_args[1]["max_tokens"] == 1024 assert call_args[1]["max_tokens"] == 1024
def test_database_always_initialized(self, mocker, mock_database):
"""Test that database client is always initialized."""
mock_openai = mocker.patch("SPARC.llm.OpenAI")
analyzer = LLMAnalyzer(api_key="test-key")
assert analyzer.db_client is not None
def test_no_api_key_stores_placeholder(self, mocker, mock_database):
"""Test that without API key, a placeholder is stored."""
mocker.patch("SPARC.llm.config")
analyzer = LLMAnalyzer(use_cache=False)
result = analyzer.analyze_patent_content("Test content", "TestCo")
assert "[NO API]" in result
mock_database.store_message.assert_called_once()
+90
View File
@@ -1,7 +1,11 @@
"""Tests for SERP API patent retrieval and parsing functionality.""" """Tests for SERP API patent retrieval and parsing functionality."""
import os
import pytest import pytest
from unittest.mock import patch, Mock
from datetime import datetime, timedelta
from SPARC.serp_api import SERP from SPARC.serp_api import SERP
from SPARC.types import Patent
class TestTextCleaning: class TestTextCleaning:
@@ -176,3 +180,89 @@ class TestPatentMinimization:
# Sections should be separated by double newlines # Sections should be separated by double newlines
assert "\n\n" in result assert "\n\n" in result
class TestDynamicDateRange:
"""Test dynamic date range computation in SERP.query."""
def test_query_uses_rolling_date_window(self, mocker):
"""Verify the date filter uses a rolling window, not hardcoded dates."""
mock_search = mocker.patch("SPARC.serp_api.serpapi.search")
mock_search.return_value = {"organic_results": []}
mocker.patch("SPARC.serp_api.config.api_key", "fake-key")
mocker.patch("SPARC.serp_api.config.patent_search_days", 90)
SERP.query("TestCorp")
call_params = mock_search.call_args[0][0]
tbs = call_params["tbs"]
# Should contain "cdr:1,cd_min:" with a date, not the old hardcoded one
assert "cdr:1,cd_min:" in tbs
assert "10/28/2025" not in tbs # old hardcoded date gone
def test_query_respects_days_back_param(self, mocker):
"""Verify days_back parameter controls the date window."""
mock_search = mocker.patch("SPARC.serp_api.serpapi.search")
mock_search.return_value = {"organic_results": []}
mocker.patch("SPARC.serp_api.config.api_key", "fake-key")
mocker.patch("SPARC.serp_api.config.patent_search_days", 90)
now = datetime.now()
SERP.query("TestCorp", days_back=30)
call_params = mock_search.call_args[0][0]
tbs = call_params["tbs"]
expected_start = (now - timedelta(days=30)).strftime("%-m/%-d/%Y")
assert expected_start in tbs
class TestFilesystemPDFCaching:
"""Test that save_patents skips download for existing files."""
def test_save_patents_skips_download_when_cached(self, mocker, tmp_path):
"""Already-downloaded PDFs should not be re-downloaded."""
mock_get = mocker.patch("SPARC.serp_api.requests.get")
mocker.patch("SPARC.serp_api.os.makedirs")
pdf_path = tmp_path / "US123.pdf"
pdf_path.write_bytes(b"%PDF-1.4 fake content")
mocker.patch("SPARC.serp_api.os.path.exists", return_value=True)
mocker.patch("SPARC.serp_api.os.path.getsize", return_value=100)
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
result = SERP.save_patents(patent)
mock_get.assert_not_called()
assert result.pdf_path == "patents/US123.pdf"
def test_save_patents_downloads_when_not_cached(self, mocker):
"""Missing PDFs should be downloaded."""
mock_response = Mock()
mock_response.content = b"%PDF-1.4 content"
mock_get = mocker.patch("SPARC.serp_api.requests.get", return_value=mock_response)
mocker.patch("SPARC.serp_api.os.makedirs")
mocker.patch("SPARC.serp_api.os.path.exists", return_value=False)
mock_open = mocker.patch("builtins.open", mocker.mock_open())
patent = Patent(patent_id="US456", pdf_link="http://example.com/test.pdf")
result = SERP.save_patents(patent)
mock_get.assert_called_once_with("http://example.com/test.pdf")
assert result.pdf_path == "patents/US456.pdf"
def test_save_patents_redownloads_empty_files(self, mocker):
"""Empty/corrupt PDFs (0 bytes) should be re-downloaded."""
mock_response = Mock()
mock_response.content = b"%PDF-1.4 content"
mock_get = mocker.patch("SPARC.serp_api.requests.get", return_value=mock_response)
mocker.patch("SPARC.serp_api.os.makedirs")
mocker.patch("SPARC.serp_api.os.path.exists", return_value=True)
mocker.patch("SPARC.serp_api.os.path.getsize", return_value=0)
mock_open = mocker.patch("builtins.open", mocker.mock_open())
patent = Patent(patent_id="US789", pdf_link="http://example.com/test.pdf")
result = SERP.save_patents(patent)
mock_get.assert_called_once()
assert result.pdf_path == "patents/US789.pdf"