SPARC/docs/DEPLOYMENT.md

SPARC Complete Deployment Guide

This guide provides step-by-step instructions for deploying the SPARC (Semiconductor Patent & Analytics Report Core) application with all features enabled, including SERP API patent retrieval, LLM analysis, database storage, and the web UI.

Table of Contents

• Prerequisites
• Step 1: Clone and Configure
• Step 2: Start Services with Docker Compose
• Step 3: Initialize the Database
• Step 4: Run the Services
• Step 5: Verify Deployment
• Step 6: Using the Application
• Step 7: View Stored Data
• Architecture Overview
• Environment Variables Reference
• Production Docker Compose
• Troubleshooting

---

Prerequisites

1. Docker & Docker Compose installed
2. API Keys (you'll need to obtain these):
   • SerpAPI Key: Sign up at https://serpapi.com/ (free tier: 100 searches/month)
   • OpenRouter API Key: Sign up at https://openrouter.ai/ (pay-as-you-go)

---

Step 1: Clone and Configure

git clone <repository-url>
cd SPARC

# Create environment file
cp .env.example .env

Edit .env with your API keys:

# Required API Keys
API_KEY=your_serpapi_key_here
OPENROUTER_API_KEY=your_openrouter_key_here

# Database Configuration (matches docker-compose.yml)
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc
USE_DATABASE=true

---

Step 2: Start Services with Docker Compose

# Start all services (PostgreSQL, API, and Dashboard)
docker-compose up -d

# Check status
docker-compose ps

# You should see:
# - sparc-postgres (healthy)
# - sparc-api (running on port 8000)
# - sparc-dashboard (running on port 8501)

The database is automatically initialized by the init-db service.

---

Step 3: Database Schema

The init-db service automatically creates the llm_messages table with the following schema:

Column	Type	Purpose
id	SERIAL	Primary key
timestamp	TIMESTAMP	Message creation time
company_name	VARCHAR(255)	Company being analyzed
analysis_type	VARCHAR(50)	'single_patent' or 'portfolio'
model	VARCHAR(100)	LLM model identifier
prompt	TEXT	Full prompt sent to LLM
response	TEXT	LLM response
metadata	JSONB	Patent IDs, content lengths
token_usage	JSONB	prompt/completion/total tokens
created_at	TIMESTAMP	Record timestamp

---

Step 4: Run the Services

Option A: Run with Docker Compose (Recommended)

All services are started automatically with docker-compose up -d from Step 2.

# View logs
docker-compose logs -f

# View specific service logs
docker-compose logs -f api
docker-compose logs -f dashboard

Option B: Run Locally (Development)

If you prefer running services locally without Docker:

# Start PostgreSQL with Docker
docker-compose up -d postgres

# Wait for database to be healthy, then initialize
python scripts/init_database.py

# Terminal 1: Start FastAPI backend
uvicorn SPARC.api:app --host 0.0.0.0 --port 8000 --reload

# Terminal 2: Start Streamlit dashboard
streamlit run dashboard.py --server.port 8501 --server.address 0.0.0.0

---

Step 5: Verify Deployment

# Check API health
curl http://localhost:8000/health

# Expected response:
# {"status":"healthy","version":"0.1.0","timestamp":"..."}

Access the services:

Service	URL
REST API	http://localhost:8000
API Documentation (Swagger)	http://localhost:8000/docs
Dashboard (Web UI)	http://localhost:8501

---

Step 6: Using the Application

Via Dashboard (Web UI)

1. Open http://localhost:8501
2. Select "Company Analysis" from the sidebar
3. Enter a company name (e.g., "Intel")
4. Click "Analyze"

This will:

• Query SerpAPI for recent patents
• Download and parse patent PDFs
• Send patent content to Claude for analysis
• Store prompt/response in PostgreSQL
• Display results in the dashboard

Via REST API

# Analyze single company
curl http://localhost:8000/analyze/Intel

# Batch analyze multiple companies (synchronous)
curl -X POST http://localhost:8000/analyze/batch \
  -H "Content-Type: application/json" \
  -d '{"companies": ["Intel", "AMD", "NVIDIA"], "max_workers": 3}'

# Async batch (for large jobs)
curl -X POST http://localhost:8000/analyze/batch/async \
  -H "Content-Type: application/json" \
  -d '{"companies": ["Intel", "AMD"]}'

# Check job status
curl http://localhost:8000/jobs/{job_id}

# List all jobs
curl http://localhost:8000/jobs

Via Python

from SPARC.analyzer import CompanyAnalyzer

analyzer = CompanyAnalyzer()
result = analyzer.analyze("Intel")
print(result.analysis)

---

Step 7: View Stored Data

# View analytics (aggregated usage)
python scripts/view_analytics.py

# View stored messages
python scripts/view_messages.py

# Query database directly
docker exec -it sparc-postgres psql -U postgres -d sparc -c \
  "SELECT company_name, analysis_type, token_usage FROM llm_messages ORDER BY timestamp DESC LIMIT 10;"

---

Architecture Overview

┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│  Dashboard   │───▶│   FastAPI    │───▶│   Analyzer   │
│  (8501)      │    │   (8000)     │    │              │
└──────────────┘    └──────────────┘    └──────┬───────┘
                                               │
                    ┌──────────────────────────┼──────────────────────────┐
                    │                          │                          │
                    ▼                          ▼                          ▼
           ┌──────────────┐           ┌──────────────┐           ┌──────────────┐
           │   SerpAPI    │           │  OpenRouter  │           │  PostgreSQL  │
           │ (Patents)    │           │  (Claude)    │           │  (Storage)   │
           └──────────────┘           └──────────────┘           └──────────────┘

Component Responsibilities

Component	Purpose
Dashboard	Streamlit web UI for interactive analysis
FastAPI	REST API for programmatic access
Analyzer	Orchestrates patent retrieval and LLM analysis
SerpAPI	Retrieves patent data from Google Patents
OpenRouter	Routes requests to Claude for AI analysis
PostgreSQL	Stores prompts, responses, and analytics

---

Environment Variables Reference

Variable	Required	Default	Description
API_KEY	Yes	-	SerpAPI key for patent search
OPENROUTER_API_KEY	Yes	-	OpenRouter API key for Claude access
DATABASE_URL	Yes*	-	PostgreSQL connection string
USE_DATABASE	No	false	Set to true to enable database storage

*Required when USE_DATABASE=true

Database URL Format

postgresql://[user]:[password]@[host]:[port]/[database]

Example:

postgresql://postgres:postgres@localhost:5432/sparc

---

Docker Compose Services

The docker-compose.yml includes all services needed for production:

Service	Container	Port	Description
postgres	sparc-postgres	5432	PostgreSQL database
init-db	sparc-init-db	-	One-time database initialization
api	sparc-api	8000	FastAPI REST API
dashboard	sparc-dashboard	8501	Streamlit web UI

Common Docker Compose Commands

# Start all services
docker-compose up -d

# Start with rebuild (after code changes)
docker-compose up -d --build

# View logs
docker-compose logs -f

# View specific service logs
docker-compose logs -f api
docker-compose logs -f dashboard

# Stop all services
docker-compose down

# Stop and remove volumes (WARNING: deletes data)
docker-compose down -v

# Restart a specific service
docker-compose restart api

---

Troubleshooting

Database Connection Issues

# Check if postgres is running
docker-compose ps

# Check postgres logs
docker-compose logs postgres

# Test database connection
docker exec -it sparc-postgres psql -U postgres -d sparc -c "SELECT 1;"

API Key Issues

# Verify environment variables are set
echo $API_KEY
echo $OPENROUTER_API_KEY

# Test SerpAPI directly
curl "https://serpapi.com/search?engine=google_patents&q=Intel&api_key=$API_KEY"

Port Conflicts

If ports 8000, 8501, or 5432 are in use:

# Find what's using the port
lsof -i :8000

# Or change ports in docker-compose.yml
ports:
  - "8080:8000"  # Use 8080 instead of 8000

Container Issues

# Rebuild containers after code changes
docker-compose build --no-cache

# Remove all containers and start fresh
docker-compose down
docker-compose up -d --build

Viewing Application Logs

# All services
docker-compose logs -f

# Specific service
docker-compose logs -f api
docker-compose logs -f dashboard

---

Quick Reference

# Docker setup (recommended)
cp .env.example .env
# Edit .env with API keys
docker-compose up -d

# Local development setup
cp .env.example .env
# Edit .env with API keys
docker-compose up -d postgres
python scripts/init_database.py
uvicorn SPARC.api:app --reload &
streamlit run dashboard.py

# Check status
curl http://localhost:8000/health
open http://localhost:8501

# View data
python scripts/view_analytics.py
python scripts/view_messages.py