0xWheatyz/SPARC
1
0
Fork 2
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: update documentation for React frontend and cache mode

Browse Source 
Update all documentation to reflect recent changes:
- Replace Streamlit dashboard references with React TypeScript dashboard
- Update dashboard port from 8501 to 8080
- Add auth.py and database.py to architecture section
- Change USE_DATABASE terminology to USE_CACHE
- Add JWT_SECRET to environment variables reference
- Document default admin credentials and user seeding

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
0xWheatyz
2026-03-14 14:30:21 -04:00
parent ebba983a1d
commit ed81ae4569
3 changed files with 96 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -11
View File
@@ -17,7 +17,7 @@ SPARC automatically collects, parses, and analyzes patents from companies to pro
- **Portfolio Analysis**: Evaluates multiple patents holistically for comprehensive insights - **Portfolio Analysis**: Evaluates multiple patents holistically for comprehensive insights
- **Batch Processing**: Analyze multiple companies concurrently with progress tracking - **Batch Processing**: Analyze multiple companies concurrently with progress tracking
- **REST API**: FastAPI web service with async job support - **REST API**: FastAPI web service with async job support
- **Dashboard**: Interactive Streamlit visualization dashboard - **Dashboard**: React TypeScript web dashboard with authentication
- **Robust Testing**: 40 tests covering all major functionality - **Robust Testing**: 40 tests covering all major functionality
## Architecture ## Architecture
@@ -27,7 +27,9 @@ SPARC/
├── serp_api.py # Patent retrieval and PDF parsing ├── serp_api.py # Patent retrieval and PDF parsing
├── llm.py # Claude AI integration via OpenRouter ├── llm.py # Claude AI integration via OpenRouter
├── analyzer.py # High-level orchestration ├── analyzer.py # High-level orchestration
├── api.py # FastAPI web service ├── api.py # FastAPI web service with auth endpoints
├── auth.py # JWT authentication module
├── database.py # PostgreSQL storage with caching
├── types.py # Data models ├── types.py # Data models
└── config.py # Environment configuration └── config.py # Environment configuration
``` ```
@@ -48,7 +50,7 @@ docker-compose up -d
# Access the services # Access the services
# - API: http://localhost:8000 # - API: http://localhost:8000
# - Dashboard: http://localhost:8501 # - Dashboard: http://localhost:8080
# - API Docs: http://localhost:8000/docs # - API Docs: http://localhost:8000/docs
``` ```
@@ -186,21 +188,22 @@ curl -X POST http://localhost:8000/analyze/batch/async \
-d '{"companies": ["nvidia", "amd", "intel", "qualcomm"]}' -d '{"companies": ["nvidia", "amd", "intel", "qualcomm"]}'
``` ```
### Visualization Dashboard ### Web Dashboard
Launch the interactive Streamlit dashboard: The React dashboard is included in Docker Compose:
```bash ```bash
streamlit run dashboard.py docker-compose up -d
``` ```
Dashboard features: Dashboard features:
- **Authentication**: User registration, login, and JWT-based sessions
- **Company Analysis**: Analyze individual companies with real-time results - **Company Analysis**: Analyze individual companies with real-time results
- **Batch Analysis**: Process multiple companies with progress tracking and charts - **Batch Analysis**: Process multiple companies with progress tracking
- **Analytics**: View historical analysis data and trends (requires database mode) - **Analytics**: View historical analysis data and trends
- **System Status**: Monitor database and analyzer health - **Admin Panel**: User management for administrators
The dashboard runs at `http://localhost:8501` by default. The dashboard runs at `http://localhost:8080` when using Docker Compose.
## Running Tests ## Running Tests
@@ -280,4 +283,4 @@ For open source projects, say how it is licensed.
Core functionality complete. Ready for production use with API keys configured. Core functionality complete. Ready for production use with API keys configured.
All major features implemented: REST API, Streamlit dashboard, Docker containerization, database storage, and multi-company batch processing. All major features implemented: REST API, React dashboard with authentication, Docker containerization, database storage with caching, and multi-company batch processing.
docs/DATABASE_MODE.md
+58 -45
View File
@@ -1,16 +1,19 @@
# Database Mode for Testing and Analytics # Database Storage and Caching
This document explains how to use SPARC's database mode for storing LLM messages for testing and analytics purposes. This document explains how SPARC uses PostgreSQL for storing LLM messages, enabling response caching and analytics.
## Overview ## Overview
SPARC supports two modes of operation: SPARC stores all LLM interactions in PostgreSQL, providing:
1. **API Mode** (default): Messages are sent to OpenRouter's API and you receive real LLM responses - **Response Caching**: Avoid redundant API calls for previously analyzed patents
2. **Database Mode**: Messages are stored in a PostgreSQL database without making API calls, useful for: - **Analytics**: Track usage patterns, token consumption, and analysis history
- Testing the application without consuming API credits - **Persistence**: Maintain analysis history across sessions
- Collecting analytics on message patterns and usage
- Development and debugging SPARC supports two cache modes:
1. **Cache Mode** (default, `USE_CACHE=true`): Check database for cached responses before making API calls
2. **Fresh Mode** (`USE_CACHE=false`): Always make fresh API calls (still stores results in database)
## Setup ## Setup
@@ -45,43 +48,43 @@ cp .env.example .env
Edit `.env` and set: Edit `.env` and set:
```env ```env
# For database mode (testing/analytics) # Database connection (required)
USE_DATABASE=true
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc
# For API mode (production) # Cache mode: use cached responses when available
USE_DATABASE=false USE_CACHE=true
# API key for fresh LLM calls
OPENROUTER_API_KEY=your_openrouter_key_here OPENROUTER_API_KEY=your_openrouter_key_here
``` ```
## Usage ## Usage
### Running in Database Mode ### Running with Cache Mode (Default)
Set `USE_DATABASE=true` in your `.env` file, then run the application normally: Set `USE_CACHE=true` in your `.env` file, then run the application normally:
```bash ```bash
python main.py python main.py
``` ```
Instead of sending messages to OpenRouter, the application will: The application will:
- Store all prompts in the database - Check the database for cached responses matching the request
- Return a placeholder response - If found, return the cached response (no API call)
- Log metadata (company name, analysis type, timestamps) - If not found, make an API call and store the response for future use
### Running in API Mode ### Running with Fresh Mode
Set `USE_DATABASE=false` in your `.env` file, then run the application normally: Set `USE_CACHE=false` in your `.env` file to always get fresh responses:
```bash ```bash
python main.py python main.py
``` ```
The application will send messages to OpenRouter and return real LLM responses. The application will:
- Always send messages to OpenRouter for real LLM responses
### Hybrid Mode (Optional) - Store all responses in the database
- Useful when you need the latest analysis or want to refresh cached data
You can also enable database logging while still using the API by initializing the database client in your code. The `LLMAnalyzer` will automatically log all API calls to the database if a database client is available.
## Viewing Analytics ## Viewing Analytics
@@ -195,16 +198,16 @@ docker-compose down -v
## Toggling Between Modes ## Toggling Between Modes
You can easily switch between modes by changing the `USE_DATABASE` environment variable: You can easily switch between modes by changing the `USE_CACHE` environment variable:
### Quick Toggle (temporary, for testing) ### Quick Toggle (temporary)
```bash ```bash
# Run in database mode # Run with caching enabled
USE_DATABASE=true python main.py USE_CACHE=true python main.py
# Run in API mode # Run with fresh API calls
USE_DATABASE=false python main.py USE_CACHE=false python main.py
``` ```
### Persistent Toggle ### Persistent Toggle
@@ -212,38 +215,48 @@ USE_DATABASE=false python main.py
Edit your `.env` file: Edit your `.env` file:
```env ```env
# For testing/analytics # Use cached responses when available (recommended for most use)
USE_DATABASE=true USE_CACHE=true
# For production use # Always make fresh API calls
USE_DATABASE=false USE_CACHE=false
``` ```
## Use Cases ## Use Cases
### Testing Without API Costs ### Cost Optimization with Caching
During development, enable database mode to test the full application flow without consuming API credits: Cache mode reduces API costs by reusing previous analysis results:
```bash ```bash
USE_DATABASE=true python main.py USE_CACHE=true python main.py
```
If the same company/patent combination was analyzed before, the cached response is returned instantly.
### Fresh Analysis
When you need the latest LLM analysis (e.g., after model updates):
```bash
USE_CACHE=false python main.py
``` ```
### Collecting Usage Analytics ### Collecting Usage Analytics
Enable database mode in a test environment to collect analytics on: The database stores all interactions, enabling analytics on:
- Which companies are analyzed most frequently - Which companies are analyzed most frequently
- Types of analyses performed - Types of analyses performed
- Prompt patterns and lengths - Token usage and costs over time
- Usage over time - Response caching hit rates
### Development and Debugging ### Development and Debugging
Database mode is useful for: Database storage is useful for:
- Testing patent parsing logic without API calls - Reviewing actual prompts sent to the LLM
- Analyzing response patterns
- Debugging the full pipeline end-to-end - Debugging the full pipeline end-to-end
- Collecting sample prompts for optimization - Understanding token usage patterns
- Understanding token usage patterns (when in API mode with logging)
## Troubleshooting ## Troubleshooting
docs/DEPLOYMENT.md
+24 -22
View File
@@ -64,7 +64,7 @@ docker-compose ps
# You should see: # You should see:
# - sparc-postgres (healthy) # - sparc-postgres (healthy)
# - sparc-api (running on port 8000) # - sparc-api (running on port 8000)
# - sparc-dashboard (running on port 8501) # - sparc-dashboard (running on port 8080)
``` ```
The database is automatically initialized by the `init-db` service. The database is automatically initialized by the `init-db` service.
@@ -116,11 +116,13 @@ docker-compose up -d postgres
# Wait for database to be healthy, then initialize # Wait for database to be healthy, then initialize
python scripts/init_database.py python scripts/init_database.py
# Terminal 1: Start FastAPI backend # Start FastAPI backend
uvicorn SPARC.api:app --host 0.0.0.0 --port 8000 --reload uvicorn SPARC.api:app --host 0.0.0.0 --port 8000 --reload
# Terminal 2: Start Streamlit dashboard # For the React frontend (separate terminal)
streamlit run dashboard.py --server.port 8501 --server.address 0.0.0.0 cd frontend
npm install
npm run dev
``` ```
--- ---
@@ -141,7 +143,7 @@ Access the services:
|---------|-----| |---------|-----|
| REST API | http://localhost:8000 | | REST API | http://localhost:8000 |
| API Documentation (Swagger) | http://localhost:8000/docs | | API Documentation (Swagger) | http://localhost:8000/docs |
| Dashboard (Web UI) | http://localhost:8501 | | Dashboard (Web UI) | http://localhost:8080 |
--- ---
@@ -149,16 +151,17 @@ Access the services:
### Via Dashboard (Web UI) ### Via Dashboard (Web UI)
1. Open http://localhost:8501 1. Open http://localhost:8080
2. Select **"Company Analysis"** from the sidebar 2. Register a new account or login (default admin: `admin` / `admin`)
3. Enter a company name (e.g., "Intel") 3. Navigate to **"Analysis"** from the sidebar
4. Click **"Analyze"** 4. Enter a company name (e.g., "Intel")
5. Click **"Analyze"**
This will: This will:
- Query SerpAPI for recent patents - Query SerpAPI for recent patents
- Download and parse patent PDFs - Download and parse patent PDFs
- Send patent content to Claude for analysis - Send patent content to Claude for analysis
- Store prompt/response in PostgreSQL - Store prompt/response in PostgreSQL (with caching)
- Display results in the dashboard - Display results in the dashboard
### Via REST API ### Via REST API
@@ -233,12 +236,12 @@ docker exec -it sparc-postgres psql -U postgres -d sparc -c \
| Component | Purpose | | Component | Purpose |
|-----------|---------| |-----------|---------|
| **Dashboard** | Streamlit web UI for interactive analysis | | **Dashboard** | React TypeScript web UI with authentication |
| **FastAPI** | REST API for programmatic access | | **FastAPI** | REST API with JWT authentication |
| **Analyzer** | Orchestrates patent retrieval and LLM analysis | | **Analyzer** | Orchestrates patent retrieval and LLM analysis |
| **SerpAPI** | Retrieves patent data from Google Patents | | **SerpAPI** | Retrieves patent data from Google Patents |
| **OpenRouter** | Routes requests to Claude for AI analysis | | **OpenRouter** | Routes requests to Claude for AI analysis |
| **PostgreSQL** | Stores prompts, responses, and analytics | | **PostgreSQL** | Stores prompts, responses, users, and cached results |
--- ---
@@ -248,10 +251,9 @@ docker exec -it sparc-postgres psql -U postgres -d sparc -c \
|----------|----------|---------|-------------| |----------|----------|---------|-------------|
| `API_KEY` | Yes | - | SerpAPI key for patent search | | `API_KEY` | Yes | - | SerpAPI key for patent search |
| `OPENROUTER_API_KEY` | Yes | - | OpenRouter API key for Claude access | | `OPENROUTER_API_KEY` | Yes | - | OpenRouter API key for Claude access |
| `DATABASE_URL` | Yes* | - | PostgreSQL connection string | | `DATABASE_URL` | Yes | - | PostgreSQL connection string |
| `USE_DATABASE` | No | `false` | Set to `true` to enable database storage | | `USE_CACHE` | No | `true` | Check database for cached responses before API calls |
| `JWT_SECRET` | Yes | - | Secret key for JWT authentication (change in production!) |
*Required when `USE_DATABASE=true`
### Database URL Format ### Database URL Format
@@ -273,9 +275,9 @@ The `docker-compose.yml` includes all services needed for production:
| Service | Container | Port | Description | | Service | Container | Port | Description |
|---------|-----------|------|-------------| |---------|-----------|------|-------------|
| `postgres` | sparc-postgres | 5432 | PostgreSQL database | | `postgres` | sparc-postgres | 5432 | PostgreSQL database |
| `init-db` | sparc-init-db | - | One-time database initialization | | `init-db` | sparc-init-db | - | One-time database initialization (seeds admin user) |
| `api` | sparc-api | 8000 | FastAPI REST API | | `api` | sparc-api | 8000 | FastAPI REST API with JWT auth |
| `dashboard` | sparc-dashboard | 8501 | Streamlit web UI | | `dashboard` | sparc-dashboard | 8080 | React TypeScript web UI |
### Common Docker Compose Commands ### Common Docker Compose Commands
@@ -382,11 +384,11 @@ cp .env.example .env
docker-compose up -d postgres docker-compose up -d postgres
python scripts/init_database.py python scripts/init_database.py
uvicorn SPARC.api:app --reload & uvicorn SPARC.api:app --reload &
streamlit run dashboard.py cd frontend && npm install && npm run dev &
# Check status # Check status
curl http://localhost:8000/health curl http://localhost:8000/health
open http://localhost:8501 open http://localhost:8080
# View data # View data
python scripts/view_analytics.py python scripts/view_analytics.py