From ed81ae4569921718602d6b7bd10accef8dbe03bb Mon Sep 17 00:00:00 2001 From: 0xWheatyz Date: Sat, 14 Mar 2026 14:30:21 -0400 Subject: [PATCH] docs: update documentation for React frontend and cache mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update all documentation to reflect recent changes: - Replace Streamlit dashboard references with React TypeScript dashboard - Update dashboard port from 8501 to 8080 - Add auth.py and database.py to architecture section - Change USE_DATABASE terminology to USE_CACHE - Add JWT_SECRET to environment variables reference - Document default admin credentials and user seeding 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- README.md | 25 +++++----- docs/DATABASE_MODE.md | 103 ++++++++++++++++++++++++------------------ docs/DEPLOYMENT.md | 46 ++++++++++--------- 3 files changed, 96 insertions(+), 78 deletions(-) diff --git a/README.md b/README.md index d4ae497..c9b0b10 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ SPARC automatically collects, parses, and analyzes patents from companies to pro - **Portfolio Analysis**: Evaluates multiple patents holistically for comprehensive insights - **Batch Processing**: Analyze multiple companies concurrently with progress tracking - **REST API**: FastAPI web service with async job support -- **Dashboard**: Interactive Streamlit visualization dashboard +- **Dashboard**: React TypeScript web dashboard with authentication - **Robust Testing**: 40 tests covering all major functionality ## Architecture @@ -27,7 +27,9 @@ SPARC/ ├── serp_api.py # Patent retrieval and PDF parsing ├── llm.py # Claude AI integration via OpenRouter ├── analyzer.py # High-level orchestration -├── api.py # FastAPI web service +├── api.py # FastAPI web service with auth endpoints +├── auth.py # JWT authentication module +├── database.py # PostgreSQL storage with caching ├── types.py # Data models └── config.py # Environment configuration ``` @@ -48,7 +50,7 @@ docker-compose up -d # Access the services # - API: http://localhost:8000 -# - Dashboard: http://localhost:8501 +# - Dashboard: http://localhost:8080 # - API Docs: http://localhost:8000/docs ``` @@ -186,21 +188,22 @@ curl -X POST http://localhost:8000/analyze/batch/async \ -d '{"companies": ["nvidia", "amd", "intel", "qualcomm"]}' ``` -### Visualization Dashboard +### Web Dashboard -Launch the interactive Streamlit dashboard: +The React dashboard is included in Docker Compose: ```bash -streamlit run dashboard.py +docker-compose up -d ``` Dashboard features: +- **Authentication**: User registration, login, and JWT-based sessions - **Company Analysis**: Analyze individual companies with real-time results -- **Batch Analysis**: Process multiple companies with progress tracking and charts -- **Analytics**: View historical analysis data and trends (requires database mode) -- **System Status**: Monitor database and analyzer health +- **Batch Analysis**: Process multiple companies with progress tracking +- **Analytics**: View historical analysis data and trends +- **Admin Panel**: User management for administrators -The dashboard runs at `http://localhost:8501` by default. +The dashboard runs at `http://localhost:8080` when using Docker Compose. ## Running Tests @@ -280,4 +283,4 @@ For open source projects, say how it is licensed. Core functionality complete. Ready for production use with API keys configured. -All major features implemented: REST API, Streamlit dashboard, Docker containerization, database storage, and multi-company batch processing. +All major features implemented: REST API, React dashboard with authentication, Docker containerization, database storage with caching, and multi-company batch processing. diff --git a/docs/DATABASE_MODE.md b/docs/DATABASE_MODE.md index 4beceb0..842c77f 100644 --- a/docs/DATABASE_MODE.md +++ b/docs/DATABASE_MODE.md @@ -1,16 +1,19 @@ -# Database Mode for Testing and Analytics +# Database Storage and Caching -This document explains how to use SPARC's database mode for storing LLM messages for testing and analytics purposes. +This document explains how SPARC uses PostgreSQL for storing LLM messages, enabling response caching and analytics. ## Overview -SPARC supports two modes of operation: +SPARC stores all LLM interactions in PostgreSQL, providing: -1. **API Mode** (default): Messages are sent to OpenRouter's API and you receive real LLM responses -2. **Database Mode**: Messages are stored in a PostgreSQL database without making API calls, useful for: - - Testing the application without consuming API credits - - Collecting analytics on message patterns and usage - - Development and debugging +- **Response Caching**: Avoid redundant API calls for previously analyzed patents +- **Analytics**: Track usage patterns, token consumption, and analysis history +- **Persistence**: Maintain analysis history across sessions + +SPARC supports two cache modes: + +1. **Cache Mode** (default, `USE_CACHE=true`): Check database for cached responses before making API calls +2. **Fresh Mode** (`USE_CACHE=false`): Always make fresh API calls (still stores results in database) ## Setup @@ -45,43 +48,43 @@ cp .env.example .env Edit `.env` and set: ```env -# For database mode (testing/analytics) -USE_DATABASE=true +# Database connection (required) DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sparc -# For API mode (production) -USE_DATABASE=false +# Cache mode: use cached responses when available +USE_CACHE=true + +# API key for fresh LLM calls OPENROUTER_API_KEY=your_openrouter_key_here ``` ## Usage -### Running in Database Mode +### Running with Cache Mode (Default) -Set `USE_DATABASE=true` in your `.env` file, then run the application normally: +Set `USE_CACHE=true` in your `.env` file, then run the application normally: ```bash python main.py ``` -Instead of sending messages to OpenRouter, the application will: -- Store all prompts in the database -- Return a placeholder response -- Log metadata (company name, analysis type, timestamps) +The application will: +- Check the database for cached responses matching the request +- If found, return the cached response (no API call) +- If not found, make an API call and store the response for future use -### Running in API Mode +### Running with Fresh Mode -Set `USE_DATABASE=false` in your `.env` file, then run the application normally: +Set `USE_CACHE=false` in your `.env` file to always get fresh responses: ```bash python main.py ``` -The application will send messages to OpenRouter and return real LLM responses. - -### Hybrid Mode (Optional) - -You can also enable database logging while still using the API by initializing the database client in your code. The `LLMAnalyzer` will automatically log all API calls to the database if a database client is available. +The application will: +- Always send messages to OpenRouter for real LLM responses +- Store all responses in the database +- Useful when you need the latest analysis or want to refresh cached data ## Viewing Analytics @@ -195,16 +198,16 @@ docker-compose down -v ## Toggling Between Modes -You can easily switch between modes by changing the `USE_DATABASE` environment variable: +You can easily switch between modes by changing the `USE_CACHE` environment variable: -### Quick Toggle (temporary, for testing) +### Quick Toggle (temporary) ```bash -# Run in database mode -USE_DATABASE=true python main.py +# Run with caching enabled +USE_CACHE=true python main.py -# Run in API mode -USE_DATABASE=false python main.py +# Run with fresh API calls +USE_CACHE=false python main.py ``` ### Persistent Toggle @@ -212,38 +215,48 @@ USE_DATABASE=false python main.py Edit your `.env` file: ```env -# For testing/analytics -USE_DATABASE=true +# Use cached responses when available (recommended for most use) +USE_CACHE=true -# For production use -USE_DATABASE=false +# Always make fresh API calls +USE_CACHE=false ``` ## Use Cases -### Testing Without API Costs +### Cost Optimization with Caching -During development, enable database mode to test the full application flow without consuming API credits: +Cache mode reduces API costs by reusing previous analysis results: ```bash -USE_DATABASE=true python main.py +USE_CACHE=true python main.py +``` + +If the same company/patent combination was analyzed before, the cached response is returned instantly. + +### Fresh Analysis + +When you need the latest LLM analysis (e.g., after model updates): + +```bash +USE_CACHE=false python main.py ``` ### Collecting Usage Analytics -Enable database mode in a test environment to collect analytics on: +The database stores all interactions, enabling analytics on: - Which companies are analyzed most frequently - Types of analyses performed -- Prompt patterns and lengths -- Usage over time +- Token usage and costs over time +- Response caching hit rates ### Development and Debugging -Database mode is useful for: -- Testing patent parsing logic without API calls +Database storage is useful for: +- Reviewing actual prompts sent to the LLM +- Analyzing response patterns - Debugging the full pipeline end-to-end -- Collecting sample prompts for optimization -- Understanding token usage patterns (when in API mode with logging) +- Understanding token usage patterns ## Troubleshooting diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index ed43b64..bb7bfd9 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -64,7 +64,7 @@ docker-compose ps # You should see: # - sparc-postgres (healthy) # - sparc-api (running on port 8000) -# - sparc-dashboard (running on port 8501) +# - sparc-dashboard (running on port 8080) ``` The database is automatically initialized by the `init-db` service. @@ -116,11 +116,13 @@ docker-compose up -d postgres # Wait for database to be healthy, then initialize python scripts/init_database.py -# Terminal 1: Start FastAPI backend +# Start FastAPI backend uvicorn SPARC.api:app --host 0.0.0.0 --port 8000 --reload -# Terminal 2: Start Streamlit dashboard -streamlit run dashboard.py --server.port 8501 --server.address 0.0.0.0 +# For the React frontend (separate terminal) +cd frontend +npm install +npm run dev ``` --- @@ -141,7 +143,7 @@ Access the services: |---------|-----| | REST API | http://localhost:8000 | | API Documentation (Swagger) | http://localhost:8000/docs | -| Dashboard (Web UI) | http://localhost:8501 | +| Dashboard (Web UI) | http://localhost:8080 | --- @@ -149,16 +151,17 @@ Access the services: ### Via Dashboard (Web UI) -1. Open http://localhost:8501 -2. Select **"Company Analysis"** from the sidebar -3. Enter a company name (e.g., "Intel") -4. Click **"Analyze"** +1. Open http://localhost:8080 +2. Register a new account or login (default admin: `admin` / `admin`) +3. Navigate to **"Analysis"** from the sidebar +4. Enter a company name (e.g., "Intel") +5. Click **"Analyze"** This will: - Query SerpAPI for recent patents - Download and parse patent PDFs - Send patent content to Claude for analysis -- Store prompt/response in PostgreSQL +- Store prompt/response in PostgreSQL (with caching) - Display results in the dashboard ### Via REST API @@ -233,12 +236,12 @@ docker exec -it sparc-postgres psql -U postgres -d sparc -c \ | Component | Purpose | |-----------|---------| -| **Dashboard** | Streamlit web UI for interactive analysis | -| **FastAPI** | REST API for programmatic access | +| **Dashboard** | React TypeScript web UI with authentication | +| **FastAPI** | REST API with JWT authentication | | **Analyzer** | Orchestrates patent retrieval and LLM analysis | | **SerpAPI** | Retrieves patent data from Google Patents | | **OpenRouter** | Routes requests to Claude for AI analysis | -| **PostgreSQL** | Stores prompts, responses, and analytics | +| **PostgreSQL** | Stores prompts, responses, users, and cached results | --- @@ -248,10 +251,9 @@ docker exec -it sparc-postgres psql -U postgres -d sparc -c \ |----------|----------|---------|-------------| | `API_KEY` | Yes | - | SerpAPI key for patent search | | `OPENROUTER_API_KEY` | Yes | - | OpenRouter API key for Claude access | -| `DATABASE_URL` | Yes* | - | PostgreSQL connection string | -| `USE_DATABASE` | No | `false` | Set to `true` to enable database storage | - -*Required when `USE_DATABASE=true` +| `DATABASE_URL` | Yes | - | PostgreSQL connection string | +| `USE_CACHE` | No | `true` | Check database for cached responses before API calls | +| `JWT_SECRET` | Yes | - | Secret key for JWT authentication (change in production!) | ### Database URL Format @@ -273,9 +275,9 @@ The `docker-compose.yml` includes all services needed for production: | Service | Container | Port | Description | |---------|-----------|------|-------------| | `postgres` | sparc-postgres | 5432 | PostgreSQL database | -| `init-db` | sparc-init-db | - | One-time database initialization | -| `api` | sparc-api | 8000 | FastAPI REST API | -| `dashboard` | sparc-dashboard | 8501 | Streamlit web UI | +| `init-db` | sparc-init-db | - | One-time database initialization (seeds admin user) | +| `api` | sparc-api | 8000 | FastAPI REST API with JWT auth | +| `dashboard` | sparc-dashboard | 8080 | React TypeScript web UI | ### Common Docker Compose Commands @@ -382,11 +384,11 @@ cp .env.example .env docker-compose up -d postgres python scripts/init_database.py uvicorn SPARC.api:app --reload & -streamlit run dashboard.py +cd frontend && npm install && npm run dev & # Check status curl http://localhost:8000/health -open http://localhost:8501 +open http://localhost:8080 # View data python scripts/view_analytics.py