forked from 0xWheatyz/SPARC
Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ae9f257dcb | |||
| 6105ba7793 | |||
| e8cdc089fa | |||
| 9c971dac72 | |||
| 6f0b448044 | |||
| 1a297eb60b | |||
| 3154f6b732 | |||
| b9bb3dc1cd | |||
| 90f9cfc826 | |||
| d387bbbdf3 | |||
| fa564e5e1e |
+122
@@ -0,0 +1,122 @@
|
|||||||
|
# SPARC Roadmap
|
||||||
|
|
||||||
|
Semiconductor Patent & Analytics Report Core -- development priorities.
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
SPARC is a patent analysis platform with a working end-to-end pipeline:
|
||||||
|
Python/FastAPI backend, React/TypeScript frontend, PostgreSQL for persistence
|
||||||
|
and caching, Docker Compose for local development, and Gitea Actions CI/CD for
|
||||||
|
image builds. Core features (patent retrieval via SerpAPI, PDF parsing, LLM
|
||||||
|
analysis via OpenRouter/Claude, batch processing, JWT authentication, analytics
|
||||||
|
dashboard) are all implemented and functional.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P1 -- High Priority
|
||||||
|
|
||||||
|
These items address correctness, security, and reliability gaps that should be
|
||||||
|
resolved before broader production use.
|
||||||
|
|
||||||
|
### Security hardening
|
||||||
|
|
||||||
|
- **Rotate default JWT secret.** `auth.py` ships a fallback
|
||||||
|
`sparc-secret-key-change-in-production` that will be used if `JWT_SECRET` is
|
||||||
|
unset. Add a startup check that refuses to start with the default secret in
|
||||||
|
non-development environments.
|
||||||
|
- **CORS allow-origins are hardcoded.** `api.py` only permits
|
||||||
|
`localhost:3000` and `localhost:5173`. Make the allowed origins configurable
|
||||||
|
via environment variable so the dashboard works when deployed behind a real
|
||||||
|
domain.
|
||||||
|
- **Database credentials in docker-compose.yml.** The compose file embeds
|
||||||
|
`postgres:postgres` in plain text. Reference a `.env` file or Docker secrets
|
||||||
|
instead.
|
||||||
|
|
||||||
|
### Error handling and resilience
|
||||||
|
|
||||||
|
- **`get_db_client()` in `auth.py` creates a new `DatabaseClient` on every
|
||||||
|
call.** This bypasses the connection pool and can exhaust database
|
||||||
|
connections under load. Refactor to share a single pooled client.
|
||||||
|
- **`_jobs` dict is in-memory only.** Job state is lost on API restart. Persist
|
||||||
|
job status in PostgreSQL or Redis so async batch results survive restarts.
|
||||||
|
- **No rate limiting on auth endpoints.** `/auth/login` and `/auth/register`
|
||||||
|
are unprotected against brute-force or abuse. Add rate limiting middleware.
|
||||||
|
|
||||||
|
### Test coverage for auth and admin
|
||||||
|
|
||||||
|
- The existing API tests (`tests/test_api.py`) bypass authentication entirely.
|
||||||
|
Add tests that exercise the JWT flow: registration, login, protected-route
|
||||||
|
access, token refresh, and admin-only endpoints.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P2 -- Medium Priority
|
||||||
|
|
||||||
|
Improvements to usability, performance, and developer experience.
|
||||||
|
|
||||||
|
### Backend
|
||||||
|
|
||||||
|
- **Add structured logging.** Replace `print()` calls throughout `analyzer.py`,
|
||||||
|
`serp_api.py`, and `llm.py` with Python `logging` so log levels and
|
||||||
|
formatting are consistent.
|
||||||
|
- **Make LLM model configurable.** `llm.py` hardcodes
|
||||||
|
`anthropic/claude-3.5-sonnet`. Accept a `MODEL` environment variable to allow
|
||||||
|
switching models without code changes.
|
||||||
|
- **SERP cache TTL is hardcoded to 24 hours.** Expose `SERP_CACHE_TTL_HOURS`
|
||||||
|
as an environment variable in `config.py`.
|
||||||
|
- **Patent PDF storage.** PDFs are saved to a local `patents/` directory. For
|
||||||
|
containerized deployments, consider object storage (S3/MinIO) or at minimum
|
||||||
|
document the volume mount requirement more prominently.
|
||||||
|
- **`analyze_single_patent` assumes local file path.** The method constructs
|
||||||
|
`patents/{patent_id}.pdf` and reads from disk, but does not download the PDF
|
||||||
|
first. Either integrate the download step or document the prerequisite.
|
||||||
|
- **`Patent.patent_id` typed as `int` in `types.py` but used as `str`
|
||||||
|
everywhere.** Fix the type annotation to `str`.
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
|
||||||
|
- **No loading/error states on several pages.** The Batch and Analytics pages
|
||||||
|
would benefit from skeleton loaders and user-friendly error messages.
|
||||||
|
- **No dark mode.** Tailwind is configured but no dark variant is applied.
|
||||||
|
- **Missing `package-lock.json` or `pnpm-lock.yaml`.** The frontend has no
|
||||||
|
lockfile committed, leading to non-reproducible builds.
|
||||||
|
|
||||||
|
### CI/CD
|
||||||
|
|
||||||
|
- **No test stage in the Gitea Actions workflow.** `build.yaml` builds and
|
||||||
|
pushes images but never runs `pytest`. Add a test job that gates the build.
|
||||||
|
- **No linting or type checking.** Add `ruff` (Python) and `tsc --noEmit`
|
||||||
|
(TypeScript) to CI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P3 -- Nice to Have
|
||||||
|
|
||||||
|
Lower-urgency enhancements and future features.
|
||||||
|
|
||||||
|
- **Export analysis reports.** Allow users to download analysis results as PDF
|
||||||
|
or CSV from the dashboard.
|
||||||
|
- **Comparison view.** Side-by-side comparison of two companies' patent
|
||||||
|
portfolios.
|
||||||
|
- **Scheduled/recurring analysis.** Periodically re-analyze tracked companies
|
||||||
|
and alert on significant changes.
|
||||||
|
- **Webhook/notification support.** Send alerts (Slack, Discord, email) when
|
||||||
|
batch jobs complete or when a company's innovation score changes
|
||||||
|
significantly.
|
||||||
|
- **Multi-model support.** Let users choose between LLM providers per analysis
|
||||||
|
(e.g., GPT-4o, Gemini, Claude) and compare outputs.
|
||||||
|
- **Patent trend charts.** Visualize patent filing frequency and technology
|
||||||
|
category distribution over time in the Analytics page.
|
||||||
|
- **API pagination.** The `/analyze/batch` and `/jobs` endpoints could benefit
|
||||||
|
from cursor-based pagination for large result sets.
|
||||||
|
- **OpenAPI client generation.** Auto-generate the TypeScript API client from
|
||||||
|
the FastAPI OpenAPI spec to keep frontend types in sync.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Infrastructure and Deployment
|
||||||
|
|
||||||
|
Kubernetes manifests, Helm charts, and cluster-level concerns (MetalLB,
|
||||||
|
storage, FluxCD sync) are tracked in the
|
||||||
|
[Talos](https://10.0.1.10/leeworks-agents/Talos) repository. File
|
||||||
|
infrastructure-related issues there, not here.
|
||||||
+91
-29
@@ -4,26 +4,33 @@ This module ties together patent retrieval, parsing, and LLM analysis
|
|||||||
to provide company performance estimation based on patent portfolios.
|
to provide company performance estimation based on patent portfolios.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import hashlib
|
||||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||||
from typing import Callable
|
from typing import Callable
|
||||||
|
|
||||||
|
from SPARC import config
|
||||||
|
from SPARC.database import DatabaseClient
|
||||||
from SPARC.serp_api import SERP
|
from SPARC.serp_api import SERP
|
||||||
from SPARC.llm import LLMAnalyzer
|
from SPARC.llm import LLMAnalyzer
|
||||||
from SPARC.types import Patent, CompanyAnalysisResult, BatchAnalysisResult
|
from SPARC.types import Patent, Patents, CompanyAnalysisResult, BatchAnalysisResult
|
||||||
|
|
||||||
|
|
||||||
class CompanyAnalyzer:
|
class CompanyAnalyzer:
|
||||||
"""Orchestrates end-to-end company performance analysis via patents."""
|
"""Orchestrates end-to-end company performance analysis via patents."""
|
||||||
|
|
||||||
def __init__(self, openrouter_api_key: str | None = None):
|
def __init__(self, openrouter_api_key: str | None = None, db_client: DatabaseClient | None = None):
|
||||||
"""Initialize the company analyzer.
|
"""Initialize the company analyzer.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
openrouter_api_key: Optional OpenRouter API key. If None, loads from config.
|
openrouter_api_key: Optional OpenRouter API key. If None, loads from config.
|
||||||
|
db_client: Optional DatabaseClient for patent caching. Created automatically if None.
|
||||||
"""
|
"""
|
||||||
self.llm_analyzer = LLMAnalyzer(api_key=openrouter_api_key)
|
self.llm_analyzer = LLMAnalyzer(api_key=openrouter_api_key)
|
||||||
|
self.db = db_client or DatabaseClient(config.database_url)
|
||||||
|
self.db.connect()
|
||||||
|
self.db.initialize_schema()
|
||||||
|
|
||||||
def analyze_company(self, company_name: str) -> str:
|
def analyze_company(self, company_name: str, patents: "Patents | None" = None) -> str:
|
||||||
"""Analyze a company's performance based on their patent portfolio.
|
"""Analyze a company's performance based on their patent portfolio.
|
||||||
|
|
||||||
This is the main entry point that orchestrates the full pipeline:
|
This is the main entry point that orchestrates the full pipeline:
|
||||||
@@ -35,40 +42,52 @@ class CompanyAnalyzer:
|
|||||||
|
|
||||||
Args:
|
Args:
|
||||||
company_name: Name of the company to analyze
|
company_name: Name of the company to analyze
|
||||||
|
patents: Optional pre-fetched Patents result to avoid duplicate API calls
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Comprehensive analysis of company's innovation and performance outlook
|
Comprehensive analysis of company's innovation and performance outlook
|
||||||
"""
|
"""
|
||||||
print(f"Retrieving patents for {company_name}...")
|
if patents is None:
|
||||||
patents = SERP.query(company_name)
|
# Check SERP query cache first
|
||||||
|
query_hash = hashlib.sha256(company_name.lower().encode()).hexdigest()
|
||||||
|
cached_ids = self.db.get_cached_serp_query(query_hash)
|
||||||
|
if cached_ids is not None:
|
||||||
|
print(f"Using cached SERP results for {company_name} ({len(cached_ids)} patents)")
|
||||||
|
patents = Patents(patents=[
|
||||||
|
Patent(patent_id=pid, pdf_link="")
|
||||||
|
for pid in cached_ids
|
||||||
|
])
|
||||||
|
else:
|
||||||
|
print(f"Retrieving patents for {company_name}...")
|
||||||
|
patents = SERP.query(company_name)
|
||||||
|
# Cache the SERP results
|
||||||
|
if patents.patents:
|
||||||
|
self.db.store_serp_query(
|
||||||
|
company_name=company_name,
|
||||||
|
query_hash=query_hash,
|
||||||
|
patent_ids=[p.patent_id for p in patents.patents],
|
||||||
|
)
|
||||||
|
|
||||||
if not patents.patents:
|
if not patents.patents:
|
||||||
return f"No patents found for {company_name}"
|
return f"No patents found for {company_name}"
|
||||||
|
|
||||||
print(f"Found {len(patents.patents)} patents. Processing...")
|
print(f"Found {len(patents.patents)} patents. Processing...")
|
||||||
|
|
||||||
# Download and parse each patent
|
# Download, parse, and minimize patents in parallel
|
||||||
processed_patents = []
|
processed_patents = []
|
||||||
for idx, patent in enumerate(patents.patents, 1):
|
with ThreadPoolExecutor(max_workers=config.patent_thread_workers) as executor:
|
||||||
print(f"Processing patent {idx}/{len(patents.patents)}: {patent.patent_id}")
|
future_to_patent = {
|
||||||
|
executor.submit(self._process_single_patent, patent, company_name, self.db): patent
|
||||||
try:
|
for patent in patents.patents
|
||||||
# Download PDF
|
}
|
||||||
patent = SERP.save_patents(patent)
|
for future in as_completed(future_to_patent):
|
||||||
|
patent = future_to_patent[future]
|
||||||
# Parse sections from PDF
|
try:
|
||||||
sections = SERP.parse_patent_pdf(patent.pdf_path)
|
result = future.result()
|
||||||
|
if result:
|
||||||
# Minimize for LLM (remove bloat)
|
processed_patents.append(result)
|
||||||
minimized_content = SERP.minimize_patent_for_llm(sections)
|
except Exception as e:
|
||||||
|
print(f"Warning: Failed to process {patent.patent_id}: {e}")
|
||||||
processed_patents.append(
|
|
||||||
{"patent_id": patent.patent_id, "content": minimized_content}
|
|
||||||
)
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Warning: Failed to process {patent.patent_id}: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
if not processed_patents:
|
if not processed_patents:
|
||||||
return f"Failed to process any patents for {company_name}"
|
return f"Failed to process any patents for {company_name}"
|
||||||
@@ -113,6 +132,46 @@ class CompanyAnalyzer:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
return f"Failed to analyze patent {patent_id}: {e}"
|
return f"Failed to analyze patent {patent_id}: {e}"
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _process_single_patent(
|
||||||
|
patent: Patent,
|
||||||
|
company_name: str = "",
|
||||||
|
db: DatabaseClient | None = None,
|
||||||
|
) -> dict | None:
|
||||||
|
"""Download, parse, and minimize a single patent. Thread-safe.
|
||||||
|
|
||||||
|
Checks DB cache before downloading. Stores results after processing.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dict with patent_id and minimized content, or None on failure.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Check DB cache first
|
||||||
|
if db:
|
||||||
|
cached = db.get_cached_patent(patent.patent_id)
|
||||||
|
if cached and cached.get("minimized_content"):
|
||||||
|
return {"patent_id": patent.patent_id, "content": cached["minimized_content"]}
|
||||||
|
|
||||||
|
# Full processing: download, parse, minimize
|
||||||
|
patent = SERP.save_patents(patent)
|
||||||
|
sections = SERP.parse_patent_pdf(patent.pdf_path)
|
||||||
|
minimized_content = SERP.minimize_patent_for_llm(sections)
|
||||||
|
|
||||||
|
# Store in DB cache
|
||||||
|
if db:
|
||||||
|
db.store_patent(
|
||||||
|
patent_id=patent.patent_id,
|
||||||
|
company_name=company_name,
|
||||||
|
pdf_link=patent.pdf_link,
|
||||||
|
raw_sections=sections,
|
||||||
|
minimized_content=minimized_content,
|
||||||
|
)
|
||||||
|
|
||||||
|
return {"patent_id": patent.patent_id, "content": minimized_content}
|
||||||
|
except Exception as e:
|
||||||
|
print(f"Warning: Failed to process {patent.patent_id}: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
def _analyze_company_safe(self, company_name: str) -> CompanyAnalysisResult:
|
def _analyze_company_safe(self, company_name: str) -> CompanyAnalysisResult:
|
||||||
"""Internal wrapper that catches exceptions and returns structured result.
|
"""Internal wrapper that catches exceptions and returns structured result.
|
||||||
|
|
||||||
@@ -123,11 +182,14 @@ class CompanyAnalyzer:
|
|||||||
CompanyAnalysisResult with success/failure status
|
CompanyAnalysisResult with success/failure status
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
patents = SERP.query(company_name)
|
# Delegate to analyze_company which handles SERP/patent caching
|
||||||
patent_count = len(patents.patents) if patents.patents else 0
|
|
||||||
|
|
||||||
analysis = self.analyze_company(company_name)
|
analysis = self.analyze_company(company_name)
|
||||||
|
|
||||||
|
# Determine patent count from cached SERP query
|
||||||
|
query_hash = hashlib.sha256(company_name.lower().encode()).hexdigest()
|
||||||
|
cached_ids = self.db.get_cached_serp_query(query_hash)
|
||||||
|
patent_count = len(cached_ids) if cached_ids else 0
|
||||||
|
|
||||||
# Check if analysis indicates failure
|
# Check if analysis indicates failure
|
||||||
if analysis.startswith("No patents found") or analysis.startswith(
|
if analysis.startswith("No patents found") or analysis.startswith(
|
||||||
"Failed to process"
|
"Failed to process"
|
||||||
|
|||||||
@@ -26,6 +26,10 @@ use_cache = os.getenv("USE_CACHE", "true").lower() in ("true", "1", "yes")
|
|||||||
# This variable is kept for backwards compatibility but has no effect
|
# This variable is kept for backwards compatibility but has no effect
|
||||||
use_database = os.getenv("USE_DATABASE", "false").lower() in ("true", "1", "yes")
|
use_database = os.getenv("USE_DATABASE", "false").lower() in ("true", "1", "yes")
|
||||||
|
|
||||||
|
# Patent search configuration
|
||||||
|
patent_search_days = int(os.getenv("PATENT_SEARCH_DAYS", "90"))
|
||||||
|
patent_thread_workers = int(os.getenv("PATENT_THREAD_WORKERS", "5"))
|
||||||
|
|
||||||
# Root path for running behind a reverse proxy (e.g., "/api" when served at /api/)
|
# Root path for running behind a reverse proxy (e.g., "/api" when served at /api/)
|
||||||
# This ensures OpenAPI docs work correctly when accessed via the proxy
|
# This ensures OpenAPI docs work correctly when accessed via the proxy
|
||||||
root_path = os.getenv("ROOT_PATH", "")
|
root_path = os.getenv("ROOT_PATH", "")
|
||||||
|
|||||||
+146
-4
@@ -1,9 +1,11 @@
|
|||||||
"""Database client for storing and retrieving LLM messages and user authentication."""
|
"""Database client for storing and retrieving LLM messages and user authentication."""
|
||||||
|
|
||||||
|
import contextlib
|
||||||
import psycopg2
|
import psycopg2
|
||||||
|
from psycopg2.pool import ThreadedConnectionPool
|
||||||
from psycopg2.extras import RealDictCursor
|
from psycopg2.extras import RealDictCursor
|
||||||
from typing import Dict, List, Optional
|
from typing import Dict, List, Optional
|
||||||
from datetime import datetime
|
from datetime import datetime, timedelta
|
||||||
import json
|
import json
|
||||||
import hashlib
|
import hashlib
|
||||||
import bcrypt
|
import bcrypt
|
||||||
@@ -12,24 +14,49 @@ import bcrypt
|
|||||||
class DatabaseClient:
|
class DatabaseClient:
|
||||||
"""Handles database operations for message storage and retrieval."""
|
"""Handles database operations for message storage and retrieval."""
|
||||||
|
|
||||||
def __init__(self, database_url: str):
|
def __init__(self, database_url: str, minconn: int = 2, maxconn: int = 10):
|
||||||
"""Initialize the database client.
|
"""Initialize the database client.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
database_url: PostgreSQL connection string
|
database_url: PostgreSQL connection string
|
||||||
|
minconn: Minimum connections in the pool
|
||||||
|
maxconn: Maximum connections in the pool
|
||||||
"""
|
"""
|
||||||
self.database_url = database_url
|
self.database_url = database_url
|
||||||
|
self._pool: ThreadedConnectionPool | None = None
|
||||||
|
self._minconn = minconn
|
||||||
|
self._maxconn = maxconn
|
||||||
|
# Legacy single connection kept for backwards compatibility
|
||||||
self.conn = None
|
self.conn = None
|
||||||
|
|
||||||
|
def _ensure_pool(self):
|
||||||
|
"""Create the connection pool if it doesn't exist yet."""
|
||||||
|
if self._pool is None or self._pool.closed:
|
||||||
|
self._pool = ThreadedConnectionPool(
|
||||||
|
self._minconn, self._maxconn, self.database_url
|
||||||
|
)
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def get_conn(self):
|
||||||
|
"""Check out a connection from the pool. Returns it on exit."""
|
||||||
|
self._ensure_pool()
|
||||||
|
conn = self._pool.getconn()
|
||||||
|
try:
|
||||||
|
yield conn
|
||||||
|
finally:
|
||||||
|
self._pool.putconn(conn)
|
||||||
|
|
||||||
def connect(self):
|
def connect(self):
|
||||||
"""Establish database connection."""
|
"""Establish database connection (legacy single-connection path)."""
|
||||||
if not self.conn or self.conn.closed:
|
if not self.conn or self.conn.closed:
|
||||||
self.conn = psycopg2.connect(self.database_url)
|
self.conn = psycopg2.connect(self.database_url)
|
||||||
|
|
||||||
def close(self):
|
def close(self):
|
||||||
"""Close database connection."""
|
"""Close database connection and pool."""
|
||||||
if self.conn and not self.conn.closed:
|
if self.conn and not self.conn.closed:
|
||||||
self.conn.close()
|
self.conn.close()
|
||||||
|
if self._pool and not self._pool.closed:
|
||||||
|
self._pool.closeall()
|
||||||
|
|
||||||
def initialize_schema(self):
|
def initialize_schema(self):
|
||||||
"""Create database tables if they don't exist."""
|
"""Create database tables if they don't exist."""
|
||||||
@@ -110,6 +137,40 @@ class DatabaseClient:
|
|||||||
ON users(email)
|
ON users(email)
|
||||||
""")
|
""")
|
||||||
|
|
||||||
|
# Create patents cache table
|
||||||
|
cursor.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS patents (
|
||||||
|
patent_id VARCHAR(64) PRIMARY KEY,
|
||||||
|
company_name VARCHAR(255),
|
||||||
|
pdf_link TEXT,
|
||||||
|
raw_sections JSONB,
|
||||||
|
minimized_content TEXT,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
|
||||||
|
cursor.execute("""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_patents_company
|
||||||
|
ON patents(company_name)
|
||||||
|
""")
|
||||||
|
|
||||||
|
# Create SERP query cache table
|
||||||
|
cursor.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS serp_queries (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
company_name VARCHAR(255),
|
||||||
|
query_hash VARCHAR(64) UNIQUE,
|
||||||
|
result_patent_ids TEXT[],
|
||||||
|
expires_at TIMESTAMP NOT NULL,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
|
||||||
|
cursor.execute("""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_serp_queries_hash
|
||||||
|
ON serp_queries(query_hash)
|
||||||
|
""")
|
||||||
|
|
||||||
self.conn.commit()
|
self.conn.commit()
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -320,6 +381,87 @@ class DatabaseClient:
|
|||||||
"period_days": days,
|
"period_days": days,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Patent Cache Methods
|
||||||
|
|
||||||
|
def get_cached_patent(self, patent_id: str) -> Optional[Dict]:
|
||||||
|
"""Look up a cached patent by ID.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dict with raw_sections and minimized_content, or None.
|
||||||
|
"""
|
||||||
|
with self.get_conn() as conn:
|
||||||
|
with conn.cursor(cursor_factory=RealDictCursor) as cursor:
|
||||||
|
cursor.execute(
|
||||||
|
"SELECT * FROM patents WHERE patent_id = %s",
|
||||||
|
(patent_id,),
|
||||||
|
)
|
||||||
|
row = cursor.fetchone()
|
||||||
|
return dict(row) if row else None
|
||||||
|
|
||||||
|
def store_patent(
|
||||||
|
self,
|
||||||
|
patent_id: str,
|
||||||
|
company_name: str,
|
||||||
|
pdf_link: str,
|
||||||
|
raw_sections: Dict,
|
||||||
|
minimized_content: str,
|
||||||
|
) -> None:
|
||||||
|
"""Store a processed patent in the cache."""
|
||||||
|
with self.get_conn() as conn:
|
||||||
|
with conn.cursor() as cursor:
|
||||||
|
cursor.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO patents (patent_id, company_name, pdf_link, raw_sections, minimized_content)
|
||||||
|
VALUES (%s, %s, %s, %s, %s)
|
||||||
|
ON CONFLICT (patent_id) DO UPDATE SET
|
||||||
|
raw_sections = EXCLUDED.raw_sections,
|
||||||
|
minimized_content = EXCLUDED.minimized_content
|
||||||
|
""",
|
||||||
|
(patent_id, company_name, pdf_link, json.dumps(raw_sections), minimized_content),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
def get_cached_serp_query(self, query_hash: str) -> Optional[List[str]]:
|
||||||
|
"""Look up cached SERP query results.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of patent IDs if cache hit and not expired, None otherwise.
|
||||||
|
"""
|
||||||
|
with self.get_conn() as conn:
|
||||||
|
with conn.cursor(cursor_factory=RealDictCursor) as cursor:
|
||||||
|
cursor.execute(
|
||||||
|
"""
|
||||||
|
SELECT result_patent_ids FROM serp_queries
|
||||||
|
WHERE query_hash = %s AND expires_at > NOW()
|
||||||
|
""",
|
||||||
|
(query_hash,),
|
||||||
|
)
|
||||||
|
row = cursor.fetchone()
|
||||||
|
return row["result_patent_ids"] if row else None
|
||||||
|
|
||||||
|
def store_serp_query(
|
||||||
|
self,
|
||||||
|
company_name: str,
|
||||||
|
query_hash: str,
|
||||||
|
patent_ids: List[str],
|
||||||
|
ttl_hours: int = 24,
|
||||||
|
) -> None:
|
||||||
|
"""Store SERP query results in the cache."""
|
||||||
|
expires_at = datetime.now() + timedelta(hours=ttl_hours)
|
||||||
|
with self.get_conn() as conn:
|
||||||
|
with conn.cursor() as cursor:
|
||||||
|
cursor.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO serp_queries (company_name, query_hash, result_patent_ids, expires_at)
|
||||||
|
VALUES (%s, %s, %s, %s)
|
||||||
|
ON CONFLICT (query_hash) DO UPDATE SET
|
||||||
|
result_patent_ids = EXCLUDED.result_patent_ids,
|
||||||
|
expires_at = EXCLUDED.expires_at
|
||||||
|
""",
|
||||||
|
(company_name, query_hash, patent_ids, expires_at),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
# User Authentication Methods
|
# User Authentication Methods
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
|
|||||||
+20
-8
@@ -1,17 +1,20 @@
|
|||||||
|
import os
|
||||||
import serpapi
|
import serpapi
|
||||||
from SPARC import config
|
from SPARC import config
|
||||||
import re
|
import re
|
||||||
import pdfplumber # pip install pdfplumber
|
import pdfplumber # pip install pdfplumber
|
||||||
import requests
|
import requests
|
||||||
|
from datetime import datetime, timedelta
|
||||||
from typing import Dict
|
from typing import Dict
|
||||||
from SPARC.types import Patents, Patent
|
from SPARC.types import Patents, Patent
|
||||||
|
|
||||||
class SERP:
|
class SERP:
|
||||||
def query(company: str) -> Patents:
|
def query(company: str, days_back: int = None) -> Patents:
|
||||||
"""Query Google Patents for a company's recent patents.
|
"""Query Google Patents for a company's recent patents.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
company: Name of the company to search for
|
company: Name of the company to search for
|
||||||
|
days_back: Number of days to look back for patents (default from config)
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Patents object containing list of patents with PDF links
|
Patents object containing list of patents with PDF links
|
||||||
@@ -23,13 +26,19 @@ class SERP:
|
|||||||
patents with restricted access). The returned count may be lower
|
patents with restricted access). The returned count may be lower
|
||||||
than the requested number of results.
|
than the requested number of results.
|
||||||
"""
|
"""
|
||||||
|
if days_back is None:
|
||||||
|
days_back = config.patent_search_days
|
||||||
|
end_date = datetime.now()
|
||||||
|
start_date = end_date - timedelta(days=days_back)
|
||||||
|
date_filter = f"cdr:1,cd_min:{start_date.strftime('%-m/%-d/%Y')},cd_max:{end_date.strftime('%-m/%-d/%Y')}"
|
||||||
|
|
||||||
# Make API call
|
# Make API call
|
||||||
params = {
|
params = {
|
||||||
"engine": "google_patents",
|
"engine": "google_patents",
|
||||||
"q": company,
|
"q": company,
|
||||||
"num": 10,
|
"num": 10,
|
||||||
"filter": 1,
|
"filter": 1,
|
||||||
"tbs": "cdr:1,cd_min:10/28/2025,cd_max:11/4/2025",
|
"tbs": date_filter,
|
||||||
"api_key": config.api_key,
|
"api_key": config.api_key,
|
||||||
}
|
}
|
||||||
search = serpapi.search(params)
|
search = serpapi.search(params)
|
||||||
@@ -46,7 +55,7 @@ class SERP:
|
|||||||
|
|
||||||
def save_patents(patent: Patent) -> Patent:
|
def save_patents(patent: Patent) -> Patent:
|
||||||
"""
|
"""
|
||||||
Save the patent PDF to the patents folder
|
Save the patent PDF to the patents folder, skipping download if already cached.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
patent: Patent object
|
patent: Patent object
|
||||||
@@ -54,12 +63,15 @@ class SERP:
|
|||||||
Returns:
|
Returns:
|
||||||
Patent object with updated PDF path
|
Patent object with updated PDF path
|
||||||
"""
|
"""
|
||||||
response = requests.get(patent.pdf_link)
|
pdf_path = f"patents/{patent.patent_id}.pdf"
|
||||||
print(patent.pdf_link)
|
os.makedirs("patents", exist_ok=True)
|
||||||
with open(f"patents/{patent.patent_id}.pdf", "wb") as f:
|
|
||||||
f.write(response.content)
|
|
||||||
|
|
||||||
patent.pdf_path = f"patents/{patent.patent_id}.pdf"
|
if not (os.path.exists(pdf_path) and os.path.getsize(pdf_path) > 0):
|
||||||
|
response = requests.get(patent.pdf_link)
|
||||||
|
with open(pdf_path, "wb") as f:
|
||||||
|
f.write(response.content)
|
||||||
|
|
||||||
|
patent.pdf_path = pdf_path
|
||||||
return patent
|
return patent
|
||||||
|
|
||||||
def parse_patent_pdf(pdf_path: str) -> Dict:
|
def parse_patent_pdf(pdf_path: str) -> Dict:
|
||||||
|
|||||||
+191
-3
@@ -1,11 +1,22 @@
|
|||||||
"""Tests for the high-level company analyzer orchestration."""
|
"""Tests for the high-level company analyzer orchestration."""
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from unittest.mock import Mock, patch, call
|
from unittest.mock import Mock, patch, call, MagicMock
|
||||||
from SPARC.analyzer import CompanyAnalyzer
|
from SPARC.analyzer import CompanyAnalyzer
|
||||||
from SPARC.types import Patent, Patents, CompanyAnalysisResult, BatchAnalysisResult
|
from SPARC.types import Patent, Patents, CompanyAnalysisResult, BatchAnalysisResult
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def mock_db(mocker):
|
||||||
|
"""Mock DatabaseClient for all tests so no real DB connection is needed."""
|
||||||
|
mock_db_cls = mocker.patch("SPARC.analyzer.DatabaseClient")
|
||||||
|
mock_db_instance = MagicMock()
|
||||||
|
mock_db_instance.get_cached_patent.return_value = None
|
||||||
|
mock_db_instance.get_cached_serp_query.return_value = None
|
||||||
|
mock_db_cls.return_value = mock_db_instance
|
||||||
|
return mock_db_instance
|
||||||
|
|
||||||
|
|
||||||
class TestCompanyAnalyzer:
|
class TestCompanyAnalyzer:
|
||||||
"""Test the CompanyAnalyzer orchestration logic."""
|
"""Test the CompanyAnalyzer orchestration logic."""
|
||||||
|
|
||||||
@@ -17,7 +28,7 @@ class TestCompanyAnalyzer:
|
|||||||
|
|
||||||
mock_llm.assert_called_once_with(api_key="test-key")
|
mock_llm.assert_called_once_with(api_key="test-key")
|
||||||
|
|
||||||
def test_analyze_company_full_pipeline(self, mocker):
|
def test_analyze_company_full_pipeline(self, mocker, mock_db):
|
||||||
"""Test complete company analysis pipeline."""
|
"""Test complete company analysis pipeline."""
|
||||||
# Mock all the dependencies
|
# Mock all the dependencies
|
||||||
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
||||||
@@ -178,6 +189,180 @@ class TestCompanyAnalyzer:
|
|||||||
assert "PDF not found" in result
|
assert "PDF not found" in result
|
||||||
|
|
||||||
|
|
||||||
|
class TestSingleQueryBugFix:
|
||||||
|
"""Test that SERP.query is only called once per company analysis."""
|
||||||
|
|
||||||
|
def test_analyze_company_safe_calls_query_once(self, mocker, mock_db):
|
||||||
|
"""_analyze_company_safe should call SERP.query exactly once."""
|
||||||
|
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
||||||
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
|
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
|
||||||
|
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
|
||||||
|
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
|
mock_query.return_value = Patents(patents=[patent])
|
||||||
|
|
||||||
|
def save_side_effect(p):
|
||||||
|
p.pdf_path = "patents/US123.pdf"
|
||||||
|
return p
|
||||||
|
|
||||||
|
mock_save.side_effect = save_side_effect
|
||||||
|
mock_parse.return_value = {"abstract": "Test"}
|
||||||
|
mock_minimize.return_value = "Content"
|
||||||
|
|
||||||
|
mock_llm_instance = Mock()
|
||||||
|
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
|
||||||
|
mock_llm.return_value = mock_llm_instance
|
||||||
|
|
||||||
|
analyzer = CompanyAnalyzer()
|
||||||
|
analyzer._analyze_company_safe("TestCorp")
|
||||||
|
|
||||||
|
# The key assertion: SERP.query called exactly once, not twice
|
||||||
|
mock_query.assert_called_once_with("TestCorp")
|
||||||
|
|
||||||
|
def test_analyze_company_with_prefetched_patents_skips_query(self, mocker):
|
||||||
|
"""analyze_company should not call SERP.query when patents are provided."""
|
||||||
|
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
||||||
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
|
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
|
||||||
|
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
|
||||||
|
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
|
prefetched = Patents(patents=[patent])
|
||||||
|
|
||||||
|
def save_side_effect(p):
|
||||||
|
p.pdf_path = "patents/US123.pdf"
|
||||||
|
return p
|
||||||
|
|
||||||
|
mock_save.side_effect = save_side_effect
|
||||||
|
mock_parse.return_value = {"abstract": "Test"}
|
||||||
|
mock_minimize.return_value = "Content"
|
||||||
|
|
||||||
|
mock_llm_instance = Mock()
|
||||||
|
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
|
||||||
|
mock_llm.return_value = mock_llm_instance
|
||||||
|
|
||||||
|
analyzer = CompanyAnalyzer()
|
||||||
|
analyzer.analyze_company("TestCorp", patents=prefetched)
|
||||||
|
|
||||||
|
# SERP.query should never be called
|
||||||
|
mock_query.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
class TestPatentCaching:
|
||||||
|
"""Test patent-level DB caching in the pipeline."""
|
||||||
|
|
||||||
|
def test_process_single_patent_uses_db_cache(self, mocker, mock_db):
|
||||||
|
"""_process_single_patent returns cached content when available."""
|
||||||
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
|
|
||||||
|
mock_db.get_cached_patent.return_value = {
|
||||||
|
"patent_id": "US123",
|
||||||
|
"minimized_content": "Cached minimized content",
|
||||||
|
}
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
|
result = CompanyAnalyzer._process_single_patent(patent, "TestCorp", mock_db)
|
||||||
|
|
||||||
|
assert result == {"patent_id": "US123", "content": "Cached minimized content"}
|
||||||
|
# Should NOT download since cache hit
|
||||||
|
mock_save.assert_not_called()
|
||||||
|
|
||||||
|
def test_process_single_patent_stores_to_db_cache(self, mocker, mock_db):
|
||||||
|
"""_process_single_patent stores result in DB after processing."""
|
||||||
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
|
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
|
||||||
|
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
|
||||||
|
|
||||||
|
# No cache hit
|
||||||
|
mock_db.get_cached_patent.return_value = None
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
|
|
||||||
|
def save_side_effect(p):
|
||||||
|
p.pdf_path = "patents/US123.pdf"
|
||||||
|
return p
|
||||||
|
|
||||||
|
mock_save.side_effect = save_side_effect
|
||||||
|
mock_parse.return_value = {"abstract": "Test abstract"}
|
||||||
|
mock_minimize.return_value = "Minimized content"
|
||||||
|
|
||||||
|
result = CompanyAnalyzer._process_single_patent(patent, "TestCorp", mock_db)
|
||||||
|
|
||||||
|
assert result == {"patent_id": "US123", "content": "Minimized content"}
|
||||||
|
mock_db.store_patent.assert_called_once_with(
|
||||||
|
patent_id="US123",
|
||||||
|
company_name="TestCorp",
|
||||||
|
pdf_link="http://example.com/test.pdf",
|
||||||
|
raw_sections={"abstract": "Test abstract"},
|
||||||
|
minimized_content="Minimized content",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_serp_query_cache_hit_skips_api(self, mocker, mock_db):
|
||||||
|
"""When SERP query is cached, API call is skipped."""
|
||||||
|
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
||||||
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
|
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
|
||||||
|
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
|
||||||
|
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
|
||||||
|
|
||||||
|
# Simulate SERP cache hit
|
||||||
|
mock_db.get_cached_serp_query.return_value = ["US123"]
|
||||||
|
# Simulate patent cache hit too
|
||||||
|
mock_db.get_cached_patent.return_value = {
|
||||||
|
"patent_id": "US123",
|
||||||
|
"minimized_content": "Cached content",
|
||||||
|
}
|
||||||
|
|
||||||
|
mock_llm_instance = Mock()
|
||||||
|
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
|
||||||
|
mock_llm.return_value = mock_llm_instance
|
||||||
|
|
||||||
|
analyzer = CompanyAnalyzer()
|
||||||
|
result = analyzer.analyze_company("TestCorp")
|
||||||
|
|
||||||
|
assert result == "Analysis"
|
||||||
|
# SERP.query should NOT be called
|
||||||
|
mock_query.assert_not_called()
|
||||||
|
# No downloads should happen
|
||||||
|
mock_save.assert_not_called()
|
||||||
|
|
||||||
|
def test_serp_query_cache_miss_stores_result(self, mocker, mock_db):
|
||||||
|
"""When SERP query cache misses, result is stored after API call."""
|
||||||
|
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
||||||
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
|
mock_parse = mocker.patch("SPARC.analyzer.SERP.parse_patent_pdf")
|
||||||
|
mock_minimize = mocker.patch("SPARC.analyzer.SERP.minimize_patent_for_llm")
|
||||||
|
mock_llm = mocker.patch("SPARC.analyzer.LLMAnalyzer")
|
||||||
|
|
||||||
|
mock_db.get_cached_serp_query.return_value = None
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
|
mock_query.return_value = Patents(patents=[patent])
|
||||||
|
|
||||||
|
def save_side_effect(p):
|
||||||
|
p.pdf_path = "patents/US123.pdf"
|
||||||
|
return p
|
||||||
|
|
||||||
|
mock_save.side_effect = save_side_effect
|
||||||
|
mock_parse.return_value = {"abstract": "Test"}
|
||||||
|
mock_minimize.return_value = "Content"
|
||||||
|
|
||||||
|
mock_llm_instance = Mock()
|
||||||
|
mock_llm_instance.analyze_patent_portfolio.return_value = "Analysis"
|
||||||
|
mock_llm.return_value = mock_llm_instance
|
||||||
|
|
||||||
|
analyzer = CompanyAnalyzer()
|
||||||
|
analyzer.analyze_company("TestCorp")
|
||||||
|
|
||||||
|
mock_db.store_serp_query.assert_called_once()
|
||||||
|
call_kwargs = mock_db.store_serp_query.call_args[1]
|
||||||
|
assert call_kwargs["company_name"] == "TestCorp"
|
||||||
|
assert call_kwargs["patent_ids"] == ["US123"]
|
||||||
|
|
||||||
|
|
||||||
class TestBatchProcessing:
|
class TestBatchProcessing:
|
||||||
"""Test multi-company batch processing functionality."""
|
"""Test multi-company batch processing functionality."""
|
||||||
|
|
||||||
@@ -316,7 +501,7 @@ class TestBatchProcessing:
|
|||||||
|
|
||||||
assert callback.call_count == 2
|
assert callback.call_count == 2
|
||||||
|
|
||||||
def test_company_analysis_result_structure(self, mocker):
|
def test_company_analysis_result_structure(self, mocker, mock_db):
|
||||||
"""Test CompanyAnalysisResult has correct structure."""
|
"""Test CompanyAnalysisResult has correct structure."""
|
||||||
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
mock_query = mocker.patch("SPARC.analyzer.SERP.query")
|
||||||
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
mock_save = mocker.patch("SPARC.analyzer.SERP.save_patents")
|
||||||
@@ -327,6 +512,9 @@ class TestBatchProcessing:
|
|||||||
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
mock_query.return_value = Patents(patents=[patent])
|
mock_query.return_value = Patents(patents=[patent])
|
||||||
|
|
||||||
|
# Simulate DB caching: after store, subsequent get returns the IDs
|
||||||
|
mock_db.get_cached_serp_query.side_effect = [None, ["US123"]]
|
||||||
|
|
||||||
def save_side_effect(p):
|
def save_side_effect(p):
|
||||||
p.pdf_path = "patents/US123.pdf"
|
p.pdf_path = "patents/US123.pdf"
|
||||||
return p
|
return p
|
||||||
|
|||||||
@@ -0,0 +1,302 @@
|
|||||||
|
"""Tests for JWT authentication flow: register, login, protected routes, refresh, admin access."""
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from SPARC.api import app
|
||||||
|
from SPARC.auth import create_access_token, create_refresh_token
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client():
|
||||||
|
"""Create test client."""
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def mock_db(monkeypatch):
|
||||||
|
"""Mock the database client used by auth endpoints.
|
||||||
|
|
||||||
|
Returns a MagicMock with all DB methods pre-configured.
|
||||||
|
"""
|
||||||
|
db = MagicMock()
|
||||||
|
|
||||||
|
# Default: no users exist
|
||||||
|
db.get_user_count.return_value = 0
|
||||||
|
db.get_user_by_id.return_value = None
|
||||||
|
db.get_user_by_email.return_value = None
|
||||||
|
db.authenticate_user.return_value = None
|
||||||
|
db.create_user.return_value = None
|
||||||
|
db.get_all_users.return_value = []
|
||||||
|
db.update_user_role.return_value = None
|
||||||
|
db.delete_user.return_value = False
|
||||||
|
|
||||||
|
with patch("SPARC.api.get_db_client", return_value=db), \
|
||||||
|
patch("SPARC.auth.get_db_client", return_value=db):
|
||||||
|
yield db
|
||||||
|
|
||||||
|
|
||||||
|
def _make_admin_user():
|
||||||
|
return {
|
||||||
|
"id": 1,
|
||||||
|
"email": "admin@test.com",
|
||||||
|
"role": "admin",
|
||||||
|
"created_at": datetime(2025, 1, 1, tzinfo=timezone.utc),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _make_regular_user():
|
||||||
|
return {
|
||||||
|
"id": 2,
|
||||||
|
"email": "user@test.com",
|
||||||
|
"role": "user",
|
||||||
|
"created_at": datetime(2025, 1, 1, tzinfo=timezone.utc),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _auth_header(user_dict):
|
||||||
|
"""Create an Authorization header with a valid access token for the given user."""
|
||||||
|
token = create_access_token(user_dict["id"], user_dict["email"], user_dict["role"])
|
||||||
|
return {"Authorization": f"Bearer {token}"}
|
||||||
|
|
||||||
|
|
||||||
|
class TestRegister:
|
||||||
|
"""POST /auth/register"""
|
||||||
|
|
||||||
|
def test_register_first_user_becomes_admin(self, client, mock_db):
|
||||||
|
"""First registered user should get admin role."""
|
||||||
|
mock_db.get_user_count.return_value = 0
|
||||||
|
mock_db.create_user.return_value = {
|
||||||
|
"id": 1,
|
||||||
|
"email": "admin@test.com",
|
||||||
|
"role": "admin",
|
||||||
|
"created_at": datetime(2025, 1, 1, tzinfo=timezone.utc),
|
||||||
|
}
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/register",
|
||||||
|
json={"email": "admin@test.com", "password": "securepass123"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert data["email"] == "admin@test.com"
|
||||||
|
assert data["role"] == "admin"
|
||||||
|
mock_db.create_user.assert_called_once_with(
|
||||||
|
email="admin@test.com", password="securepass123", role="admin"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_register_subsequent_user_gets_user_role(self, client, mock_db):
|
||||||
|
"""Non-first user should get regular user role."""
|
||||||
|
mock_db.get_user_count.return_value = 1
|
||||||
|
mock_db.create_user.return_value = _make_regular_user()
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/register",
|
||||||
|
json={"email": "user@test.com", "password": "securepass123"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert data["role"] == "user"
|
||||||
|
|
||||||
|
def test_register_duplicate_email_returns_400(self, client, mock_db):
|
||||||
|
"""Registering with an existing email should return 400."""
|
||||||
|
mock_db.get_user_count.return_value = 1
|
||||||
|
mock_db.create_user.return_value = None # indicates duplicate
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/register",
|
||||||
|
json={"email": "existing@test.com", "password": "securepass123"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 400
|
||||||
|
assert "already registered" in response.json()["detail"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
class TestLogin:
|
||||||
|
"""POST /auth/login"""
|
||||||
|
|
||||||
|
def test_login_valid_credentials_returns_tokens(self, client, mock_db):
|
||||||
|
"""Valid credentials should return access and refresh tokens."""
|
||||||
|
user = _make_regular_user()
|
||||||
|
mock_db.authenticate_user.return_value = user
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/login",
|
||||||
|
json={"email": "user@test.com", "password": "correctpassword"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert "access_token" in data
|
||||||
|
assert "refresh_token" in data
|
||||||
|
assert data["token_type"] == "bearer"
|
||||||
|
|
||||||
|
def test_login_invalid_credentials_returns_401(self, client, mock_db):
|
||||||
|
"""Invalid credentials should return 401."""
|
||||||
|
mock_db.authenticate_user.return_value = None
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/login",
|
||||||
|
json={"email": "user@test.com", "password": "wrongpassword"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 401
|
||||||
|
assert "invalid" in response.json()["detail"].lower()
|
||||||
|
|
||||||
|
|
||||||
|
class TestGetMe:
|
||||||
|
"""GET /auth/me"""
|
||||||
|
|
||||||
|
def test_valid_access_token_returns_user(self, client, mock_db):
|
||||||
|
"""A valid access token should return the user's data."""
|
||||||
|
user = _make_regular_user()
|
||||||
|
mock_db.get_user_by_id.return_value = user
|
||||||
|
|
||||||
|
response = client.get("/auth/me", headers=_auth_header(user))
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert data["email"] == "user@test.com"
|
||||||
|
assert data["id"] == 2
|
||||||
|
|
||||||
|
def test_missing_token_returns_401(self, client):
|
||||||
|
"""No token should return 401 (403 from HTTPBearer)."""
|
||||||
|
response = client.get("/auth/me")
|
||||||
|
assert response.status_code in (401, 403)
|
||||||
|
|
||||||
|
def test_expired_token_returns_401(self, client, mock_db):
|
||||||
|
"""An expired token should return 401."""
|
||||||
|
# Create a token that has already expired
|
||||||
|
from datetime import timedelta
|
||||||
|
|
||||||
|
import jwt as pyjwt
|
||||||
|
from SPARC.auth import JWT_ALGORITHM, JWT_SECRET
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"sub": "1",
|
||||||
|
"email": "user@test.com",
|
||||||
|
"role": "user",
|
||||||
|
"exp": datetime.now(timezone.utc) - timedelta(hours=1),
|
||||||
|
"type": "access",
|
||||||
|
}
|
||||||
|
expired_token = pyjwt.encode(payload, JWT_SECRET, algorithm=JWT_ALGORITHM)
|
||||||
|
|
||||||
|
response = client.get(
|
||||||
|
"/auth/me", headers={"Authorization": f"Bearer {expired_token}"}
|
||||||
|
)
|
||||||
|
assert response.status_code == 401
|
||||||
|
|
||||||
|
def test_refresh_token_as_access_returns_401(self, client, mock_db):
|
||||||
|
"""Using a refresh token as an access token should return 401."""
|
||||||
|
user = _make_regular_user()
|
||||||
|
refresh_token = create_refresh_token(user["id"], user["email"], user["role"])
|
||||||
|
|
||||||
|
response = client.get(
|
||||||
|
"/auth/me", headers={"Authorization": f"Bearer {refresh_token}"}
|
||||||
|
)
|
||||||
|
assert response.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
class TestRefreshToken:
|
||||||
|
"""POST /auth/refresh"""
|
||||||
|
|
||||||
|
def test_valid_refresh_token_returns_new_tokens(self, client, mock_db):
|
||||||
|
"""A valid refresh token should issue new access and refresh tokens."""
|
||||||
|
user = _make_regular_user()
|
||||||
|
mock_db.get_user_by_id.return_value = user
|
||||||
|
refresh = create_refresh_token(user["id"], user["email"], user["role"])
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/refresh", json={"refresh_token": refresh}
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert "access_token" in data
|
||||||
|
assert "refresh_token" in data
|
||||||
|
|
||||||
|
def test_invalid_refresh_token_returns_401(self, client, mock_db):
|
||||||
|
"""An invalid refresh token should return 401."""
|
||||||
|
response = client.post(
|
||||||
|
"/auth/refresh", json={"refresh_token": "invalid-token-string"}
|
||||||
|
)
|
||||||
|
assert response.status_code == 401
|
||||||
|
|
||||||
|
def test_access_token_as_refresh_returns_401(self, client, mock_db):
|
||||||
|
"""Using an access token as a refresh token should return 401."""
|
||||||
|
user = _make_regular_user()
|
||||||
|
access = create_access_token(user["id"], user["email"], user["role"])
|
||||||
|
|
||||||
|
response = client.post(
|
||||||
|
"/auth/refresh", json={"refresh_token": access}
|
||||||
|
)
|
||||||
|
assert response.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
|
class TestAdminUsers:
|
||||||
|
"""GET /admin/users and PATCH /admin/users/{id}/role"""
|
||||||
|
|
||||||
|
def test_admin_can_list_users(self, client, mock_db):
|
||||||
|
"""Admin token should allow listing users."""
|
||||||
|
admin = _make_admin_user()
|
||||||
|
mock_db.get_user_by_id.return_value = admin
|
||||||
|
mock_db.get_all_users.return_value = [admin, _make_regular_user()]
|
||||||
|
|
||||||
|
response = client.get("/admin/users", headers=_auth_header(admin))
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert len(data) == 2
|
||||||
|
|
||||||
|
def test_regular_user_cannot_list_users(self, client, mock_db):
|
||||||
|
"""Regular user token should be rejected with 403."""
|
||||||
|
user = _make_regular_user()
|
||||||
|
mock_db.get_user_by_id.return_value = user
|
||||||
|
|
||||||
|
response = client.get("/admin/users", headers=_auth_header(user))
|
||||||
|
|
||||||
|
assert response.status_code == 403
|
||||||
|
|
||||||
|
def test_no_token_cannot_list_users(self, client):
|
||||||
|
"""No token should be rejected."""
|
||||||
|
response = client.get("/admin/users")
|
||||||
|
assert response.status_code in (401, 403)
|
||||||
|
|
||||||
|
def test_admin_can_change_user_role(self, client, mock_db):
|
||||||
|
"""Admin should be able to change another user's role."""
|
||||||
|
admin = _make_admin_user()
|
||||||
|
mock_db.get_user_by_id.return_value = admin
|
||||||
|
mock_db.update_user_role.return_value = {
|
||||||
|
"id": 2,
|
||||||
|
"email": "user@test.com",
|
||||||
|
"role": "admin",
|
||||||
|
"created_at": datetime(2025, 1, 1, tzinfo=timezone.utc),
|
||||||
|
}
|
||||||
|
|
||||||
|
response = client.patch(
|
||||||
|
"/admin/users/2/role",
|
||||||
|
json={"role": "admin"},
|
||||||
|
headers=_auth_header(admin),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json()["role"] == "admin"
|
||||||
|
|
||||||
|
def test_admin_cannot_change_own_role(self, client, mock_db):
|
||||||
|
"""Admin should not be able to change their own role."""
|
||||||
|
admin = _make_admin_user()
|
||||||
|
mock_db.get_user_by_id.return_value = admin
|
||||||
|
|
||||||
|
response = client.patch(
|
||||||
|
"/admin/users/1/role",
|
||||||
|
json={"role": "user"},
|
||||||
|
headers=_auth_header(admin),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 400
|
||||||
|
assert "own role" in response.json()["detail"].lower()
|
||||||
@@ -1,7 +1,11 @@
|
|||||||
"""Tests for SERP API patent retrieval and parsing functionality."""
|
"""Tests for SERP API patent retrieval and parsing functionality."""
|
||||||
|
|
||||||
|
import os
|
||||||
import pytest
|
import pytest
|
||||||
|
from unittest.mock import patch, Mock
|
||||||
|
from datetime import datetime, timedelta
|
||||||
from SPARC.serp_api import SERP
|
from SPARC.serp_api import SERP
|
||||||
|
from SPARC.types import Patent
|
||||||
|
|
||||||
|
|
||||||
class TestTextCleaning:
|
class TestTextCleaning:
|
||||||
@@ -176,3 +180,89 @@ class TestPatentMinimization:
|
|||||||
|
|
||||||
# Sections should be separated by double newlines
|
# Sections should be separated by double newlines
|
||||||
assert "\n\n" in result
|
assert "\n\n" in result
|
||||||
|
|
||||||
|
|
||||||
|
class TestDynamicDateRange:
|
||||||
|
"""Test dynamic date range computation in SERP.query."""
|
||||||
|
|
||||||
|
def test_query_uses_rolling_date_window(self, mocker):
|
||||||
|
"""Verify the date filter uses a rolling window, not hardcoded dates."""
|
||||||
|
mock_search = mocker.patch("SPARC.serp_api.serpapi.search")
|
||||||
|
mock_search.return_value = {"organic_results": []}
|
||||||
|
mocker.patch("SPARC.serp_api.config.api_key", "fake-key")
|
||||||
|
mocker.patch("SPARC.serp_api.config.patent_search_days", 90)
|
||||||
|
|
||||||
|
SERP.query("TestCorp")
|
||||||
|
|
||||||
|
call_params = mock_search.call_args[0][0]
|
||||||
|
tbs = call_params["tbs"]
|
||||||
|
# Should contain "cdr:1,cd_min:" with a date, not the old hardcoded one
|
||||||
|
assert "cdr:1,cd_min:" in tbs
|
||||||
|
assert "10/28/2025" not in tbs # old hardcoded date gone
|
||||||
|
|
||||||
|
def test_query_respects_days_back_param(self, mocker):
|
||||||
|
"""Verify days_back parameter controls the date window."""
|
||||||
|
mock_search = mocker.patch("SPARC.serp_api.serpapi.search")
|
||||||
|
mock_search.return_value = {"organic_results": []}
|
||||||
|
mocker.patch("SPARC.serp_api.config.api_key", "fake-key")
|
||||||
|
mocker.patch("SPARC.serp_api.config.patent_search_days", 90)
|
||||||
|
|
||||||
|
now = datetime.now()
|
||||||
|
SERP.query("TestCorp", days_back=30)
|
||||||
|
|
||||||
|
call_params = mock_search.call_args[0][0]
|
||||||
|
tbs = call_params["tbs"]
|
||||||
|
expected_start = (now - timedelta(days=30)).strftime("%-m/%-d/%Y")
|
||||||
|
assert expected_start in tbs
|
||||||
|
|
||||||
|
|
||||||
|
class TestFilesystemPDFCaching:
|
||||||
|
"""Test that save_patents skips download for existing files."""
|
||||||
|
|
||||||
|
def test_save_patents_skips_download_when_cached(self, mocker, tmp_path):
|
||||||
|
"""Already-downloaded PDFs should not be re-downloaded."""
|
||||||
|
mock_get = mocker.patch("SPARC.serp_api.requests.get")
|
||||||
|
mocker.patch("SPARC.serp_api.os.makedirs")
|
||||||
|
|
||||||
|
pdf_path = tmp_path / "US123.pdf"
|
||||||
|
pdf_path.write_bytes(b"%PDF-1.4 fake content")
|
||||||
|
|
||||||
|
mocker.patch("SPARC.serp_api.os.path.exists", return_value=True)
|
||||||
|
mocker.patch("SPARC.serp_api.os.path.getsize", return_value=100)
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US123", pdf_link="http://example.com/test.pdf")
|
||||||
|
result = SERP.save_patents(patent)
|
||||||
|
|
||||||
|
mock_get.assert_not_called()
|
||||||
|
assert result.pdf_path == "patents/US123.pdf"
|
||||||
|
|
||||||
|
def test_save_patents_downloads_when_not_cached(self, mocker):
|
||||||
|
"""Missing PDFs should be downloaded."""
|
||||||
|
mock_response = Mock()
|
||||||
|
mock_response.content = b"%PDF-1.4 content"
|
||||||
|
mock_get = mocker.patch("SPARC.serp_api.requests.get", return_value=mock_response)
|
||||||
|
mocker.patch("SPARC.serp_api.os.makedirs")
|
||||||
|
mocker.patch("SPARC.serp_api.os.path.exists", return_value=False)
|
||||||
|
mock_open = mocker.patch("builtins.open", mocker.mock_open())
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US456", pdf_link="http://example.com/test.pdf")
|
||||||
|
result = SERP.save_patents(patent)
|
||||||
|
|
||||||
|
mock_get.assert_called_once_with("http://example.com/test.pdf")
|
||||||
|
assert result.pdf_path == "patents/US456.pdf"
|
||||||
|
|
||||||
|
def test_save_patents_redownloads_empty_files(self, mocker):
|
||||||
|
"""Empty/corrupt PDFs (0 bytes) should be re-downloaded."""
|
||||||
|
mock_response = Mock()
|
||||||
|
mock_response.content = b"%PDF-1.4 content"
|
||||||
|
mock_get = mocker.patch("SPARC.serp_api.requests.get", return_value=mock_response)
|
||||||
|
mocker.patch("SPARC.serp_api.os.makedirs")
|
||||||
|
mocker.patch("SPARC.serp_api.os.path.exists", return_value=True)
|
||||||
|
mocker.patch("SPARC.serp_api.os.path.getsize", return_value=0)
|
||||||
|
mock_open = mocker.patch("builtins.open", mocker.mock_open())
|
||||||
|
|
||||||
|
patent = Patent(patent_id="US789", pdf_link="http://example.com/test.pdf")
|
||||||
|
result = SERP.save_patents(patent)
|
||||||
|
|
||||||
|
mock_get.assert_called_once()
|
||||||
|
assert result.pdf_path == "patents/US789.pdf"
|
||||||
|
|||||||
Reference in New Issue
Block a user