Compare commits

..

12 Commits

Author SHA1 Message Date
agent-company 9370d2c898 [Phase 6] ci: add scripts/publish-openapi.js and publish-openapi workflow
Validate Flux manifests / kustomize-build (pull_request) Failing after 8s
Closes leeworks-agents/api-company#100

- scripts/publish-openapi.js: Node.js script using only built-in modules
  (https, fs, path — no extra deps) that uploads each API's openapi.yaml
  to the RapidAPI Platform API via PUT multipart/form-data.
  - Reads RAPIDAPI_KEY/RAPIDAPI_PLATFORM_KEY from environment.
  - Skips silently per API if API ID / version ID not set.
  - Skips all uploads if RAPIDAPI_KEY not set (exits 0).
  - Exits non-zero on any HTTP error.
  - Logs: '✓ Published {api-name} spec to RapidAPI' on success.
  - Passes 'node --check' syntax validation.
- .gitea/workflows/publish-openapi.yaml: triggers on push to main when
  apis/*/openapi.yaml changes.
  - Runs Redocly CLI lint (fails pipeline on spec errors).
  - Detects changed specs via git diff.
  - Calls publish-openapi.js; no secrets = skip, not crash.
2026-05-29 20:04:26 +00:00
AI-Manager 7cfcd0f46b Merge pull request '[Phase 0] ci: add no-op smoke-test workflow to verify Act Runner (#96)' (#98) from feature/issue-96-smoke-test-workflow into main
Validate Flux manifests / kustomize-build (push) Failing after 17s
Build Docs Site / Aggregate OpenAPI Specs (push) Failing after 29s
2026-05-29 15:03:23 +00:00
agent-company 2f249a9e18 [Phase 0] ci: add no-op smoke-test workflow to verify Act Runner
Validate Flux manifests / kustomize-build (pull_request) Failing after 21s
Adds .gitea/workflows/smoke-test.yaml as a manual workflow_dispatch trigger
that runs four echo commands.  When the Act Runner comes online (#77) the
operator can fire this from the Gitea Actions UI to confirm runner
registration and job dispatch work before any real tooling is needed.

Closes leeworks-agents/api-company#96
2026-05-29 15:03:14 +00:00
AI-Manager cea4658b91 Merge pull request '[Phase 0] flux: add postBuild.substituteFrom for grafana-admin secret (#97)' (#99) from feature/issue-97-flux-postbuild-substitutefrom into main
Validate Flux manifests / kustomize-build (push) Failing after 17s
Build Docs Site / Aggregate OpenAPI Specs (push) Failing after 44s
2026-05-29 15:03:00 +00:00
agent-company c5a8f9f58c [Phase 0] flux: add postBuild.substituteFrom for grafana-admin secret
Validate Flux manifests / kustomize-build (pull_request) Failing after 15s
Adds postBuild.substituteFrom to the Flux Kustomization reference manifest
so that ${GRAFANA_ADMIN_PASSWORD} in flux/monitoring/helmrelease.yaml is
substituted from the grafana-admin secret at reconcile time.

Without this block Flux passes the literal string to the Helm chart, causing
Grafana to start with a broken admin password.

The live copy in 0xWheatyz/Talos at
testing1/first-cluster/cluster/flux/api-company-source/kustomization.yaml
must also be updated with the same block (companion to issue #90).

Closes leeworks-agents/api-company#97
2026-05-29 15:02:13 +00:00
AI-Manager 8e64a9e2ac Merge pull request '[Phase 0] docs: add docs/tooling.md with tooling versions and git remote config' (#95) from feature/docs-tooling-md into main
Validate Flux manifests / kustomize-build (push) Failing after 21s
Build Docs Site / Aggregate OpenAPI Specs (push) Failing after 44s
2026-05-29 10:02:32 +00:00
agent-company 348baf96bb docs: add docs/tooling.md with Phase 0 tooling versions and git remote config
Validate Flux manifests / kustomize-build (pull_request) Failing after 22s
Documents Node.js v22.22.2, Python 3.11.2, Helm v3.21.0, Flux CLI v2.4.0,
kustomize v5.6.0 as available in the agent container. Confirms git remote
points to gitea.leeworks.dev/leeworks-agents/api-company. Includes stack
summary (runtime choices, Helm registry, Flux bootstrap reference) and
a live-verification block for operator use.

Closes leeworks-agents/api-company#94
2026-05-29 10:02:06 +00:00
AI-Manager cb9909172b Merge pull request '[Phase 3] fix(flux): move bedag HelmRepository to top-level to fix Flux race condition' (#93) from feature/fix-bedag-helmrepository-87 into main
Validate Flux manifests / kustomize-build (push) Failing after 8s
Build Docs Site / Aggregate OpenAPI Specs (push) Failing after 43s
2026-05-29 05:06:11 +00:00
AI-Manager 38cddd6dfc Merge pull request '[Phase 4] feat(monitoring): add APIDataCriticallyStale alert and fix APIDataStale for duration' (#92) from feature/add-data-freshness-alerts-88 into main
Build Docs Site / Aggregate OpenAPI Specs (push) Failing after 38s
Validate Flux manifests / kustomize-build (push) Failing after 18s
2026-05-29 05:06:00 +00:00
AI-Manager cd0ed86571 Merge pull request '[Phase 5] feat(docs-site): add legal pages (ToS, Privacy Policy, AUP)' (#91) from feature/add-legal-pages-89 into main
Validate Flux manifests / kustomize-build (push) Failing after 5s
Build Docs Site / Aggregate OpenAPI Specs (push) Failing after 43s
2026-05-29 05:05:55 +00:00
agent-company 1a1a109aa3 feat(monitoring): add APIDataCriticallyStale alert and fix APIDataStale for duration
Validate Flux manifests / kustomize-build (pull_request) Failing after 17s
- Fix APIDataStale 'for' duration from 5m to 30m (issue spec requires 30m)
- Add APIDataCriticallyStale alert at 60-day threshold (5184000s) with for: 1h
  and severity: critical

Both alerts fire on api_data_freshness_seconds metric for all three API jobs.
Existing APIHighErrorRate and APIHighLatency alerts are unmodified.
kustomize build flux/ passes.

Closes leeworks-agents/api-company#88
2026-05-29 05:04:56 +00:00
agent-company b616e11bdf fix(flux): move bedag HelmRepository to top-level to fix Flux race condition
Validate Flux manifests / kustomize-build (pull_request) Failing after 13s
The bedag HelmRepository was only declared inside flux/docs-site/, causing
a race condition where zip-enrichment, holidays, and air-quality HelmReleases
would fail with 'HelmRepository not found' if they reconciled before docs-site.

Move flux/docs-site/helmrepository.yaml -> flux/bedag-helmrepository.yaml and
add it as a top-level resource in flux/kustomization.yaml so it is always
available before any HelmRelease that depends on the bedag chart.

Closes leeworks-agents/api-company#87
2026-05-29 05:04:48 +00:00
9 changed files with 310 additions and 3 deletions
+58
View File
@@ -0,0 +1,58 @@
# Validate and publish OpenAPI specs to RapidAPI when apis/*/openapi.yaml changes on main.
# Requires secrets (configured once RapidAPI listings are live):
# RAPIDAPI_PLATFORM_KEY, RAPIDAPI_ZIP_API_ID, RAPIDAPI_ZIP_VERSION_ID,
# RAPIDAPI_HOLIDAYS_API_ID, RAPIDAPI_HOLIDAYS_VERSION_ID,
# RAPIDAPI_AQI_API_ID, RAPIDAPI_AQI_VERSION_ID
#
# If RAPIDAPI_PLATFORM_KEY is not set the publish step exits 0 with a skip message.
name: Validate and Publish OpenAPI Specs
on:
push:
branches:
- main
paths:
- 'apis/*/openapi.yaml'
jobs:
publish:
name: Lint and publish specs
runs-on: ubuntu-latest
steps:
- name: Checkout (with history for diff)
uses: actions/checkout@v4
with:
fetch-depth: 2
- name: Set up Node.js 20
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install Redocly CLI
run: npm install -g @redocly/cli@latest
- name: Lint OpenAPI specs
run: |
echo "Linting all OpenAPI specs..."
npx @redocly/cli lint apis/*/openapi.yaml
- name: Detect changed specs
id: changed
run: |
changed=$(git diff --name-only HEAD~1 HEAD | grep 'openapi\.yaml' || true)
echo "Changed specs: ${changed:-none}"
echo "files=${changed}" >> "$GITHUB_OUTPUT"
- name: Publish specs to RapidAPI
env:
RAPIDAPI_KEY: ${{ secrets.RAPIDAPI_PLATFORM_KEY }}
RAPIDAPI_ZIP_API_ID: ${{ secrets.RAPIDAPI_ZIP_API_ID }}
RAPIDAPI_ZIP_VERSION_ID: ${{ secrets.RAPIDAPI_ZIP_VERSION_ID }}
RAPIDAPI_HOLIDAYS_API_ID: ${{ secrets.RAPIDAPI_HOLIDAYS_API_ID }}
RAPIDAPI_HOLIDAYS_VERSION_ID: ${{ secrets.RAPIDAPI_HOLIDAYS_VERSION_ID }}
RAPIDAPI_AQI_API_ID: ${{ secrets.RAPIDAPI_AQI_API_ID }}
RAPIDAPI_AQI_VERSION_ID: ${{ secrets.RAPIDAPI_AQI_VERSION_ID }}
run: node scripts/publish-openapi.js
+21
View File
@@ -0,0 +1,21 @@
# Smoke test: confirms Gitea Act Runner is online and accepting jobs.
# Run manually via workflow_dispatch after runner is registered (#77).
# Closes leeworks-agents/api-company#96
name: Runner Smoke Test
on:
workflow_dispatch:
jobs:
smoke:
name: Smoke Test
runs-on: ubuntu-latest
steps:
- name: Confirm runner is online
run: |
echo "Runner is online!"
echo "Job ID: $GITHUB_JOB"
echo "Runner OS: $(uname -a)"
echo "Date: $(date -u)"
echo "Smoke test PASSED"
+74
View File
@@ -0,0 +1,74 @@
# Tooling Versions
> Phase 0 checklist item — documents the local agent container tooling and confirms git remote config.
> Live cluster verification (kubectl/flux against the running cluster) requires operator confirmation.
## Agent Container Tooling
| Tool | Version | Source |
|------|---------|--------|
| Node.js | v22.22.2 | `node --version` |
| Python 3 | 3.11.2 | `python3 --version` |
| Helm | v3.21.0 | `helm version --short` |
| Flux CLI | v2.4.0 | `flux version --client` |
| kustomize | v5.6.0 | `kustomize version` |
| kubectl | available | `kubectl` (cluster context not set in agent container — live commands require kubeconfig from operator) |
## Git Remote Config
```
origin ssh://git@gitea.leeworks.dev/leeworks-agents/api-company (fetch)
origin ssh://git@gitea.leeworks.dev/leeworks-agents/api-company (push)
upstream ssh://git@gitea.leeworks.dev/0xWheatyz/api-company (fetch)
upstream ssh://git@gitea.leeworks.dev/0xWheatyz/api-company (push)
```
Remote `origin` confirmed pointing to `gitea.leeworks.dev/leeworks-agents/api-company`.
> Note: `0xWheatyz/api-company` upstream repo does not yet exist on Gitea — tracked by issue #47.
## Stack Summary
### Runtime choices
| API Service | Runtime | Rationale |
|-------------|---------|-----------|
| zip-enrichment | Node.js (Fastify) | Lightweight, fast JSON serialisation; large ZIP dataset fits SQLite well |
| holidays | Node.js (Fastify) | Simple lookup API; Fastify handles high req/s with low memory |
| air-quality | Python (FastAPI) | AQI ingestion benefits from Python data-science ecosystem |
| docs-site | Astro (static) | Zero-runtime static site; built by CI, served from container |
### Helm chart registry
All HelmReleases use charts sourced from public Helm chart repositories declared in `flux/` as `HelmRepository` resources:
- `flux/gitea-runner/helmrepository.yaml` — Gitea Act Runner chart
- `flux/monitoring/``kube-prometheus-stack` and Gatus charts
- `flux/zip-enrichment/`, `flux/holidays/`, `flux/air-quality/` — per-API service charts
The in-cluster container registry is `registry.leeworks.dev` (Gitea built-in packages/container registry, documented in `docs/registry.md`).
### Flux version and bootstrap
- **Flux CLI**: v2.4.0 (available in agent container for manifest authoring)
- **Flux controllers**: bootstrapped into `0xWheatyz/Talos` cluster (FluxCD v2.x)
- `flux-system` namespace managed by Talos GitOps repo
- Flux watches `0xWheatyz/Talos``testing1/first-cluster/cluster/`
- `api-company-source` GitRepository + Kustomization adds this repo to Flux (issues #2, #90)
- **Bootstrap reference**: `0xWheatyz/Talos` repo — see Talos cluster documentation
## Live Verification (Operator)
The following commands require a valid `kubeconfig` (not available in agent container):
```bash
kubectl config current-context # confirm cluster context
kubectl version --client # kubectl client version
flux check # Flux controller health
flux get sources git api-company # GitRepository READY status
flux get kustomizations api-company # Kustomization READY status
helm version # Helm client version
kustomize version # kustomize version
```
_Reference: MASTER_BUILD_PROMPT.md §Phase 0 step 2 — Closes leeworks-agents/api-company#94_
+7 -1
View File
@@ -1,7 +1,8 @@
# This manifest is FOR REFERENCE — the live version must be committed to
# 0xWheatyz/Talos at testing1/first-cluster/cluster/flux/api-company/
# 0xWheatyz/Talos at testing1/first-cluster/cluster/flux/api-company-source/
#
# See leeworks-agents/api-company#2
# See leeworks-agents/api-company#97
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
@@ -17,3 +18,8 @@ spec:
prune: true
wait: true
timeout: 5m
postBuild:
substituteFrom:
- kind: Secret
name: grafana-admin
optional: false
-1
View File
@@ -2,5 +2,4 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- helmrepository.yaml
- helmrelease.yaml
+1
View File
@@ -1,6 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- bedag-helmrepository.yaml
- external-secrets
- gitea-runner
- monitoring
+15 -1
View File
@@ -71,7 +71,7 @@ spec:
- alert: APIDataStale
expr: |
api_data_freshness_seconds{job=~"zip|holidays|air-quality"} > 2592000
for: 5m
for: 30m
labels:
severity: warning
team: api-company
@@ -79,6 +79,20 @@ spec:
summary: "Stale dataset on {{ $labels.job }} ({{ $labels.dataset }})"
description: "{{ $labels.job }} dataset '{{ $labels.dataset }}' has not been re-seeded in {{ $value | humanizeDuration }} (threshold: 30 days). Re-seed required."
# -------------------------------------------------------------------
# APIDataCriticallyStale — data freshness > 60 days
# -------------------------------------------------------------------
- alert: APIDataCriticallyStale
expr: |
api_data_freshness_seconds{job=~"zip|holidays|air-quality"} > 5184000
for: 1h
labels:
severity: critical
team: api-company
annotations:
summary: "API data is critically stale on {{ $labels.job }}"
description: "{{ $labels.job }} data has not been re-seeded in more than 60 days ({{ $value | humanizeDuration }})"
# -------------------------------------------------------------------
# APIDown — any API job absent for 2 min
# -------------------------------------------------------------------
+134
View File
@@ -0,0 +1,134 @@
#!/usr/bin/env node
/**
* publish-openapi.js
*
* Uploads each API's openapi.yaml to the RapidAPI Platform API.
* Environment variables required per API (skips silently if not set):
* RAPIDAPI_KEY — RapidAPI Platform API bearer token
* RAPIDAPI_ZIP_API_ID — API ID for ZIP Enrichment on RapidAPI
* RAPIDAPI_ZIP_VERSION_ID — Version ID for ZIP Enrichment
* RAPIDAPI_HOLIDAYS_API_ID
* RAPIDAPI_HOLIDAYS_VERSION_ID
* RAPIDAPI_AQI_API_ID
* RAPIDAPI_AQI_VERSION_ID
*
* Usage: node scripts/publish-openapi.js
*/
const fs = require('fs');
const path = require('path');
const https = require('https');
const RAPIDAPI_KEY = process.env.RAPIDAPI_KEY || process.env.RAPIDAPI_PLATFORM_KEY;
if (!RAPIDAPI_KEY) {
console.log('⚠️ RAPIDAPI_KEY / RAPIDAPI_PLATFORM_KEY not set — skipping all spec uploads.');
process.exit(0);
}
const APIS = [
{
name: 'zip-enrichment',
specPath: path.join(__dirname, '..', 'apis', 'zip-enrichment', 'openapi.yaml'),
apiId: process.env.RAPIDAPI_ZIP_API_ID,
versionId: process.env.RAPIDAPI_ZIP_VERSION_ID,
},
{
name: 'holidays',
specPath: path.join(__dirname, '..', 'apis', 'holidays', 'openapi.yaml'),
apiId: process.env.RAPIDAPI_HOLIDAYS_API_ID,
versionId: process.env.RAPIDAPI_HOLIDAYS_VERSION_ID,
},
{
name: 'air-quality',
specPath: path.join(__dirname, '..', 'apis', 'air-quality', 'openapi.yaml'),
apiId: process.env.RAPIDAPI_AQI_API_ID,
versionId: process.env.RAPIDAPI_AQI_VERSION_ID,
},
];
/**
* Build a multipart/form-data body from a file buffer.
* Returns { body: Buffer, boundary: string }
*/
function buildMultipart(fieldName, filename, fileBuffer, contentType = 'application/yaml') {
const boundary = '----FormBoundary' + Math.random().toString(36).slice(2);
const CRLF = '\r\n';
const parts = [
Buffer.from(
`--${boundary}${CRLF}` +
`Content-Disposition: form-data; name="${fieldName}"; filename="${filename}"${CRLF}` +
`Content-Type: ${contentType}${CRLF}${CRLF}`
),
fileBuffer,
Buffer.from(`${CRLF}--${boundary}--${CRLF}`),
];
return { body: Buffer.concat(parts), boundary };
}
/**
* Upload a spec file to RapidAPI Platform API.
* Returns a promise that resolves with the response status code.
*/
function uploadSpec(api) {
return new Promise((resolve, reject) => {
if (!api.apiId || !api.versionId) {
console.log(`⏭️ Skipping ${api.name}: API ID or Version ID not configured.`);
return resolve(null);
}
if (!fs.existsSync(api.specPath)) {
console.log(`⏭️ Skipping ${api.name}: spec file not found at ${api.specPath}`);
return resolve(null);
}
const fileBuffer = fs.readFileSync(api.specPath);
const { body, boundary } = buildMultipart('spec', 'openapi.yaml', fileBuffer);
const options = {
hostname: 'platformapi1.p.rapidapi.com',
path: `/v1/apis/${api.apiId}/versions/${api.versionId}`,
method: 'PUT',
headers: {
'Authorization': `Bearer ${RAPIDAPI_KEY}`,
'X-RapidAPI-Key': RAPIDAPI_KEY,
'Content-Type': `multipart/form-data; boundary=${boundary}`,
'Content-Length': body.length,
},
};
const req = https.request(options, (res) => {
let data = '';
res.on('data', (chunk) => { data += chunk; });
res.on('end', () => {
if (res.statusCode >= 200 && res.statusCode < 300) {
console.log(`✓ Published ${api.name} spec to RapidAPI (HTTP ${res.statusCode})`);
resolve(res.statusCode);
} else {
reject(new Error(`Failed to publish ${api.name}: HTTP ${res.statusCode}${data}`));
}
});
});
req.on('error', reject);
req.write(body);
req.end();
});
}
async function main() {
let hasError = false;
for (const api of APIS) {
try {
await uploadSpec(api);
} catch (err) {
console.error(`${err.message}`);
hasError = true;
}
}
if (hasError) {
process.exit(1);
}
}
main();