Files
api-company/docs/secrets-checklist.md
T
AI-Agent c7b44b9da6
Validate Flux manifests / kustomize-build (pull_request) Failing after 33s
docs: update secrets-checklist.md item #4 for Alertmanager dual consumer
gatus-slack-webhook (monitoring) is now consumed by two components:
1. Gatus (flux/monitoring/gatus-helmrelease.yaml) - public status-page alerts
2. Alertmanager (flux/monitoring/helmrelease.yaml via alertmanagerSpec.secrets)
   - SLO / firing-alert Slack notifications (enabled by PR #212 / issue #210)

Updates:
- Checklist item #4: mention both Gatus AND Alertmanager Slack SLO alerts
- Secret Details §4: add Consumers row + dual-consumer note explaining
  alertmanagerSpec.secrets mounts this secret and the slack_api_url_file path
- Dependency order: note Alertmanager is also unblocked by item #4

Closes leeworks-agents/api-company#224
2026-06-06 10:02:04 +00:00

14 KiB
Raw Blame History

Kubernetes Secrets Checklist

All infrastructure blockers reduce to creating six Kubernetes secrets and one Gitea Actions secret. Follow this list top-to-bottom; each step unblocks the next.

Human operator only — the agent cannot log into Gitea's admin panel or run kubectl in the cluster.


Checklist

  • 1. gitea-leeworks-agents-token (flux-system) — unblocks Flux GitRepository auth
  • 2. gitea-runner-token (gitea-runner) — unblocks Gitea Actions runner registration
  • 3. grafana-admin (monitoring) — unblocks Grafana login
  • 4. gatus-slack-webhook (monitoring) — unblocks Gatus alert notifications AND Alertmanager Slack SLO alerts
  • 5. GITEA_TOKEN in each API repo's Actions Secrets — unblocks CI image push
  • 6. Gitea packages enabled + DNS record for registry.leeworks.dev — unblocks image push to registry
  • 7. Add api-company Flux source + kustomization to 0xWheatyz/Talos — unblocks all GitOps reconciliation
  • 8. gitea-registry (zip-enrichment, holidays, air-quality, docs-site) — imagePullSecret for pods pulling from registry.leeworks.dev
  • 9. gitea-image-automation-token (flux-system) — write-scoped token for Flux ImageUpdateAutomation to push image-tag commits
  • 10. rapidapi-proxy-secret (zip-enrichment, holidays, air-quality) — RapidAPI Proxy Secret for server-side request validation
  • 11. GITEA_TOKEN Actions secret in leeworks-agents/vin-decoder repo — enables CI image push for VIN Decoder (leeworks-agents/api-company#126)
  • 12. gitea-registry imagePullSecret in vin-decoder namespace — enables pod image pulls from registry.leeworks.dev (leeworks-agents/api-company#127)
  • 13. rapidapi-proxy-secret in vin-decoder namespace — enables RapidAPI proxy-secret header validation (leeworks-agents/api-company#128)

Secret Details

1. gitea-leeworks-agents-token

Field Value
Name gitea-leeworks-agents-token
Namespace flux-system
Purpose Flux GitRepository authenticates to Gitea over HTTPS to pull leeworks-agents/api-company
Source Gitea web UI → User Settings → Applications → Generate Token (scopes: read:repository)
Unblocks Issue #2 (Flux GitRepository + Kustomization for api-company)
kubectl create secret generic gitea-leeworks-agents-token \
  -n flux-system \
  --from-literal=username=leeworks-agents \
  --from-literal=password=<GITEA_TOKEN>

2. gitea-runner-token

Field Value
Name gitea-runner-token
Namespace gitea-runner
Purpose The gitea-act-runner HelmRelease reads this token to register the runner with Gitea
Source Gitea Admin Panel → Site Administration → Actions → Runners → Create new Runner — copy registration token
Unblocks Issue #3 (gitea-act-runner Flux deployment)
kubectl create secret generic gitea-runner-token \
  -n gitea-runner \
  --from-literal=token=<RUNNER_TOKEN>

After creating the secret, Flux reconciles the gitea-act-runner HelmRelease and the runner appears as Online in Gitea Admin → Actions → Runners.


3. grafana-admin

Field Value
Name grafana-admin
Namespace monitoring
Purpose Sets the Grafana admin user password on first boot
Source Choose a strong password and store it in a password manager
Unblocks Issue #7 (Prometheus + Grafana HelmRelease)
kubectl create secret generic grafana-admin \
  -n monitoring \
  --from-literal=admin-password=<PASSWORD>

Grafana will be accessible at https://grafana.leeworks.dev (login: admin / <PASSWORD>).


4. gatus-slack-webhook

Field Value
Name gatus-slack-webhook
Namespace monitoring
Purpose Gatus posts downtime alerts to a Slack channel via incoming webhook. Also mounted by Alertmanager (alertmanagerSpec.secrets) to send SLO / firing-alert Slack notifications.
Source Slack → Your workspace → Apps → Incoming Webhooks → Add to Slack → copy webhook URL
Unblocks Issue #8 (Gatus status page at status.leeworks.dev); Alertmanager Slack receiver (issue #210)
Consumers Two components share this secret: (1) Gatus (flux/monitoring/gatus-helmrelease.yaml) — public status-page alerts; (2) Alertmanager (flux/monitoring/helmrelease.yaml via alertmanagerSpec.secrets) — SLO / firing-alert Slack notifications

Note (dual consumer): Creating this one secret (gatus-slack-webhook in the monitoring namespace) unblocks both Gatus alert delivery AND Alertmanager's Slack receiver. The Alertmanager configuration references this secret at global.slack_api_url_file: /etc/alertmanager/secrets/gatus-slack-webhook/url (enabled in PR #212 / issue #210).

kubectl create secret generic gatus-slack-webhook \
  -n monitoring \
  --from-literal=url=https://hooks.slack.com/services/YOUR/WEBHOOK/URL

5. GITEA_TOKEN — Gitea Actions Secret (per repo)

Field Value
Name GITEA_TOKEN
Scope Gitea Actions Secret — set in each repo's Settings, not a Kubernetes secret
Purpose CI workflows use this token to push container images to registry.leeworks.dev
Source Same token as step 1, or a dedicated CI token with write:packages scope
Unblocks CI pipelines for all three API repos

Set in Gitea web UI for each of these repos:

  • leeworks-agents/api-company
  • leeworks-agents/zip-enrichment
  • leeworks-agents/holidays
  • leeworks-agents/air-quality

Path: Repo → Settings → Actions → Secrets → Add Secret

  • Name: GITEA_TOKEN
  • Value: <GITEA_TOKEN>

6. Enable Gitea Packages + DNS for registry.leeworks.dev

This is a Gitea instance configuration step, not a Kubernetes secret.

Step Action
6a Enable packages in Gitea app.ini: set [packages] ENABLED = true then restart Gitea
6b Add DNS A record: registry.leeworks.dev → cluster ingress IP

Find cluster ingress IP:

kubectl get svc -n ingress-nginx

See docs/registry.md for context on why the Gitea built-in registry was chosen.

Unblocks: Issue #4 (container registry), and transitively all CI image-push workflows.


7. Add api-company Flux Source + Kustomization to 0xWheatyz/Talos

Reference manifests are already committed at flux/api-company-source/ in this repo. The operator must copy them into the Talos cluster repo so FluxCD picks them up:

0xWheatyz/Talos:testing1/first-cluster/cluster/flux/api-company-source/

Unblocks: Issue #2 (Flux reconciliation of all flux/ manifests in this repo).



10. rapidapi-proxy-secret — RapidAPI Proxy Secret (per API namespace)

Field Value
Name rapidapi-proxy-secret
Namespaces zip-enrichment, holidays, air-quality
Purpose Every API service validates the X-RapidAPI-Proxy-Secret header on every route. Requests without a valid secret return HTTP 403.
Source RapidAPI dashboard → API Settings → Security → Proxy Secret (generated after each API listing is created)
Unblocks Phase 3 server middleware; API services will start but reject all traffic without this secret
for NS in zip-enrichment holidays air-quality; do
  kubectl create secret generic rapidapi-proxy-secret \
    --namespace=$NS \
    --from-literal=X-RapidAPI-Proxy-Secret=<value-from-rapidapi-dashboard>
done

Source: RapidAPI dashboard → select your API → Settings → Security → Proxy Secret

Note: Placeholder ExternalSecret manifests are committed at flux/zip-enrichment/externalsecret.yaml, flux/holidays/externalsecret.yaml, and flux/air-quality/externalsecret.yaml. These will auto-sync this secret from the configured backend once the External Secrets Operator (ESO) is deployed (see issue #61). Until then, create manually using the commands above.

Verify:

for NS in zip-enrichment holidays air-quality; do
  echo -n "$NS: "
  kubectl get secret rapidapi-proxy-secret -n $NS -o jsonpath='{.data.X-RapidAPI-Proxy-Secret}' | base64 -d | wc -c
  echo " chars"
done

11. GITEA_TOKEN Actions secret in leeworks-agents/vin-decoder

Field Value
Name GITEA_TOKEN
Scope Gitea Actions Secret — set in repo Settings
Purpose CI workflow pushes container image to registry.leeworks.dev/vin-decoder/api:<sha>
Source Gitea token with write:packages scope (reuse from item #5 if it has write:packages)
Tracked leeworks-agents/api-company#126
Unblocks CI image push for VIN Decoder

Path: Gitea → leeworks-agents/vin-decoder → Settings → Secrets → GITEA_TOKEN


12. gitea-registry imagePullSecret in vin-decoder namespace

Field Value
Name gitea-registry
Namespace vin-decoder
Type kubernetes.io/dockerconfigjson
Purpose Allows VIN Decoder pods to pull images from registry.leeworks.dev without ImagePullBackOff
Tracked leeworks-agents/api-company#127
kubectl create namespace vin-decoder --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret docker-registry gitea-registry \
  --namespace=vin-decoder \
  --docker-server=registry.leeworks.dev \
  --docker-username=leeworks-agents \
  --docker-password=<GITEA_TOKEN_WITH_READ_PACKAGES> \
  --docker-email=agent@leeworks.dev

13. rapidapi-proxy-secret in vin-decoder namespace

Field Value
Name rapidapi-proxy-secret
Namespace vin-decoder
Purpose VIN Decoder validates X-RapidAPI-Proxy-Secret header; returns HTTP 403 if missing/wrong
Source RapidAPI dashboard → VIN Decoder listing → Settings → Security → Proxy Secret
Tracked leeworks-agents/api-company#128
# Placeholder (unblocks deploy testing):
kubectl create secret generic rapidapi-proxy-secret \
  --namespace=vin-decoder \
  --from-literal=X-RapidAPI-Proxy-Secret=PLACEHOLDER_REPLACE_AFTER_RAPIDAPI_LISTING

# Update with real value after RapidAPI listing (#131) is live:
kubectl create secret generic rapidapi-proxy-secret \
  -n vin-decoder \
  --from-literal=X-RapidAPI-Proxy-Secret=<VIN_RAPIDAPI_PROXY_SECRET> \
  --save-config --dry-run=client -o yaml | kubectl apply -f -

Dependency Order

7 (Flux wiring)          → all flux/ resources reconcile
1 (gitea-leeworks-token) → Flux can pull this repo over HTTPS
2 (gitea-runner-token)   → runner online → CI runs
3 (grafana-admin)        → Grafana login works
4 (gatus-slack-webhook)  → Gatus alerting works + Alertmanager Slack SLO alerts
5 + 6 (GITEA_TOKEN + registry packages) → CI pushes images → API services deploy
8 (gitea-registry)       → pods can pull images from registry.leeworks.dev → services start
9 (gitea-image-automation-token) → Flux ImageUpdateAutomation pushes tag-update commits
10 (rapidapi-proxy-secret × 3)  → API server middleware validates RapidAPI requests → revenue enabled

Once all nine items are complete, the full stack (runner, registry, Prometheus, Grafana, Gatus, docs-site, three API services) reconciles automatically via FluxCD with no further manual steps.


8. gitea-registry — imagePullSecret for API service namespaces

Field Value
Name gitea-registry
Namespaces zip-enrichment, holidays, air-quality, docs-site
Type kubernetes.io/dockerconfigjson
Purpose Allows pods to pull images from registry.leeworks.dev without ImagePullBackOff
Source Gitea token with read:packages scope (can reuse the same token as step 1 if it has that scope)
Unblocks Issues #58 (Phase 0 hard deploy blocker) and transitively Phase 3 service deploys
for NS in zip-enrichment holidays air-quality docs-site; do
  kubectl create secret docker-registry gitea-registry \
    --namespace=$NS \
    --docker-server=registry.leeworks.dev \
    --docker-username=leeworks-agents \
    --docker-password=<GITEA_TOKEN_WITH_READ_PACKAGES> \
    --docker-email=agent@leeworks.dev
done

Verify:

kubectl get secret gitea-registry -n zip-enrichment -o jsonpath='{.type}'
# expected: kubernetes.io/dockerconfigjson

9. gitea-image-automation-token — Flux ImageUpdateAutomation write token

Field Value
Name gitea-image-automation-token
Namespace flux-system
Purpose Allows Flux ImageUpdateAutomation to push image-tag update commits back to leeworks-agents/api-company
Source Gitea token with write:repository scope (the existing gitea-leeworks-agents-token only has read:repository — create a separate token or verify scope)
Unblocks Issue #57 (Flux ImageUpdateAutomation for api-company)
kubectl create secret generic gitea-image-automation-token \
  -n flux-system \
  --from-literal=username=leeworks-agents \
  --from-literal=password=<TOKEN_WITH_WRITE_REPO_SCOPE>

Verify after creation:

flux get imageupdateautomations -n flux-system
# Expected: api-company shows READY=True

Note: If you create a new token with write:repository scope, keep the existing gitea-leeworks-agents-token for read-only Flux GitRepository pulls and use this new secret exclusively for ImageUpdateAutomation.