AI-Agent
c7b44b9da6
Validate Flux manifests / kustomize-build (pull_request) Failing after 33s
gatus-slack-webhook (monitoring) is now consumed by two components: 1. Gatus (flux/monitoring/gatus-helmrelease.yaml) - public status-page alerts 2. Alertmanager (flux/monitoring/helmrelease.yaml via alertmanagerSpec.secrets) - SLO / firing-alert Slack notifications (enabled by PR #212 / issue #210) Updates: - Checklist item #4: mention both Gatus AND Alertmanager Slack SLO alerts - Secret Details §4: add Consumers row + dual-consumer note explaining alertmanagerSpec.secrets mounts this secret and the slack_api_url_file path - Dependency order: note Alertmanager is also unblocked by item #4 Closes leeworks-agents/api-company#224
337 lines
14 KiB
Markdown
337 lines
14 KiB
Markdown
# Kubernetes Secrets Checklist
|
||
|
||
All infrastructure blockers reduce to creating six Kubernetes secrets and one Gitea Actions secret.
|
||
Follow this list top-to-bottom; each step unblocks the next.
|
||
|
||
**Human operator only** — the agent cannot log into Gitea's admin panel or run `kubectl` in the cluster.
|
||
|
||
---
|
||
|
||
## Checklist
|
||
|
||
- [ ] 1. `gitea-leeworks-agents-token` (flux-system) — unblocks Flux GitRepository auth
|
||
- [ ] 2. `gitea-runner-token` (gitea-runner) — unblocks Gitea Actions runner registration
|
||
- [ ] 3. `grafana-admin` (monitoring) — unblocks Grafana login
|
||
- [ ] 4. `gatus-slack-webhook` (monitoring) — unblocks Gatus alert notifications AND Alertmanager Slack SLO alerts
|
||
- [ ] 5. `GITEA_TOKEN` in each API repo's Actions Secrets — unblocks CI image push
|
||
- [ ] 6. Gitea packages enabled + DNS record for `registry.leeworks.dev` — unblocks image push to registry
|
||
- [ ] 7. Add api-company Flux source + kustomization to 0xWheatyz/Talos — unblocks all GitOps reconciliation
|
||
- [ ] 8. `gitea-registry` (zip-enrichment, holidays, air-quality, docs-site) — imagePullSecret for pods pulling from `registry.leeworks.dev`
|
||
- [ ] 9. `gitea-image-automation-token` (flux-system) — write-scoped token for Flux ImageUpdateAutomation to push image-tag commits
|
||
- [ ] 10. `rapidapi-proxy-secret` (zip-enrichment, holidays, air-quality) — RapidAPI Proxy Secret for server-side request validation
|
||
- [ ] 11. `GITEA_TOKEN` Actions secret in `leeworks-agents/vin-decoder` repo — enables CI image push for VIN Decoder (leeworks-agents/api-company#126)
|
||
- [ ] 12. `gitea-registry` imagePullSecret in `vin-decoder` namespace — enables pod image pulls from `registry.leeworks.dev` (leeworks-agents/api-company#127)
|
||
- [ ] 13. `rapidapi-proxy-secret` in `vin-decoder` namespace — enables RapidAPI proxy-secret header validation (leeworks-agents/api-company#128)
|
||
|
||
---
|
||
|
||
## Secret Details
|
||
|
||
### 1. `gitea-leeworks-agents-token`
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `gitea-leeworks-agents-token` |
|
||
| Namespace | `flux-system` |
|
||
| Purpose | Flux `GitRepository` authenticates to Gitea over HTTPS to pull `leeworks-agents/api-company` |
|
||
| Source | Gitea web UI → User Settings → Applications → Generate Token (scopes: `read:repository`) |
|
||
| Unblocks | Issue #2 (Flux GitRepository + Kustomization for api-company) |
|
||
|
||
```bash
|
||
kubectl create secret generic gitea-leeworks-agents-token \
|
||
-n flux-system \
|
||
--from-literal=username=leeworks-agents \
|
||
--from-literal=password=<GITEA_TOKEN>
|
||
```
|
||
|
||
---
|
||
|
||
### 2. `gitea-runner-token`
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `gitea-runner-token` |
|
||
| Namespace | `gitea-runner` |
|
||
| Purpose | The `gitea-act-runner` HelmRelease reads this token to register the runner with Gitea |
|
||
| Source | Gitea Admin Panel → Site Administration → Actions → Runners → **Create new Runner** — copy registration token |
|
||
| Unblocks | Issue #3 (gitea-act-runner Flux deployment) |
|
||
|
||
```bash
|
||
kubectl create secret generic gitea-runner-token \
|
||
-n gitea-runner \
|
||
--from-literal=token=<RUNNER_TOKEN>
|
||
```
|
||
|
||
After creating the secret, Flux reconciles the `gitea-act-runner` HelmRelease and the runner appears as **Online** in Gitea Admin → Actions → Runners.
|
||
|
||
---
|
||
|
||
### 3. `grafana-admin`
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `grafana-admin` |
|
||
| Namespace | `monitoring` |
|
||
| Purpose | Sets the Grafana `admin` user password on first boot |
|
||
| Source | Choose a strong password and store it in a password manager |
|
||
| Unblocks | Issue #7 (Prometheus + Grafana HelmRelease) |
|
||
|
||
```bash
|
||
kubectl create secret generic grafana-admin \
|
||
-n monitoring \
|
||
--from-literal=admin-password=<PASSWORD>
|
||
```
|
||
|
||
Grafana will be accessible at `https://grafana.leeworks.dev` (login: `admin` / `<PASSWORD>`).
|
||
|
||
---
|
||
|
||
### 4. `gatus-slack-webhook`
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `gatus-slack-webhook` |
|
||
| Namespace | `monitoring` |
|
||
| Purpose | Gatus posts downtime alerts to a Slack channel via incoming webhook. Also mounted by Alertmanager (`alertmanagerSpec.secrets`) to send SLO / firing-alert Slack notifications. |
|
||
| Source | Slack → Your workspace → Apps → Incoming Webhooks → Add to Slack → copy webhook URL |
|
||
| Unblocks | Issue #8 (Gatus status page at `status.leeworks.dev`); Alertmanager Slack receiver (issue #210) |
|
||
| Consumers | **Two components** share this secret: (1) **Gatus** (`flux/monitoring/gatus-helmrelease.yaml`) — public status-page alerts; (2) **Alertmanager** (`flux/monitoring/helmrelease.yaml` via `alertmanagerSpec.secrets`) — SLO / firing-alert Slack notifications |
|
||
|
||
> **Note (dual consumer):** Creating this one secret (`gatus-slack-webhook` in the `monitoring` namespace) unblocks **both** Gatus alert delivery AND Alertmanager's Slack receiver. The Alertmanager configuration references this secret at `global.slack_api_url_file: /etc/alertmanager/secrets/gatus-slack-webhook/url` (enabled in PR #212 / issue #210).
|
||
|
||
```bash
|
||
kubectl create secret generic gatus-slack-webhook \
|
||
-n monitoring \
|
||
--from-literal=url=https://hooks.slack.com/services/YOUR/WEBHOOK/URL
|
||
```
|
||
|
||
---
|
||
|
||
### 5. `GITEA_TOKEN` — Gitea Actions Secret (per repo)
|
||
|
||
| Field | Value |
|
||
|----------|-------|
|
||
| Name | `GITEA_TOKEN` |
|
||
| Scope | Gitea Actions Secret — set in each repo's Settings, **not** a Kubernetes secret |
|
||
| Purpose | CI workflows use this token to push container images to `registry.leeworks.dev` |
|
||
| Source | Same token as step 1, or a dedicated CI token with `write:packages` scope |
|
||
| Unblocks | CI pipelines for all three API repos |
|
||
|
||
Set in Gitea web UI for **each** of these repos:
|
||
- `leeworks-agents/api-company`
|
||
- `leeworks-agents/zip-enrichment`
|
||
- `leeworks-agents/holidays`
|
||
- `leeworks-agents/air-quality`
|
||
|
||
Path: **Repo → Settings → Actions → Secrets → Add Secret**
|
||
- Name: `GITEA_TOKEN`
|
||
- Value: `<GITEA_TOKEN>`
|
||
|
||
---
|
||
|
||
### 6. Enable Gitea Packages + DNS for `registry.leeworks.dev`
|
||
|
||
This is a Gitea instance configuration step, not a Kubernetes secret.
|
||
|
||
| Step | Action |
|
||
|------|--------|
|
||
| 6a | Enable packages in Gitea `app.ini`: set `[packages] ENABLED = true` then restart Gitea |
|
||
| 6b | Add DNS A record: `registry.leeworks.dev` → cluster ingress IP |
|
||
|
||
Find cluster ingress IP:
|
||
```bash
|
||
kubectl get svc -n ingress-nginx
|
||
```
|
||
|
||
See `docs/registry.md` for context on why the Gitea built-in registry was chosen.
|
||
|
||
Unblocks: Issue #4 (container registry), and transitively all CI image-push workflows.
|
||
|
||
---
|
||
|
||
### 7. Add api-company Flux Source + Kustomization to 0xWheatyz/Talos
|
||
|
||
Reference manifests are already committed at `flux/api-company-source/` in this repo.
|
||
The operator must copy them into the Talos cluster repo so FluxCD picks them up:
|
||
|
||
```
|
||
0xWheatyz/Talos:testing1/first-cluster/cluster/flux/api-company-source/
|
||
```
|
||
|
||
Unblocks: Issue #2 (Flux reconciliation of all `flux/` manifests in this repo).
|
||
|
||
---
|
||
|
||
---
|
||
|
||
### 10. `rapidapi-proxy-secret` — RapidAPI Proxy Secret (per API namespace)
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `rapidapi-proxy-secret` |
|
||
| Namespaces | `zip-enrichment`, `holidays`, `air-quality` |
|
||
| Purpose | Every API service validates the `X-RapidAPI-Proxy-Secret` header on every route. Requests without a valid secret return HTTP 403. |
|
||
| Source | RapidAPI dashboard → API Settings → Security → **Proxy Secret** (generated after each API listing is created) |
|
||
| Unblocks | Phase 3 server middleware; API services will start but reject all traffic without this secret |
|
||
|
||
```bash
|
||
for NS in zip-enrichment holidays air-quality; do
|
||
kubectl create secret generic rapidapi-proxy-secret \
|
||
--namespace=$NS \
|
||
--from-literal=X-RapidAPI-Proxy-Secret=<value-from-rapidapi-dashboard>
|
||
done
|
||
```
|
||
|
||
**Source:** RapidAPI dashboard → select your API → Settings → Security → Proxy Secret
|
||
|
||
> **Note:** Placeholder `ExternalSecret` manifests are committed at
|
||
> `flux/zip-enrichment/externalsecret.yaml`, `flux/holidays/externalsecret.yaml`,
|
||
> and `flux/air-quality/externalsecret.yaml`. These will auto-sync this secret
|
||
> from the configured backend once the External Secrets Operator (ESO) is
|
||
> deployed (see issue #61). Until then, create manually using the commands above.
|
||
|
||
Verify:
|
||
```bash
|
||
for NS in zip-enrichment holidays air-quality; do
|
||
echo -n "$NS: "
|
||
kubectl get secret rapidapi-proxy-secret -n $NS -o jsonpath='{.data.X-RapidAPI-Proxy-Secret}' | base64 -d | wc -c
|
||
echo " chars"
|
||
done
|
||
```
|
||
|
||
|
||
### 11. `GITEA_TOKEN` Actions secret in `leeworks-agents/vin-decoder`
|
||
|
||
| Field | Value |
|
||
|----------|-------|
|
||
| Name | `GITEA_TOKEN` |
|
||
| Scope | Gitea Actions Secret — set in repo Settings |
|
||
| Purpose | CI workflow pushes container image to `registry.leeworks.dev/vin-decoder/api:<sha>` |
|
||
| Source | Gitea token with `write:packages` scope (reuse from item #5 if it has `write:packages`) |
|
||
| Tracked | leeworks-agents/api-company#126 |
|
||
| Unblocks | CI image push for VIN Decoder |
|
||
|
||
Path: **Gitea → leeworks-agents/vin-decoder → Settings → Secrets → `GITEA_TOKEN`**
|
||
|
||
---
|
||
|
||
### 12. `gitea-registry` imagePullSecret in `vin-decoder` namespace
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `gitea-registry` |
|
||
| Namespace | `vin-decoder` |
|
||
| Type | `kubernetes.io/dockerconfigjson` |
|
||
| Purpose | Allows VIN Decoder pods to pull images from `registry.leeworks.dev` without ImagePullBackOff |
|
||
| Tracked | leeworks-agents/api-company#127 |
|
||
|
||
```bash
|
||
kubectl create namespace vin-decoder --dry-run=client -o yaml | kubectl apply -f -
|
||
kubectl create secret docker-registry gitea-registry \
|
||
--namespace=vin-decoder \
|
||
--docker-server=registry.leeworks.dev \
|
||
--docker-username=leeworks-agents \
|
||
--docker-password=<GITEA_TOKEN_WITH_READ_PACKAGES> \
|
||
--docker-email=agent@leeworks.dev
|
||
```
|
||
|
||
---
|
||
|
||
### 13. `rapidapi-proxy-secret` in `vin-decoder` namespace
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `rapidapi-proxy-secret` |
|
||
| Namespace | `vin-decoder` |
|
||
| Purpose | VIN Decoder validates `X-RapidAPI-Proxy-Secret` header; returns HTTP 403 if missing/wrong |
|
||
| Source | RapidAPI dashboard → VIN Decoder listing → Settings → Security → Proxy Secret |
|
||
| Tracked | leeworks-agents/api-company#128 |
|
||
|
||
```bash
|
||
# Placeholder (unblocks deploy testing):
|
||
kubectl create secret generic rapidapi-proxy-secret \
|
||
--namespace=vin-decoder \
|
||
--from-literal=X-RapidAPI-Proxy-Secret=PLACEHOLDER_REPLACE_AFTER_RAPIDAPI_LISTING
|
||
|
||
# Update with real value after RapidAPI listing (#131) is live:
|
||
kubectl create secret generic rapidapi-proxy-secret \
|
||
-n vin-decoder \
|
||
--from-literal=X-RapidAPI-Proxy-Secret=<VIN_RAPIDAPI_PROXY_SECRET> \
|
||
--save-config --dry-run=client -o yaml | kubectl apply -f -
|
||
```
|
||
|
||
---
|
||
|
||
## Dependency Order
|
||
|
||
```
|
||
7 (Flux wiring) → all flux/ resources reconcile
|
||
1 (gitea-leeworks-token) → Flux can pull this repo over HTTPS
|
||
2 (gitea-runner-token) → runner online → CI runs
|
||
3 (grafana-admin) → Grafana login works
|
||
4 (gatus-slack-webhook) → Gatus alerting works + Alertmanager Slack SLO alerts
|
||
5 + 6 (GITEA_TOKEN + registry packages) → CI pushes images → API services deploy
|
||
8 (gitea-registry) → pods can pull images from registry.leeworks.dev → services start
|
||
9 (gitea-image-automation-token) → Flux ImageUpdateAutomation pushes tag-update commits
|
||
10 (rapidapi-proxy-secret × 3) → API server middleware validates RapidAPI requests → revenue enabled
|
||
```
|
||
|
||
Once all nine items are complete, the full stack (runner, registry, Prometheus, Grafana, Gatus, docs-site, three API services) reconciles automatically via FluxCD with no further manual steps.
|
||
|
||
---
|
||
|
||
### 8. `gitea-registry` — imagePullSecret for API service namespaces
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `gitea-registry` |
|
||
| Namespaces | `zip-enrichment`, `holidays`, `air-quality`, `docs-site` |
|
||
| Type | `kubernetes.io/dockerconfigjson` |
|
||
| Purpose | Allows pods to pull images from `registry.leeworks.dev` without ImagePullBackOff |
|
||
| Source | Gitea token with `read:packages` scope (can reuse the same token as step 1 if it has that scope) |
|
||
| Unblocks | Issues #58 (Phase 0 hard deploy blocker) and transitively Phase 3 service deploys |
|
||
|
||
```bash
|
||
for NS in zip-enrichment holidays air-quality docs-site; do
|
||
kubectl create secret docker-registry gitea-registry \
|
||
--namespace=$NS \
|
||
--docker-server=registry.leeworks.dev \
|
||
--docker-username=leeworks-agents \
|
||
--docker-password=<GITEA_TOKEN_WITH_READ_PACKAGES> \
|
||
--docker-email=agent@leeworks.dev
|
||
done
|
||
```
|
||
|
||
Verify:
|
||
```bash
|
||
kubectl get secret gitea-registry -n zip-enrichment -o jsonpath='{.type}'
|
||
# expected: kubernetes.io/dockerconfigjson
|
||
```
|
||
|
||
---
|
||
|
||
### 9. `gitea-image-automation-token` — Flux ImageUpdateAutomation write token
|
||
|
||
| Field | Value |
|
||
|-----------|-------|
|
||
| Name | `gitea-image-automation-token` |
|
||
| Namespace | `flux-system` |
|
||
| Purpose | Allows Flux `ImageUpdateAutomation` to push image-tag update commits back to `leeworks-agents/api-company` |
|
||
| Source | Gitea token with **`write:repository`** scope (the existing `gitea-leeworks-agents-token` only has `read:repository` — create a separate token or verify scope) |
|
||
| Unblocks | Issue #57 (Flux ImageUpdateAutomation for api-company) |
|
||
|
||
```bash
|
||
kubectl create secret generic gitea-image-automation-token \
|
||
-n flux-system \
|
||
--from-literal=username=leeworks-agents \
|
||
--from-literal=password=<TOKEN_WITH_WRITE_REPO_SCOPE>
|
||
```
|
||
|
||
Verify after creation:
|
||
```bash
|
||
flux get imageupdateautomations -n flux-system
|
||
# Expected: api-company shows READY=True
|
||
```
|
||
|
||
> **Note:** If you create a new token with `write:repository` scope, keep the existing `gitea-leeworks-agents-token` for read-only Flux GitRepository pulls and use this new secret exclusively for `ImageUpdateAutomation`.
|