The maintainer's login Secrets carry pg_service.conf ([evolution]), pgpass and ca.crt. Add a strict read_pg_service (known keys only, verify-full unless a local socket, bounded size) behind --pg-service in evo-server and evo-world-clone, mount each login Secret whole in the store Jobs instead of reading aevolution-db, and fix the migrate Job's PGSERVICE to the [evolution] section. Record the store node choice, per-environment databases and the backup plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6.7 KiB
AEvolution environments (M8.2)
The decision is recorded in ADR-0069. This runbook applies the platform layer, creates the per-environment references, and verifies isolation. Every step is manual, and none is performed by Argo CD.
Environments
| dev | stage | world0 | |
|---|---|---|---|
| Namespace | evolution-dev (exists) |
evolution-stage |
evolution-world0 |
| Purpose | Development | Clone, migration and soak rehearsal | Persistent world (launch gated by M8.8) |
| Protected by admission | No | No | Yes |
| Cleanup tool | Allowed | Allowed | Refused |
| AppProject / Application | aevolution-dev (live, manual) |
deploy/argocd/stage/*.example |
deploy/argocd/world0/*.example (deny window) |
| Database | Its own database and logins on the AEvolution PostgreSQL VM | Its own | Its own |
| World IDs | Development range | Clone targets (ADR-0067) | World0's own; never reused elsewhere |
Secret references (created by an operator, never committed)
Each environment has its own database: dev uses aevolution on the PostgreSQL
VM pg-forge-01. Stage needs a separate database, with its own migrator login and
runtime group. Sharing one database would let one environment's runtime login act
on the other's regions, because the fenced functions are per database, not per
environment.
Each namespace gets its own Secrets, with environment-specific values:
harbor-pull: a pull-only Harbor robot (docker-registry type).evolution-postgres-migrator: the migrator/owner login for that environment's database.evolution-postgres-runtime: the runtime login, a member of that environment's NOLOGIN runtime group (ADR-0060).
Both login Secrets have the keys pg_service.conf, pgpass and ca.crt:
pg_service.confhas an[evolution]section withhost, a numerichostaddr,port,dbname,user,sslmode=verify-fullandsslrootcert=/run/postgres/ca.crt.connect_timeoutis allowed.evo-serverandevo-world-cloneread it with--pg-serviceand refuse any other key.pgpassis mounted, then copied to private memory before use, because libpq ignores group-readable password files.
aevolution-db (the PG*/POSTGRES_* connection variables) is not used by
AEvolution workloads. PGPASSWORD and PGUSER are never placed in a container
environment, and the policy check forbids it.
A World0 credential must never exist in the dev or stage namespaces, and no dev or stage credential may reach the World0 database. Rotate by replacing the Secret and restarting the consumer.
Apply (cluster administrator)
# 1. Platform guardrails, per environment (dev adds labels, quota and policies to
# the existing namespace; its completed proof Job already complies).
kubectl apply -k deploy/kubernetes/platform/dev
kubectl apply -k deploy/kubernetes/platform/stage
kubectl apply -k deploy/kubernetes/platform/world0 # an empty, protected namespace
# 2. World0 admission protection (validate first, then apply).
kubectl apply --dry-run=server -f deploy/kubernetes/platform/admission/
kubectl apply -f deploy/kubernetes/platform/admission/
# 3. Projects (resolve the Application paths only when M8.4 adds workloads).
kubectl apply -f deploy/argocd/stage/project.yaml.example
kubectl apply -f deploy/argocd/world0/project.yaml.example
# 4. Verify: reads, can-i and server dry runs only.
bash tools/ops/verify-isolation.sh
Create the Secrets listed above in each namespace before any workload syncs.
Stores (M8.3, ADR-0070)
Run this once per disposable environment (dev, then stage).
1. Label the store node, and apply the storage class and claim.
# Label exactly one worker. The maintainer allows any worker (never the control plane);
# worker-1 is chosen (lowest measured latency; worker-3's disk is 87 % full).
kubectl label node rke2-worker-1 aevolution.io/stores=true
kubectl apply -f deploy/kubernetes/platform/storage/storageclass.yaml
kubectl apply -k deploy/kubernetes/platform/dev # adds the aevolution-objects claim
2. Prepare the store Jobs. Copy deploy/kubernetes/stores/stores.yaml.example,
replace its REPLACE_* markers (namespace, database address and port, runtime
role, image digests), and create the Secrets above.
3. Run and qualify.
kubectl apply -f stores-dev.yaml # policy + both Jobs
kubectl logs -n evolution-dev job/aevolution-db-migrate # schema and audit
kubectl logs -n evolution-dev job/aevolution-store-check # expect exit 0 and "probe created"
Evidence to record in docs/engineering/VALIDATION.md:
- Pod restart: delete the check Job and apply it again. It must print
probe verified, with the same digest. - Missing volume: run the check with the
objectsvolume removed. It must exit 4 withobjects refused missing-or-not-a-directory. - Full volume: add
--reserve-mib 1048576and use a new probe region, such as--region 2. It must exit 4 withprobe refused full. - Retention:
kubectl delete pvc aevolution-objects -n evolution-dev. The PV must becomeReleased, not deleted. To rebind:- clear the PV's
spec.claimRef; - recreate the claim with
volumeNameset to that PV; - rerun the check, which must print
probe verified.
- clear the PV's
- Credential rotation:
- Change the runtime login's password on the database VM, and run the check
before updating the Secret. It must exit 3 with
database refused connect. - Update the Secret's
pgpassand rerun. It must exit 0.
- Change the runtime login's password on the database VM, and run the check
before updating the Secret. It must exit 3 with
- Privilege boundary: every run must print
database tables=denied functions=reachable.
Break-glass
How membership is granted is not decided yet; see docs/BACKLOG.md. Until it is, nobody holds the group, so World0 deletions are impossible short of removing the policy.
To delete World0 data or identity on purpose:
- Record the operation and its approval.
- Use a credential whose groups include
aevolution:world0-break-glass, for example a short-lived client certificate with that organization. - Take and verify a backup first (M8.7).
Removing the admission policy itself is the break-glass of last resort. It needs cluster-admin and must be recorded the same way. Never test by deleting World0: the verify script uses server-side dry runs.
Residual risks
defaultAppProject. It still allows every destination, so its applications (other projects, two with automated prune) can create objects in AEvolution namespaces. Admission denies only World0 deletions. The maintainer accepted this risk on 2026-09-29 and is leavingdefaultas it is (docs/BACKLOG.md).- World0 volumes.
local-pathprovisions volumes with Delete. Patch each World0 PV toRetainwhen it is bound, and the policy then keeps it there. Volumes stay node-local until M8.3.