At the maintainer's request everything but Secrets and the store node label now goes through Argo CD. aevolution-platform syncs one bundle (World0 admission policy, Retain storage class, each environment's namespace, guardrails, service accounts and object claim) with self-heal, no prune and Prune=false,Delete=false on every object; the admission policy also protects the platform app and project. Store Jobs become kustomize components used as PreSync (migrate) and PostSync (check-store) hooks per environment, and the policy check refuses any unresolved marker in a path an Application syncs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7.0 KiB
AEvolution environments (M8.2)
The decision is recorded in ADR-0069. This runbook applies the platform layer, creates the per-environment references, and verifies isolation. Every step is manual, and none is performed by Argo CD.
Environments
| dev | stage | world0 | |
|---|---|---|---|
| Namespace | evolution-dev (exists) |
evolution-stage |
evolution-world0 |
| Purpose | Development | Clone, migration and soak rehearsal | Persistent world (launch gated by M8.8) |
| Protected by admission | No | No | Yes |
| Cleanup tool | Allowed | Allowed | Refused |
| AppProject / Application | aevolution-dev (live, manual) |
deploy/argocd/stage/*.example |
deploy/argocd/world0/*.example (deny window) |
| Database | Its own database and logins on the AEvolution PostgreSQL VM | Its own | Its own |
| World IDs | Development range | Clone targets (ADR-0067) | World0's own; never reused elsewhere |
Secret references (created by an operator, never committed)
Each environment has its own database: dev uses aevolution on the PostgreSQL
VM pg-forge-01. Stage needs a separate database, with its own migrator login and
runtime group. Sharing one database would let one environment's runtime login act
on the other's regions, because the fenced functions are per database, not per
environment.
Each namespace gets its own Secrets, with environment-specific values:
harbor-pull: a pull-only Harbor robot (docker-registry type).evolution-postgres-migrator: the migrator/owner login for that environment's database.evolution-postgres-runtime: the runtime login, a member of that environment's NOLOGIN runtime group (ADR-0060).
Both login Secrets have the keys pg_service.conf, pgpass and ca.crt:
pg_service.confhas an[evolution]section withhost, a numerichostaddr,port,dbname,user,sslmode=verify-fullandsslrootcert=/run/postgres/ca.crt.connect_timeoutis allowed.evo-serverandevo-world-cloneread it with--pg-serviceand refuse any other key.pgpassis mounted, then copied to private memory before use, because libpq ignores group-readable password files.
aevolution-db (the PG*/POSTGRES_* connection variables) is not used by
AEvolution workloads. PGPASSWORD and PGUSER are never placed in a container
environment, and the policy check forbids it.
A World0 credential must never exist in the dev or stage namespaces, and no dev or stage credential may reach the World0 database. Rotate by replacing the Secret and restarting the consumer.
Delivery through Argo CD (maintainer preference, 2026-09-29)
Everything except Secrets and one node label is delivered from Git by Argo CD.
Bootstrap once. Argo CD cannot create its own first project and application:
kubectl apply -f deploy/argocd/platform/project.yaml -f deploy/argocd/platform/application.yaml
aevolution-platform then syncs deploy/kubernetes/platform/cluster:
- the World0 admission policy;
- the
aevolution-local-retainstorage class; - each environment's namespace, quota, limit range, default-deny policies, service accounts and object claim (dev and stage).
It self-heals, so a deleted quota, policy or label comes back. It never prunes,
and every object carries Prune=false,Delete=false, so removing a file or
deleting the Application never deletes a namespace, claim, class or policy. The
admission policy also denies deleting the platform Application or project. Objects
applied by hand earlier are adopted (server-side apply).
Workloads. Each environment's Application syncs its path, currently manually:
aevolution-devsyncsdeploy/kubernetes/dev, which is live.- Stage uses
deploy/argocd/stage, whose path is resolved when its values are supplied.
The store Jobs (deploy/kubernetes/stores/<env>) are included in the environment
path as sync hooks:
- the database migration runs as a PreSync hook, so a failed migration stops the sync;
check-storeruns as a PostSync hook.
An environment's stores overlay is included only once it has its published image
digests and its runtime group role. The policy check refuses any REPLACE_ marker
in a synced path.
Outside Argo (by design, for now).
- Secrets are created by an operator; a secret manager is in docs/BACKLOG.md.
- One node label:
kubectl label node rke2-worker-1 aevolution.io/stores=true. The maintainer allows any worker; worker-1 is chosen because it had the lowest measured latency, and worker-3's disk is 87 % full.
Verify: bash tools/ops/verify-isolation.sh makes only reads, can-i queries and
server dry runs.
Stores (M8.3, ADR-0070)
Once deploy/kubernetes/stores/dev has its values and is included in
deploy/kubernetes/dev, a manual sync of aevolution-dev runs the migration and
then the store check. The hook Jobs keep their logs until the next sync. Record
this evidence in docs/engineering/VALIDATION.md:
- Pod restart: sync again. The check must print
probe verifiedwith the same digest. - Privilege boundary: every run prints
database tables=denied functions=reachable. - Missing volume: temporarily remove the
objectsvolume in a branch, or run the image by hand without it. The check must exit 4 withobjects refused missing-or-not-a-directory. - Full volume: add
--reserve-mib 1048576and a new probe region, such as--region 2. The check must exit 4 withprobe refused full. - Retention: delete the claim by hand, since Argo never prunes it. The PV must
become
Released, not deleted. To rebind:- clear the PV's
spec.claimRef; - let the platform app recreate the claim, with
volumeNamepinned in Git; - sync, and check that it prints
probe verified.
- clear the PV's
- Credential rotation:
- Change the runtime login's password on the database VM and sync before updating
the Secret. The check must exit 3 with
database refused connect. - Update the Secret's
pgpassand sync again. It must exit 0.
- Change the runtime login's password on the database VM and sync before updating
the Secret. The check must exit 3 with
Break-glass
How membership is granted is not decided yet; see docs/BACKLOG.md. Until it is, nobody holds the group, so World0 deletions are impossible short of removing the policy.
To delete World0 data or identity on purpose:
- Record the operation and its approval.
- Use a credential whose groups include
aevolution:world0-break-glass, for example a short-lived client certificate with that organization. - Take and verify a backup first (M8.7).
Removing the admission policy itself is the break-glass of last resort. It needs cluster-admin and must be recorded the same way. Never test by deleting World0: the verify script uses server-side dry runs.
Residual risks
defaultAppProject. It still allows every destination, so its applications (other projects, two with automated prune) can create objects in AEvolution namespaces. Admission denies only World0 deletions. The maintainer accepted this risk on 2026-09-29 and is leavingdefaultas it is (docs/BACKLOG.md).- World0 volumes.
local-pathprovisions volumes with Delete. Patch each World0 PV toRetainwhen it is bound, and the policy then keeps it there. Volumes stay node-local until M8.3.