ADR-0087 records the maintainer's route (void.aeternalisgames.com:7400 via a dedicated internal MetalLB VIP to the evolution-dev region pod, TLS end to end, CIDR-limited) and operator-opened windows enforced by evo-server. The route is a kustomize component included nowhere; the policy check encodes its shape and refuses enabling until VIP/CIDR, the dev quota and --session-window exist, with a 20-case self-test in the CI deploy stage. The base listener stays loopback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7.6 KiB
ADR-0069: Environment isolation and World0 protection
Status
ADR-0087 allows exactly one LoadBalancer Service, in evolution-dev only, for the dev session route once enabled; node ports stay forbidden and stage/World0 are unchanged. Accepted for M8.2, 2026-09-29. It builds on the M8.1 inventory (infrastructure/INVENTORY.md), ADR-0020 (delivery), and OPERATIONS.md (World0 protection). Nothing here deploys a world; World0 launch stays gated by M8.8.
Context
The inventory found five things that matter here:
- Argo CD's
defaultAppProject allows every cluster resource in every namespace. Three applications use it, two with automated prune and self-heal. - The only StorageClass reclaims with Delete.
- Collection ran as
system:admin. - There are no quotas, limit ranges or priority classes.
- Only the AEvolution application is manual-sync.
The roadmap's named risk is that one permissive application or credential could
destroy every world. Argo project boundaries cannot address that alone, because a
default-project application, a cascading parent application or a cluster-admin
is not constrained by an AEvolution AppProject.
Decision
Three environments, each in its own namespace: evolution-dev,
evolution-stage and evolution-world0. Each has its own AppProject, service
accounts, Secrets, database, database logins and object PVC. No identity, Secret or
database is shared between them.
Platform layer (deploy/kubernetes/platform/<env>), applied by a cluster
administrator and never by an Argo CD Application, so no sync or prune can remove
it. It contains:
- The Namespace, labelled with its environment and with Pod Security
restrictedenforced. Only World0 carriesaevolution.io/protected=true. - A ResourceQuota:
- dev and stage: 2 CPU and 2 GiB requests, 4 GiB of memory limits, 4 PVCs and 40 GiB;
- World0: 4 CPU and 4 GiB requests, 8 GiB of limits, 8 PVCs and 200 GiB;
- no LoadBalancer or NodePort services anywhere.
- A LimitRange (defaults 50m/64Mi with a 256Mi limit, at most 2Gi per container) sized from the measured host profile.
- A default-deny NetworkPolicy for every pod, plus DNS egress only. Workloads add narrow rules in M8.4.
- Service accounts
aevolution-migrator,aevolution-regionandaevolution-backup, with no token and no RBAC; nothing needs the Kubernetes API. They are platform-owned, so workload projects cannot create them.
Secrets never enter Git. Operators create them per environment, by reference name (ENVIRONMENTS.md). CI holds no cluster or data credentials (ADR-0020).
World0 admission protection. A native ValidatingAdmissionPolicy
(deploy/kubernetes/platform/admission) is GA on the cluster's Kubernetes 1.32. It
needs no webhook or dependency, and it fails closed. It applies to everyone,
including system:masters and Argo's controller. Unless the requester is in the
group aevolution:world0-break-glass, it denies:
- deleting the World0 namespace, or removing its protected label;
- deleting its PVCs, Secrets or service accounts;
- deleting a PV bound to it, or moving such a PV away from
Retain; - deleting the
aevolution-world0Application or AppProject.
A cascading parent app, a prune, a default-project application or a stray
kubectl delete therefore fails at admission. Kubernetes does not apply policies
to policy objects, so removing the policy is itself the break-glass action,
reserved to cluster-admins.
Delivery.
- The stage and World0 AppProjects (examples) each target one namespace. They allow no cluster kinds, and whitelist only namespaced workload kinds.
- World0 carries a permanent deny sync window with
manualSync: true, so no automated sync can ever run. - Applications never auto-sync and never carry the cascade finalizer. World0's must not appear in a pruning parent.
- Durable PVCs will carry
Prune=false,Delete=false.
Cleanup. tools/ops/cleanup-environment.sh accepts only dev or stage. It
refuses anything else before invoking kubectl, re-checks the namespace labels,
never touches platform objects or Secrets, and dry-runs unless --execute is given.
Verification.
tools/ci/deploy-policy-check.pyruns offline in CI (Ubuntupython3-yaml) and asserts every invariant above over all manifests.tools/ops/verify-isolation.shruns live. It makes onlycan-iqueries, reads and server dry-runs, and covers negative RBAC, cross-environment denial, World0 admission denials, project scopes and the cleanup refusal.
Consequences
- Residual risk: creation. A
default-project application can still create objects in AEvolution namespaces; admission protects only World0's data and identity from deletion. The maintainer accepted this risk (2026-09-29) and is leavingdefaultunrestricted (docs/BACKLOG.md). - Residual risk: advisory locks. The runtime login's advisory-lock capability (ADR-0068) remains.
- Local-path storage. Its PVs are Delete-reclaim. A World0 PV must be patched to Retain when bound, and the policy then keeps it there. Volumes remain node-local until M8.3.
- Quotas and defaults. Quotas are starting values from measured profiles. They are raised by editing the platform overlay, never by the workload.
- Existing dev namespace. Applying the dev overlay to the existing namespace adds labels, the quota and the default-deny policy. Its proof Job already complies.
Alternatives rejected
- An admission webhook or policy engine (Kyverno, Gatekeeper). Either is a new dependency with its own availability risk; the native policy suffices.
- Relying on AppProjects alone. They do not constrain other projects, parents or cluster-admins.
- Namespaces created by Argo
CreateNamespace. A prune or cascade could then remove them.
Amendment: platform delivered by Argo CD (2026-09-29)
The maintainer prefers that every deployment and configuration goes through Argo
CD. The platform layer is therefore an Argo Application, aevolution-platform
(deploy/argocd/platform), no longer a hand-applied layer. The original concern,
that a sync, prune or cascade could delete namespaces and durable objects, is met
differently:
- Never prune. The Application self-heals but never prunes.
- Protected objects. Every object in the bundle carries
argocd.argoproj.io/sync-options: Prune=false,Delete=false, so neither a Git removal nor deleting the Application deletes it. - No cascade. There is no cascade finalizer.
- Protected by admission. The World0 admission policy also denies deleting the
aevolution-platformApplication and AppProject. - Scoped project. Its project may target only the three AEvolution namespaces, and only four cluster kinds: Namespace, StorageClass, ValidatingAdmissionPolicy and its binding.
Self-heal adds a protection the hand-applied layer lacked: drift, such as a deleted quota or label, is restored. The admission policy itself is still not subject to admission, so a policy change is a reviewed Git change.
What stays outside Argo:
- Secrets, until a secret manager is chosen (docs/BACKLOG.md);
- the store node label;
- the one-time bootstrap of the platform project and Application.
Amendment: World0 volume claim references (M8 review, 2026-09-30)
The policy recognizes a World0 PersistentVolume by its claim reference. Clearing
or reassigning that reference first would have removed the protection. An UPDATE
that removes the reference, or changes its namespace or claim name, is now denied
without the break-glass group. The binder may still fill in uid and
resourceVersion. verify-isolation.sh tests this with a server-side dry run when
a World0 volume exists.