Add M8.2 environment isolation and World0 protection

Per-environment platform layers (dev, stage, world0) with restricted pod
security, quotas, limit ranges, default-deny networking and token-less
service accounts, applied by an administrator rather than Argo. A native
ValidatingAdmissionPolicy denies deleting World0 data, identity and Argo
objects to every requester without the break-glass group. Stage and
World0 Argo projects are scoped to one namespace, World0 blocks
automated sync, and a cleanup tool refuses World0. An offline policy
check runs as a new CI suite; a live verification script uses only
can-i, reads and server dry runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-09-29 12:43:58 -04:00
co-authored by Claude Opus 5.5
parent 18708340c3
commit df0c4f4efb
28 changed files with 953 additions and 2 deletions
+11
View File
@@ -152,3 +152,14 @@ plus the pinned libpq, UID 10001, no credentials). It smoke-tests the image with
- the runtime Secret, object PVC and a NetworkPolicy.
Pin the digest and provision the region first. Without a gateway, the region
receives no inputs.
## Environments and World0 protection (M8.2)
`kubernetes/platform/<env>` holds each environment's namespace, quota, limit range,
default-deny policies and service accounts. `kubernetes/platform/admission` holds
the World0 admission policy. A cluster administrator applies these by hand; they
are never part of an Argo Application. The stage and World0 Argo projects and
applications are in `argocd/stage` and `argocd/world0` as examples.
See docs/infrastructure/ENVIRONMENTS.md for the runbook, and ADR-0069. The CI
`deploy` suite runs `tools/ci/deploy-policy-check.py` over every manifest.
@@ -0,0 +1,21 @@
# M8.2 (stage). REPLACE_WORKLOAD_PATH is deliberately unresolved: the stage
# workload is M8.4 work. Manual sync only; no automated policy, no prune on sync of
# durable objects (PVCs carry argocd.argoproj.io/sync-options: Prune=false,Delete=false).
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: aevolution-stage
namespace: argocd
spec:
project: aevolution-stage
source:
repoURL: https://git.aeternalisgames.com/dingo/AEvolution.git
targetRevision: main
path: REPLACE_WORKLOAD_PATH
destination:
server: https://kubernetes.default.svc
namespace: evolution-stage
syncPolicy:
syncOptions:
- FailOnSharedResource=true
- PruneLast=true
+26
View File
@@ -0,0 +1,26 @@
# M8.2 (stage). Apply by hand to the argocd namespace; never from an Application.
# One destination namespace only, no cluster-scoped kinds, no Secrets or quotas.
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: aevolution-stage
namespace: argocd
spec:
description: AEvolution stage (evolution-stage only)
sourceRepos:
- https://git.aeternalisgames.com/dingo/AEvolution.git
destinations:
- server: https://kubernetes.default.svc
namespace: evolution-stage
clusterResourceWhitelist: []
namespaceResourceWhitelist:
- {group: "", kind: Service}
- {group: "", kind: ConfigMap}
- {group: "", kind: PersistentVolumeClaim}
- {group: apps, kind: Deployment}
- {group: batch, kind: Job}
- {group: networking.k8s.io, kind: NetworkPolicy}
# Not listed, so never Argo-managed: Namespaces, ServiceAccounts (platform layer),
# Secrets, quotas, limit ranges and RBAC.
orphanedResources:
warn: true
@@ -0,0 +1,25 @@
# M8.2 (world0). REPLACE_WORKLOAD_PATH is deliberately unresolved: the world0
# workload is M8.4 work. Manual sync only; no automated policy, no prune on sync of
# durable objects (PVCs carry argocd.argoproj.io/sync-options: Prune=false,Delete=false).
# World0: never add the resources-finalizer.argocd.argoproj.io finalizer (deleting
# the Application must orphan, not cascade), never add an automated policy, and never
# list this Application in a parent app-of-apps with prune. The admission policy
# also denies deleting this Application without break-glass.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: aevolution-world0
namespace: argocd
spec:
project: aevolution-world0
source:
repoURL: https://git.aeternalisgames.com/dingo/AEvolution.git
targetRevision: main
path: REPLACE_WORKLOAD_PATH
destination:
server: https://kubernetes.default.svc
namespace: evolution-world0
syncPolicy:
syncOptions:
- FailOnSharedResource=true
- PruneLast=true
+33
View File
@@ -0,0 +1,33 @@
# M8.2 (world0). Apply by hand to the argocd namespace; never from an Application.
# One destination namespace only, no cluster-scoped kinds, no Secrets or quotas.
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: aevolution-world0
namespace: argocd
spec:
description: AEvolution world0 (evolution-world0 only)
sourceRepos:
- https://git.aeternalisgames.com/dingo/AEvolution.git
destinations:
- server: https://kubernetes.default.svc
namespace: evolution-world0
clusterResourceWhitelist: []
namespaceResourceWhitelist:
- {group: "", kind: Service}
- {group: "", kind: ConfigMap}
- {group: "", kind: PersistentVolumeClaim}
- {group: apps, kind: Deployment}
- {group: batch, kind: Job}
- {group: networking.k8s.io, kind: NetworkPolicy}
# Not listed, so never Argo-managed: Namespaces, ServiceAccounts (platform layer),
# Secrets, quotas, limit ranges and RBAC.
orphanedResources:
warn: true
# No automated sync can ever run; a human may still sync by hand.
syncWindows:
- kind: deny
schedule: "* * * * *"
duration: 24h
applications: ["*"]
manualSync: true
+13
View File
@@ -45,6 +45,19 @@ RUN cmake --preset analysis -B build/ecs-analysis -DCMAKE_CXX_COMPILER=clang++-1
&& cmake --build build/ecs-analysis --target evo-ecs-experiment --parallel 2 \
&& ctest --test-dir build/ecs-analysis -R ecs-experiment --output-on-failure --no-tests=error
# M8.2 offline deployment-policy check (ADR-0069): Ubuntu's python3-yaml only, no
# cluster access. Fails the build when an isolation or World0 invariant breaks.
FROM ${UBUNTU_IMAGE} AS ci-deploy
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-yaml \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /source
COPY deploy/ deploy/
COPY tools/ci/deploy-policy-check.py tools/ci/
COPY tools/ops/ tools/ops/
RUN python3 tools/ci/deploy-policy-check.py
FROM ci-toolchain AS postgres-build
RUN apt-get update && apt-get install -y --no-install-recommends \
curl bzip2 make flex bison perl libssl-dev zlib1g-dev pkg-config \
@@ -0,0 +1,88 @@
# M8.2 World0 protection (ADR-0069). A native ValidatingAdmissionPolicy (GA since
# Kubernetes 1.30; the cluster runs 1.32): no webhook, no new dependency. It applies
# to every requester, including cluster-admin (system:masters), the Argo CD
# application controller and any Application in any AppProject, so no permissive
# app, prune, parent-app cascade or credential can remove World0 data or identity.
# Only a member of the group `aevolution:world0-break-glass` passes. Membership is
# granted out of band (client certificate O= or OIDC group) for a recorded operation.
#
# Denied without break-glass:
# - deleting Namespace evolution-world0, or removing its aevolution.io/protected label;
# - deleting PersistentVolumeClaims, Secrets or ServiceAccounts in evolution-world0;
# - deleting a PersistentVolume bound to evolution-world0, or changing its reclaim
# policy away from Retain once set (set Retain when the volume is bound: the
# only StorageClass, local-path, provisions with Delete);
# - deleting the aevolution-world0 Argo CD Application or AppProject.
# Kubernetes does not run admission policies on policy objects themselves, so
# removing this policy is itself the audited break-glass step (cluster-admin).
# Apply by hand: kubectl apply -f deploy/kubernetes/platform/admission/
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: aevolution-world0-protection
labels:
app.kubernetes.io/part-of: aevolution
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["DELETE", "UPDATE"]
resources: ["namespaces", "persistentvolumes"]
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["DELETE"]
resources: ["persistentvolumeclaims", "secrets", "serviceaccounts"]
- apiGroups: ["argoproj.io"]
apiVersions: ["*"]
operations: ["DELETE"]
resources: ["applications", "appprojects"]
variables:
- name: breakGlass
expression: "request.userInfo.groups.exists(g, g == 'aevolution:world0-break-glass')"
- name: resource
expression: "request.resource.resource"
- name: world0Namespace
expression: "variables.resource == 'namespaces' && request.name == 'evolution-world0'"
- name: world0Volume
expression: >-
variables.resource == 'persistentvolumes' &&
has(oldObject.spec.claimRef) && has(oldObject.spec.claimRef.namespace) &&
oldObject.spec.claimRef.namespace == 'evolution-world0'
- name: world0Object
expression: >-
request.namespace == 'evolution-world0' &&
variables.resource in ['persistentvolumeclaims', 'secrets', 'serviceaccounts']
- name: world0Delivery
expression: >-
request.resource.group == 'argoproj.io' && request.name == 'aevolution-world0'
validations:
- expression: >-
variables.breakGlass || request.operation != 'DELETE' ||
!(variables.world0Namespace || variables.world0Volume || variables.world0Object ||
variables.world0Delivery)
message: "World0 data, identity and delivery objects cannot be deleted without the aevolution:world0-break-glass group"
reason: Forbidden
- expression: >-
variables.breakGlass || request.operation != 'UPDATE' || !variables.world0Namespace ||
(has(object.metadata.labels) && 'aevolution.io/protected' in object.metadata.labels &&
object.metadata.labels['aevolution.io/protected'] == 'true')
message: "The World0 namespace must keep aevolution.io/protected=true"
reason: Forbidden
- expression: >-
variables.breakGlass || request.operation != 'UPDATE' || !variables.world0Volume ||
oldObject.spec.persistentVolumeReclaimPolicy != 'Retain' ||
object.spec.persistentVolumeReclaimPolicy == 'Retain'
message: "A World0 PersistentVolume set to Retain cannot be changed back"
reason: Forbidden
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: aevolution-world0-protection
labels:
app.kubernetes.io/part-of: aevolution
spec:
policyName: aevolution-world0-protection
validationActions: [Deny]
@@ -0,0 +1,15 @@
# M8.2 per-environment guardrails (docs/infrastructure/ENVIRONMENTS.md). Applied by a
# cluster administrator with `kubectl apply -k deploy/kubernetes/platform/<env>`, never
# through an Argo CD Application: namespaces, quotas and identities must not be
# prunable by a workload sync. Each overlay sets the namespace and its quota values.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- quota.yaml
- limitrange.yaml
- networkpolicies.yaml
- serviceaccounts.yaml
labels:
- pairs:
app.kubernetes.io/part-of: aevolution
app.kubernetes.io/managed-by: aevolution-platform
@@ -0,0 +1,17 @@
# Defaults for containers that omit resources (the quota requires them), sized from
# the measured region-host profile (peak RSS about 9.7 MiB): 64Mi request, 256Mi
# limit, 50m CPU request and no default CPU limit. No container may exceed 2Gi.
apiVersion: v1
kind: LimitRange
metadata:
name: aevolution-defaults
spec:
limits:
- type: Container
defaultRequest:
cpu: 50m
memory: 64Mi
default:
memory: 256Mi
max:
memory: 2Gi
@@ -0,0 +1,30 @@
# Deny by default: every pod in the namespace has no ingress or egress except DNS.
# Workloads add narrow allowances with their own policies (for example the region
# host's egress to exactly the database VM address and port, and ingress to its
# operations port from the metrics scraper) in M8.4. Enforced by Canal/Calico.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: aevolution-default-deny
spec:
podSelector: {}
policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: aevolution-allow-dns
spec:
podSelector: {}
policyTypes: [Egress]
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- {protocol: UDP, port: 53}
- {protocol: TCP, port: 53}
@@ -0,0 +1,17 @@
# Values are replaced per environment. Memory is quota'd on requests and limits;
# CPU only on requests (region hosts run without CPU limits: throttling would delay
# lease renewals, docs/infrastructure/INVENTORY.md). No LoadBalancer or NodePort.
apiVersion: v1
kind: ResourceQuota
metadata:
name: aevolution-quota
spec:
hard:
requests.cpu: "2"
requests.memory: 2Gi
limits.memory: 4Gi
pods: "20"
persistentvolumeclaims: "4"
requests.storage: 40Gi
services.loadbalancers: "0"
services.nodeports: "0"
@@ -0,0 +1,26 @@
# One identity per role; none needs the Kubernetes API, so none mounts a token and
# no Role or RoleBinding is granted. Secrets are referenced by name only and are
# created per environment by an operator (ENVIRONMENTS.md); they never enter Git.
apiVersion: v1
kind: ServiceAccount
metadata:
name: aevolution-migrator # Runs migrations with the migrator database login.
automountServiceAccountToken: false
imagePullSecrets:
- name: harbor-pull
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: aevolution-region # evo-server with the runtime database login.
automountServiceAccountToken: false
imagePullSecrets:
- name: harbor-pull
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: aevolution-backup # Read-only backup/clone jobs (M8.7).
automountServiceAccountToken: false
imagePullSecrets:
- name: harbor-pull
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: evolution-dev
resources:
- namespace.yaml
- ../base
labels:
- pairs:
aevolution.io/environment: dev
@@ -0,0 +1,10 @@
# Disposable development. Existing namespace; applying adds labels and guardrails.
apiVersion: v1
kind: Namespace
metadata:
name: evolution-dev
labels:
aevolution.io/environment: dev
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: v1.32
pod-security.kubernetes.io/warn: restricted
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: evolution-stage
resources:
- namespace.yaml
- ../base
labels:
- pairs:
aevolution.io/environment: stage
@@ -0,0 +1,10 @@
# Clone, migration and soak rehearsals (M8.6/8.9). Disposable, never World0 data.
apiVersion: v1
kind: Namespace
metadata:
name: evolution-stage
labels:
aevolution.io/environment: stage
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: v1.32
pod-security.kubernetes.io/warn: restricted
@@ -0,0 +1,29 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: evolution-world0
resources:
- namespace.yaml
- ../base
patches:
- target:
kind: ResourceQuota
name: aevolution-quota
patch: |-
- op: replace
path: /spec/hard/requests.cpu
value: "4"
- op: replace
path: /spec/hard/requests.memory
value: 4Gi
- op: replace
path: /spec/hard/limits.memory
value: 8Gi
- op: replace
path: /spec/hard/persistentvolumeclaims
value: "8"
- op: replace
path: /spec/hard/requests.storage
value: 200Gi
labels:
- pairs:
aevolution.io/environment: world0
@@ -0,0 +1,11 @@
# Persistent World0. Guarded by deploy/kubernetes/platform/admission. Nothing here deploys a world; launch stays gated by M8.8.
apiVersion: v1
kind: Namespace
metadata:
name: evolution-world0
labels:
aevolution.io/environment: world0
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: v1.32
pod-security.kubernetes.io/warn: restricted
aevolution.io/protected: "true"