Add M8.4 region host deployment, provisioning hook and drill

Deploy evo-region per environment (ADR-0071): one replica with the
Recreate strategy (a rolling update would deadlock on the lease),
/readyz readiness, /livez liveness, no CPU limit and a drain inside the
grace period, with the image digest doubling as the build identity. A
new idempotent evo-world-clone provision subcommand creates genesis from
development_opening (byte-identical to the qualification fixture) as a
Sync hook after the migration. Dev and stage paths include their region;
a dev AppProject allows Deployment and Service; region-drill.sh times
rollout and forced-deletion recovery; the policy check enforces the
host's invariants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-09-29 21:32:56 -04:00
co-authored by Claude Opus 5.5
parent 98240ef96e
commit 8ef69fab1d
27 changed files with 879 additions and 159 deletions
+8 -6
View File
@@ -147,17 +147,19 @@ plus the pinned libpq, UID 10001, no credentials). It smoke-tests the image with
`tools/ci/server-image-smoke.sh` and publishes only when Gitea variable
`HARBOR_SERVER_REPOSITORY` is set.
`kubernetes/region-server/deployment.yaml.example` is outside Argo. It shows:
- probes: `/readyz` means authority, `/livez` means only liveness;
- the runtime Secret, object PVC and a NetworkPolicy.
Pin the digest and provision the region first. Without a gateway, the region
receives no inputs.
The region host is deployed from `kubernetes/region/<env>` (M8.4, ADR-0071),
included in each environment's Argo path. It replaces the former
`region-server` example. Probes: `/readyz` means authority, and `/livez` means
only that the process is alive. Without a gateway, the region receives no inputs.
## Environments and World0 protection (M8.2)
`kubernetes/platform/<env>` holds each environment's namespace, quota, limit range,
default-deny policies and service accounts. `kubernetes/platform/admission` holds
the World0 admission policy. A cluster administrator applies these by hand; they
the World0 admission policy. Since the M8.3 amendment they are delivered by the
`aevolution-platform` Argo Application (`kubernetes/platform/cluster`), which
self-heals and never prunes. Originally a cluster administrator applied them by
hand; they
are never part of an Argo Application. The stage and World0 Argo projects and
applications are in `argocd/stage` and `argocd/world0` as examples.
+25
View File
@@ -0,0 +1,25 @@
# M8.4 dev workload project (replaces deploy/argocd/project.yaml.example). Apply by hand
# (kubectl apply -f deploy/argocd/dev/project.yaml): Argo CD does not manage its own
# projects here. One namespace, no cluster kinds; ServiceAccounts only for the
# foundation proof Job (the platform owns the environment's service accounts).
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: aevolution-dev
namespace: argocd
spec:
description: AEvolution dev (evolution-dev only)
sourceRepos:
- https://git.aeternalisgames.com/dingo/AEvolution.git
destinations:
- server: https://kubernetes.default.svc
namespace: evolution-dev
clusterResourceWhitelist: []
namespaceResourceWhitelist:
- {group: "", kind: ServiceAccount}
- {group: "", kind: Service}
- {group: apps, kind: Deployment}
- {group: batch, kind: Job}
- {group: networking.k8s.io, kind: NetworkPolicy}
orphanedResources:
warn: true
+1
View File
@@ -6,6 +6,7 @@ resources:
- networkpolicy.yaml
- job.yaml
- ../stores/dev
- ../region/dev
images:
- name: evo-sim
newName: registry.aeternalisgames.com/aeternalis/aevolution
@@ -1,143 +0,0 @@
# M7.8 development region server (ADR-0066). NOT in the active Argo kustomization.
# Before applying to evolution-dev: pin the server image digest, provision the region
# (owner-only), create the runtime Secret and object PVC, and add DB/DNS egress for
# the real endpoint. Never target World0 from this example.
#
# Authority, not liveness, gates traffic: /readyz is 200 only while this pod holds
# the region lease and has recovered and verified the committed head. /livez only
# says the loop is responding. A probe can wait up to one bounded I/O step (commit
# deadline 5 s), so probe timeouts below exceed it.
#
# There is no gateway yet: stdin is not attached, so the region hosts authority but
# receives no inputs (tick debt grows by design; nothing is invented).
apiVersion: apps/v1
kind: Deployment
metadata:
name: evolution-region-7-1
namespace: evolution-dev
labels: {app.kubernetes.io/name: evolution-region, evolution.aeternalis/region: "7-1"}
spec:
replicas: 1
strategy: {type: Recreate} # Never two pods racing; the lease still arbitrates.
selector:
matchLabels: {app.kubernetes.io/name: evolution-region, evolution.aeternalis/region: "7-1"}
template:
metadata:
labels: {app.kubernetes.io/name: evolution-region, evolution.aeternalis/region: "7-1"}
spec:
automountServiceAccountToken: false
terminationGracePeriodSeconds: 30 # SIGTERM drains within --drain-ms 20000.
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile: {type: RuntimeDefault}
initContainers:
# libpq rejects group-readable password files: copy to a private 0600 file.
- name: pgpass
image: registry.aeternalisgames.com/aeternalis/aevolution-server:replace-with-immutable-digest
command: ["/bin/sh", "-c", "install -m 0600 /run/postgres/pgpass /run/evo/pgpass"]
securityContext: {readOnlyRootFilesystem: true, allowPrivilegeEscalation: false, capabilities: {drop: [ALL]}}
volumeMounts:
- {name: connection, mountPath: /run/postgres, readOnly: true}
- {name: private, mountPath: /run/evo}
containers:
- name: region
image: registry.aeternalisgames.com/aeternalis/aevolution-server:replace-with-immutable-digest
# Flags are "--name value" pairs; see docs/infrastructure/POSTGRESQL.md.
command:
- /usr/local/bin/evo-server
- --profile
- lineage
- --world
- "7"
- --region
- "1"
- --frame
- "9"
- --pg-host
- replace-with-tls-hostname
- --pg-hostaddr
- replace-with-numeric-address
- --pg-port
- "5432"
- --pg-database
- evolution
- --pg-user
- replace-with-runtime-login
- --pg-passfile
- /run/evo/pgpass
- --pg-ca
- /run/postgres/ca.crt
- --objects
- /var/lib/evolution/objects
- --build
- replace-with-retained-build-identity-hex
- --ops-listen
- 0.0.0.0:9100
- --drain-ms
- "20000"
ports:
- {name: ops, containerPort: 9100}
startupProbe:
httpGet: {path: /livez, port: ops}
periodSeconds: 5
timeoutSeconds: 8
failureThreshold: 12
readinessProbe:
httpGet: {path: /readyz, port: ops}
periodSeconds: 5
timeoutSeconds: 8
failureThreshold: 1 # Any doubt about authority removes the pod at once.
livenessProbe:
httpGet: {path: /livez, port: ops}
periodSeconds: 10
timeoutSeconds: 8
failureThreshold: 6
resources:
requests: {cpu: 250m, memory: 128Mi}
limits: {cpu: "1", memory: 512Mi}
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities: {drop: [ALL]}
volumeMounts:
- {name: connection, mountPath: /run/postgres, readOnly: true}
- {name: private, mountPath: /run/evo, readOnly: true}
- {name: objects, mountPath: /var/lib/evolution/objects}
- {name: scratch, mountPath: /tmp}
volumes:
- name: connection
secret:
secretName: evolution-postgres-runtime # Runtime login only; never migrator.
defaultMode: 0440
items:
- {key: pgpass, path: pgpass}
- {key: ca.crt, path: ca.crt}
- name: private
emptyDir: {medium: Memory, sizeLimit: 1Mi}
- name: objects
persistentVolumeClaim: {claimName: evolution-region-objects}
- name: scratch
emptyDir: {medium: Memory, sizeLimit: 8Mi}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: evolution-region-7-1
namespace: evolution-dev
spec:
podSelector:
matchLabels: {app.kubernetes.io/name: evolution-region, evolution.aeternalis/region: "7-1"}
policyTypes: [Ingress, Egress]
ingress:
# Metrics/probes only from the monitoring namespace (kubelet probes use the node).
- from:
- namespaceSelector:
matchLabels: {kubernetes.io/metadata.name: monitoring}
ports: [{port: 9100, protocol: TCP}]
egress:
# Replace with the actual database endpoint; no DNS is needed with a numeric hostaddr.
- to: [{ipBlock: {cidr: 192.0.2.10/32}}]
ports: [{port: 5432, protocol: TCP}]
@@ -0,0 +1,103 @@
# M8.4 region host (ADR-0071). One replica, Recreate: a rolling update would start
# the new pod while the old one still holds the lease; the new pod could never become
# ready, so the old one would never be removed. Recreate drains the old pod (commits,
# releases the lease) before the new one starts and recovers. Readiness is /readyz
# (current lease + verified recovered head), liveness is /livez only. No CPU limit:
# throttling would delay lease renewals.
apiVersion: apps/v1
kind: Deployment
metadata:
name: evo-region
annotations:
argocd.argoproj.io/sync-wave: "1" # After migration and provisioning.
labels:
app.kubernetes.io/name: evo-region
app.kubernetes.io/component: region
spec:
replicas: 1
strategy:
type: Recreate
revisionHistoryLimit: 3
progressDeadlineSeconds: 180
selector:
matchLabels:
app.kubernetes.io/name: evo-region
template:
metadata:
labels:
app.kubernetes.io/name: evo-region
app.kubernetes.io/component: region
app.kubernetes.io/part-of: aevolution
spec:
serviceAccountName: aevolution-region
automountServiceAccountToken: false
nodeSelector:
aevolution.io/stores: "true"
# Longer than --drain-ms, so a graceful stop commits and releases the lease.
terminationGracePeriodSeconds: 45
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile: {type: RuntimeDefault}
containers:
- name: evo-server
image: aevolution-server
# libpq ignores group-readable password files: copy to private memory, then
# exec so evo-server is PID 1 and receives SIGTERM directly.
command: ["/bin/bash", "-c"]
args:
- |
set -eu
umask 077
cp /run/postgres/pgpass /tmp/pgpass
exec /usr/local/bin/evo-server --profile lineage \
--world "$REGION_WORLD" --region "$REGION_ID" --frame "$REGION_FRAME" \
--pg-service /run/postgres/pg_service.conf --pg-passfile /tmp/pgpass \
--objects /objects --objects-reserve-mib 1024 \
--build "$REGION_BUILD" --lease-profile default \
--flush-ms 1000 --drain-ms 20000 --ops-listen 0.0.0.0:9100
env:
- {name: REGION_WORLD, value: set-by-overlay}
- {name: REGION_ID, value: set-by-overlay}
- {name: REGION_FRAME, value: "9"}
- {name: REGION_BUILD, value: set-by-overlay}
ports:
- {name: ops, containerPort: 9100, protocol: TCP}
readinessProbe:
httpGet: {path: /readyz, port: ops}
periodSeconds: 2
timeoutSeconds: 1
failureThreshold: 1
livenessProbe:
httpGet: {path: /livez, port: ops}
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {memory: 256Mi}
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: {drop: [ALL]}
volumeMounts:
- {name: objects, mountPath: /objects}
- {name: connection, mountPath: /run/postgres, readOnly: true}
- {name: scratch, mountPath: /tmp}
volumes:
- name: objects
persistentVolumeClaim:
claimName: aevolution-objects
- name: connection
secret:
secretName: evolution-postgres-runtime
defaultMode: 0440
items:
- {key: pg_service.conf, path: pg_service.conf}
- {key: pgpass, path: pgpass}
- {key: ca.crt, path: ca.crt}
- name: scratch
emptyDir: {medium: Memory, sizeLimit: 1Mi}
@@ -0,0 +1,13 @@
# M8.4 region host component (ADR-0071), included by an environment's workload path
# through region/<env>, which sets the region identity and the server image digest
# (also its build identity). Needs stores/<env> (database egress, migration) and the
# platform layer (namespace, service accounts, object claim).
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- provision-job.yaml
- deployment.yaml
- service.yaml
labels:
- pairs:
app.kubernetes.io/part-of: aevolution
@@ -0,0 +1,77 @@
# Sync hook at wave 0: after the database egress policy (wave -2) and the migration
# (wave -1), before the region host (wave 1). Creates the region's genesis from
# the development opening once, with the migrator (owner) login, and records the
# server image digest as a retained build. Every later sync reports
# "already-provisioned" and changes nothing.
apiVersion: batch/v1
kind: Job
metadata:
name: aevolution-region-provision
labels:
app.kubernetes.io/component: migrate
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
argocd.argoproj.io/sync-wave: "0"
spec:
backoffLimit: 0
activeDeadlineSeconds: 120
template:
metadata:
labels:
app.kubernetes.io/component: migrate
app.kubernetes.io/part-of: aevolution
spec:
serviceAccountName: aevolution-migrator
automountServiceAccountToken: false
restartPolicy: Never
nodeSelector:
aevolution.io/stores: "true"
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
fsGroup: 10001
seccompProfile: {type: RuntimeDefault}
containers:
- name: provision
image: aevolution-server
command: ["/bin/bash", "-c"]
args:
- |
set -eu
umask 077
cp /run/postgres/pgpass /tmp/pgpass
exec /usr/local/bin/evo-world-clone provision \
--world "$REGION_WORLD" --region "$REGION_ID" --opening development-v1 \
--pg-service /run/postgres/pg_service.conf --pg-passfile /tmp/pgpass \
--objects /objects --build "$REGION_BUILD" --retain-build true
env:
- {name: REGION_WORLD, value: set-by-overlay}
- {name: REGION_ID, value: set-by-overlay}
- {name: REGION_BUILD, value: set-by-overlay}
resources:
requests: {cpu: 50m, memory: 64Mi}
limits: {memory: 256Mi}
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities: {drop: [ALL]}
volumeMounts:
- {name: objects, mountPath: /objects}
- {name: connection, mountPath: /run/postgres, readOnly: true}
- {name: scratch, mountPath: /tmp}
volumes:
- name: objects
persistentVolumeClaim:
claimName: aevolution-objects
- name: connection
secret:
secretName: evolution-postgres-migrator
defaultMode: 0440
items:
- {key: pg_service.conf, path: pg_service.conf}
- {key: pgpass, path: pgpass}
- {key: ca.crt, path: ca.crt}
- name: scratch
emptyDir: {medium: Memory, sizeLimit: 1Mi}
@@ -0,0 +1,16 @@
# Operations endpoints (/livez, /readyz, /metrics) inside the cluster only; no
# NodePort or LoadBalancer (the quota forbids both). Scraping is enabled in M8.5.
apiVersion: v1
kind: Service
metadata:
name: evo-region-ops
annotations:
argocd.argoproj.io/sync-wave: "1" # After migration and provisioning.
labels:
app.kubernetes.io/name: evo-region
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: evo-region
ports:
- {name: ops, port: 9100, targetPort: ops, protocol: TCP}
@@ -0,0 +1,14 @@
# Region host for evolution-dev: world 100, region 1, frame 9 (development opening).
# The server image digest appears twice on purpose: as the image and as the build
# identity the region records in every commit. The policy check requires they match.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: evolution-dev
resources:
- ../base
images:
- name: aevolution-server
newName: registry.aeternalisgames.com/aeternalis/aevolution-server
digest: sha256:661e433ef0cf4fcb619f1e2ed623297ca20ade896aa16d29e67f47e6ccfa2075
patches:
- path: region-values.yaml
@@ -0,0 +1,28 @@
# Region identity and build (the server image digest without "sha256:").
apiVersion: apps/v1
kind: Deployment
metadata:
name: evo-region
spec:
template:
spec:
containers:
- name: evo-server
env:
- {name: REGION_WORLD, value: "100"}
- {name: REGION_ID, value: "1"}
- {name: REGION_BUILD, value: 661e433ef0cf4fcb619f1e2ed623297ca20ade896aa16d29e67f47e6ccfa2075}
---
apiVersion: batch/v1
kind: Job
metadata:
name: aevolution-region-provision
spec:
template:
spec:
containers:
- name: provision
env:
- {name: REGION_WORLD, value: "100"}
- {name: REGION_ID, value: "1"}
- {name: REGION_BUILD, value: 661e433ef0cf4fcb619f1e2ed623297ca20ade896aa16d29e67f47e6ccfa2075}
@@ -0,0 +1,14 @@
# Region host for evolution-stage: world 200, region 1, frame 9 (development opening).
# The server image digest appears twice on purpose: as the image and as the build
# identity the region records in every commit. The policy check requires they match.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: evolution-stage
resources:
- ../base
images:
- name: aevolution-server
newName: registry.aeternalisgames.com/aeternalis/aevolution-server
digest: sha256:661e433ef0cf4fcb619f1e2ed623297ca20ade896aa16d29e67f47e6ccfa2075
patches:
- path: region-values.yaml
@@ -0,0 +1,28 @@
# Region identity and build (the server image digest without "sha256:").
apiVersion: apps/v1
kind: Deployment
metadata:
name: evo-region
spec:
template:
spec:
containers:
- name: evo-server
env:
- {name: REGION_WORLD, value: "200"}
- {name: REGION_ID, value: "1"}
- {name: REGION_BUILD, value: 661e433ef0cf4fcb619f1e2ed623297ca20ade896aa16d29e67f47e6ccfa2075}
---
apiVersion: batch/v1
kind: Job
metadata:
name: aevolution-region-provision
spec:
template:
spec:
containers:
- name: provision
env:
- {name: REGION_WORLD, value: "200"}
- {name: REGION_ID, value: "1"}
- {name: REGION_BUILD, value: 661e433ef0cf4fcb619f1e2ed623297ca20ade896aa16d29e67f47e6ccfa2075}
+2 -1
View File
@@ -2,4 +2,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: evolution-stage
resources:
- ../stores/stage
- ../stores/stage
- ../region/stage