diff --git a/.gitea/workflows/promote-dev.yaml b/.gitea/workflows/promote-dev.yaml index 7f1b215..ed311c5 100644 --- a/.gitea/workflows/promote-dev.yaml +++ b/.gitea/workflows/promote-dev.yaml @@ -32,6 +32,11 @@ jobs: python3 -m pip install --user --break-system-packages \ --disable-pip-version-check PyYAML + - name: Prepare pinned kustomize + run: | + set -eu + bash tools/ci/prepare-kustomize.sh + - name: Update dev image and build identity env: IMAGE_DIGEST: ${{ inputs.image_digest }} diff --git a/deploy/kubernetes/region/dev-session-route/listener.yaml b/deploy/kubernetes/region/dev-session-route/listener.yaml index c3d7125..36c8dd7 100644 --- a/deploy/kubernetes/region/dev-session-route/listener.yaml +++ b/deploy/kubernetes/region/dev-session-route/listener.yaml @@ -12,7 +12,7 @@ spec: env: - {name: REGION_SESSION, value: "dev"} - {name: REGION_SESSION_LISTEN, value: "0.0.0.0:7400"} - - {name: REGION_SESSION_WINDOW, value: "dev-20261006T170005-18538:1791307205"} + - {name: REGION_SESSION_WINDOW, value: "dev-20261006T173934:1791309574"} ports: # QUIC (ADR-0090): UDP only; nothing listens on TCP 7400. - {name: session, containerPort: 7400, protocol: UDP} diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md new file mode 100644 index 0000000..4460cc2 --- /dev/null +++ b/docs/DEPLOYMENT.md @@ -0,0 +1,180 @@ +# AEvolution image deployment guide + +This guide covers the supported path for promoting an immutable server image to +`evolution-dev`. It is deliberately conservative: image publication, GitOps +promotion, region-build retention, and Argo reconciliation are separate gates. + +## Rules + +- Promote immutable digests, never mutable tags. +- Do not edit live Deployments or run ad-hoc SQL against the region database. +- Do not reset the database, PVC, object store, region history, or world data. +- Stage and dev have independent image pins. A dev promotion must not change stage. +- A server image must be retained in the database before the region can acquire it. +- Use the cluster clock for `REGION_SESSION_WINDOW`; the expiry must be in the future + and within the server's maximum allowed window. + +## 1. Start from current main + +```bash +git fetch origin main +git checkout main +git reset --hard origin/main +git status --short +``` + +The working tree must be clean before promotion. + +## 2. Build and publish the server image + +Use the repository Foundation/build workflow for `main`. Do not build a local image +and substitute a tag in GitOps. + +After the workflow succeeds, record the exact published digest: + +```text +registry.aeternalisgames.com/aeternalis/aevolution-server@sha256:<64 lowercase hex> +``` + +Do not continue if the publication step was skipped or no immutable digest was +reported. + +## 3. Promote the digest to dev + +The supported Gitea action is **Promote server image to dev**. Supply the complete +`sha256:...` digest as `image_digest`. + +The action updates both of these files, and they must match: + +```text +deploy/kubernetes/region/dev/kustomization.yaml +deploy/kubernetes/region/dev/region-values.yaml +``` + +The action prepares the pinned kustomize binary before running policy validation. +If running the equivalent process locally, use: + +```bash +python3 -m pip install --user --break-system-packages PyYAML +bash tools/ci/prepare-kustomize.sh +python3 tools/ci/deploy-policy-check.py +git diff --check +``` + +Then commit and push only the intended dev promotion changes: + +```bash +git add deploy/kubernetes/region/dev/kustomization.yaml \ + deploy/kubernetes/region/dev/region-values.yaml +git commit -m "deploy(dev): promote selected server image" +git push origin HEAD:main +``` + +The policy check must report zero failures. Do not bypass it with +`kubectl apply`, `--force`, or a mutable image tag. + +## 4. Refresh the session window when the dev route is enabled + +Generate the window using the cluster-synchronized clock: + +```bash +ssh shard@172.10.1.160 ' + id="dev-$(date -u +%Y%m%dT%H%M%S)-$RANDOM" + expiry="$(date -u -d "+20 minutes" +%s)" + printf "%s:%s\\n" "$id" "$expiry" +' +``` + +Update the value in: + +```text +deploy/kubernetes/region/dev-session-route/listener.yaml +``` + +Run the policy check, commit, push, and sync again. The identifier is opaque; the +contract is `identifier:unix-expiry`. + +## 5. Run a full Argo sync + +A resource-targeted sync is not sufficient for an image promotion. It can update the +Deployment while skipping the sync-wave provision hook. + +Use a full sync of `aevolution-dev`, not a sync limited to `Deployment/evo-region`. +Verify the Application revision and operation before proceeding. + +The sync order is: + +| Wave | Resource | Purpose | +|---:|---|---| +| -2 | database egress/policy | Permit the migration/provision path | +| -1 | database migration Job | Apply schema migrations and privilege audit | +| 0 | `aevolution-region-provision` hook | Retain the selected server build | +| 1 | `evo-region` Deployment | Start the region using that retained build | + +## 6. Verify the provision hook + +The hook must use the same server digest as the Deployment. A successful existing +region normally reports `already-provisioned`, but the command still runs +`--retain-build` before that check and therefore records a newly promoted build. + +Check: + +```bash +kubectl -n evolution-dev get job aevolution-region-provision -o wide +kubectl -n evolution-dev logs job/aevolution-region-provision +``` + +If the new build is not retained, the region will start and immediately show: + +```text +phase quarantined epoch=0 committed=0 simulated=0 +exit 4 +``` + +That is a build-retention failure, not a reason to reset the region. Re-run the +supported sync hook using the current rendered manifest. + +## 7. Verify the region + +All of these must be true: + +```bash +kubectl -n evolution-dev get pod -l app.kubernetes.io/name=evo-region +kubectl -n evolution-dev logs deployment/evo-region --tail=50 +kubectl -n argocd get application aevolution-dev +``` + +Expected runtime indicators include: + +```text +phase running +ready 1 +``` + +Also verify `Ready=True`, zero new restarts, and Argo `Synced / Healthy`. `Running` +alone is not sufficient. + +## 8. Probe and acceptance + +For an authenticated M10 probe, retain the matching operator bundle until all tests +finish: + +- `dev-ca.pem` +- `issuer.key` + +The normal dev overlay has `REGION_INPUTS` empty, so a probe can authenticate and +receive an intention receipt without producing a simulated outcome. Full durable +outcome testing requires the repository-supported fault-drill/synthetic-input path; +do not enable it with an ad-hoc live environment change. + +## Recovery checklist + +If the region enters CrashLoopBackOff: + +1. Capture the current image digest, `REGION_BUILD`, Argo revision, previous logs, and + Job image. +2. If logs show `quarantined epoch=0` with zero store failures, compare the build + against the retained-build Job/database path. +3. If the provision Job is stale, perform a full Argo sync so the sync-wave 0 hook runs. +4. Never delete the database, PVC, object store, or committed region history. +5. Do not force-sync only the Deployment while the provision hook is pending.