feat(ops): M8.9 upgrade rehearsal on clones, barrier and node plan (ADR-0076)

Drill-only --synthetic-until/--accelerate and evo-world-clone compare
(compare_region_histories) prove a candidate build runs the same world on
clones of one cut before any barrier. upgrade-check.sh rehearses a failed
migration under load and the version barrier with real hosts; upgrade-barrier.sh
retires an old build in dev/stage only once the new digest is Ready. Nodes:
aevolution.io/stores pins every objects-volume pod, aevolution.io/compute is
preference-only (Alertmanager first), checked by policy and scheduling-check.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-09-30 17:51:17 -04:00
co-authored by Claude Opus 5.5
parent 645d16b27a
commit 5b482ad6a3
19 changed files with 1028 additions and 12 deletions
+17
View File
@@ -232,3 +232,20 @@ rebuilt as isolated world ...` counts as restore evidence. The restore uses fres
roles on a throwaway cluster; nothing reaches the source database. A restored copy
keeps the source world identity and is never served; restoring in place after host
loss is M8.8.
## Upgrades and nodes (M8.9, ADR-0076)
**Build upgrade (same rules).**
1. Rehearse on clones first: clone the cut twice into an isolated database, run the
current and candidate builds with `--synthetic-inputs development-v1
--synthetic-until T --accelerate 20`, then `evo-world-clone compare` must exit 0.
2. Pin the new server digest in `deploy/kubernetes/region/<env>` (image and
`REGION_BUILD`), add the old digest to `REGION_RETAINED`, and sync. The migrate
hook (wave -1) stops the sync on failure, leaving the old host running.
3. When the new pod is Ready: `tools/ops/upgrade-barrier.sh <env> --retire OLD`.
Keep OLD in `REGION_RETAINED`: its history stays recoverable.
**Adding a node.** Label it `aevolution.io/compute=true` and run
`tools/ops/scheduling-check.sh <env>`. Only stateless pods (Alertmanager today) may
move there. Never move `aevolution.io/stores` to rebalance: the objects volume stays
on its node; a region moves only by restore or clone onto the new node's storage.