Files
aevolution/docs/adr/0068-upgrade-barrier.md
T
2026-10-08 12:08:53 -04:00

123 lines
6.1 KiB
Markdown

[Reviewed]
# ADR-0068: Upgrade barrier, build-bound authority and durability qualification
## Status
Accepted for M7.9b, 2026-09-29. It completes M7.9 with ADR-0067 (clones). It
changes the ADR-0060 acquisition contract (schema v8). Rules and format
conversions remain unimplemented: no converter exists, so every change is
refused explicitly.
## Context
M7.9 requires:
- explicit rule and format rejection or migration;
- evidence for a migration failure and for fencing old writers;
- measured recovery and batching latency.
PERSISTENCE_ARCHITECTURE's upgrade path is: barrier, clone, migrate, compare,
approve, then a compatible deployment at a world barrier. That implies:
- A new build must be able to take over a region.
- Every writer of the old build must be stopped for good, including one that is
paused, idle, or restarted from an old image.
- History written by the old build must stay recoverable.
Before v8, any retained build could publish, and acquisition did not name a
build. A fenced old host could therefore re-acquire an idle region and renew it
forever.
## Decision
**Build-bound authority (schema v8).**
- `retained_builds.retired_at` marks a build as retired from publication. The
row, and so the recoverability of its history, stays.
- A `region_commits` insert trigger refuses any new commit that names a retired
build (EVL06). This covers publications, provisioned geneses and clone
geneses.
- `acquire_region_lease(world, region, duration, build)` grants authority only to
a retained, unretired build (EVL06 otherwise). The C++ API takes the build. The
host quarantines on `unsupported` (exit 4): a retired image never serves or
holds an idle lease.
- The build-less 3-argument form remains for pre-v8 hosts during a rolling
schema upgrade. It refuses once any build has been retired.
- Both forms wrap the internal `grant_region_lease`, which the runtime cannot
execute. The runtime audit now covers twelve functions.
**Barrier.** The owner-only `upgrade_barrier(build)` retires one build and clears
every current lease holder in the database, in one transaction:
- Existing holders are fenced at their next renewal or publication.
- They cannot re-acquire by either form.
- A commit already waiting on a lease row is judged after the barrier commits.
- Retiring the last current build is refused.
- Epochs are kept, so the next acquisition takes a fresh one.
Operator order:
1. Migrate the schema. Old hosts keep their leases, because renew and publish are
unchanged.
2. Retain the new build.
3. Deploy hosts with `--build new --retained old`.
4. Run the barrier for the old build.
**Explicit rejection.** A region never changes profile, versions, quantum or
history profile (EVL06 at publication; `unsupported` at recovery). A clone
inherits them unchanged. No rules or format converter is registered. Adding one
requires a new ADR, a versioned conversion that is replayed and compared on a
clone, and approval. Retirement is irreversible through this interface. Undoing
it is an audited owner break-glass action, valid only while formats and rules are
identical.
**Qualification fixture.** The fixture is:
- the production lease policy (20 s lease, 5 s renewal, 2 s margin) and a 1 s flush;
- one lineage region paced at 50 ms per tick;
- a local socket and filesystem, on WSL2.
It measures:
- acknowledgement latency;
- the time from SIGKILL to the successor's readiness, against the proposed 60 s
target (asserted for this fixture only);
- head verification, full audit and clone cost.
The results are in engineering/DURABILITY_QUALIFICATION.md.
## Consequences
- Recovery after losing a worker is dominated by the lease: a successor waits out
the dead holder's remaining lease, up to 20 s, and verification adds
milliseconds. Shortening the lease trades that time against false fencing
under database latency (ADR-0060).
- Migrations run in one transaction under an advisory lock and a 5 s lock timeout.
A publication waiting on migration locks may become ambiguous; resolution
handles it (ADR-0063). A failed migration leaves the previous schema exactly.
- Binaries built before M7.9b (3-argument acquisition) work against v8 until the
first barrier. Binaries from M7.9b onwards require schema v8.
## Alternatives rejected
- **Retirement only, checked at publication.** An idle old host could still hold
a lease forever.
- **Bumping epochs without retirement.** Old hosts would simply re-acquire.
- **Dropping the 3-argument acquisition in v8.** Every old host would be stranded
mid-upgrade before the new build was deployed.
- **In-place rules migration of a live region.** It would change physical laws
without the clone-and-compare approval that the vision requires.
## Amendment: serialized retirement (schema v9, 2026-09-29 review)
In v8, acquisition checked eligibility before locking the lease row, without
serializing against the barrier. A request queued behind a barrier could
therefore acquire for the build just retired, then keep renewing (reproduced by
review). Schema v9 fixes this with one lock order: the upgrade lock (advisory,
database-wide), then the world classification lock, then the lease row, then
head and commit rows.
- `upgrade_barrier` takes the upgrade lock exclusively before anything else. This
also serializes barriers, so two operators cannot each retire what they believe
is a non-last build.
- Both acquisition forms, `check_region_lease` (and with it publication, renewal
and release), provisioning and the retired-build trigger take the upgrade lock
shared first. Their eligibility checks therefore run after any concurrent
barrier commits.
The lock order is the same everywhere, so publication cannot deadlock against the
barrier. Regression tests hold a barrier open, prove the contender is blocked,
commit, and check that a queued acquisition of the retired build is refused (both
forms) and that the last current build cannot be retired by racing barriers.
A runtime login can still take PostgreSQL advisory locks directly, and so could
delay barriers or acquisitions. It cannot gain authority that way. An owner can
terminate such sessions. Removing that capability needs a pooler or role policy
(M8).