Serialize build retirement and clone classification, bound ops accepts
Review fixes for M7.6-M7.9. Schema v9 applies one lock order: the barrier takes a database-wide upgrade lock exclusively, while acquisitions, lease checks, provisioning and the retired-build trigger take it shared first, so a queued acquisition can no longer regain authority for a just-retired build and concurrent barriers cannot retire the last current build. Provisioning takes a per-world lock before classifying a world as original or a clone of one source. The operations listener makes at most four accept attempts per serve(). Race regressions prove each contender is blocked before commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -954,3 +954,8 @@ lease:
|
||||
run. Host-loss durability is M8.
|
||||
|
||||
Next: M8.1, the host inventory and budgets.
|
||||
|
||||
Review fixes (2026-09-29): schema v9 serializes build retirement with acquisition,
|
||||
leases and provisioning under one lock order. It also serializes clone-world
|
||||
classification per world. The operations listener's accept work per loop pass is
|
||||
now bounded. Regression tests and five mutations cover each fix.
|
||||
|
||||
@@ -99,3 +99,15 @@ zlib, CA certificates):
|
||||
- A later gateway must honor readiness and the lease epoch before routing.
|
||||
- Promoting to World0 needs a scanned, signed image and a reviewed alerting policy
|
||||
(M8).
|
||||
|
||||
## Amendment: bounded accept work (2026-09-29 review)
|
||||
`serve()` used to accept, and close excess connections, until the backlog was
|
||||
empty. A replenished stream of connections could therefore hold the host loop
|
||||
without bound. Each call now makes at most `max_accepts_per_serve` (4) accept
|
||||
attempts, refused ones included. Pending connections wait in the 16-entry listen
|
||||
backlog, and deadlines are judged after the accept work.
|
||||
|
||||
Tests check the per-call budget, including at the connection cap, and that an
|
||||
accepted probe is still answered under replenished pressure. A real server keeps
|
||||
its lease and readiness while a client opens and closes connections continuously
|
||||
through 60 ticks.
|
||||
|
||||
@@ -109,3 +109,18 @@ alone does not prove retention; the audit does.
|
||||
defeats isolation.
|
||||
- **Recording provenance only in a table.** The chain would not commit to its
|
||||
origin, and cross-database clones could not verify it.
|
||||
|
||||
## Amendment: serialized world classification (schema v9, 2026-09-29 review)
|
||||
The rule "original, or a clone of exactly one source" was enforced by unlocked
|
||||
existence checks. Two concurrent clones of different sources into different
|
||||
regions of one world could therefore both commit (reproduced by review). Schema
|
||||
v9 fixes this:
|
||||
- Both provisioning functions take a per-world advisory lock, keyed by a hash of
|
||||
the world ID, after the shared upgrade lock (ADR-0068 amendment).
|
||||
- They then check the classification. Because the checks run after the lock wait,
|
||||
they see every committed provisioning of that world.
|
||||
- Unrelated worlds provision in parallel. A hash collision only adds waiting.
|
||||
|
||||
Regression tests race different-source clones, and an original region against a
|
||||
clone, into one world, and expect exactly one to succeed. Same-source clones into
|
||||
different regions both succeed.
|
||||
|
||||
@@ -94,3 +94,28 @@ The results are in engineering/DURABILITY_QUALIFICATION.md.
|
||||
mid-upgrade before the new build was deployed.
|
||||
- **In-place rules migration of a live region.** It would change physical laws
|
||||
without the clone-and-compare approval that the vision requires.
|
||||
|
||||
## Amendment: serialized retirement (schema v9, 2026-09-29 review)
|
||||
In v8, acquisition checked eligibility before locking the lease row, without
|
||||
serializing against the barrier. A request queued behind a barrier could
|
||||
therefore acquire for the build just retired, then keep renewing (reproduced by
|
||||
review). Schema v9 fixes this with one lock order: the upgrade lock (advisory,
|
||||
database-wide), then the world classification lock, then the lease row, then
|
||||
head and commit rows.
|
||||
- `upgrade_barrier` takes the upgrade lock exclusively before anything else. This
|
||||
also serializes barriers, so two operators cannot each retire what they believe
|
||||
is a non-last build.
|
||||
- Both acquisition forms, `check_region_lease` (and with it publication, renewal
|
||||
and release), provisioning and the retired-build trigger take the upgrade lock
|
||||
shared first. Their eligibility checks therefore run after any concurrent
|
||||
barrier commits.
|
||||
|
||||
The lock order is the same everywhere, so publication cannot deadlock against the
|
||||
barrier. Regression tests hold a barrier open, prove the contender is blocked,
|
||||
commit, and check that a queued acquisition of the retired build is refused (both
|
||||
forms) and that the last current build cannot be retired by racing barriers.
|
||||
|
||||
A runtime login can still take PostgreSQL advisory locks directly, and so could
|
||||
delay barriers or acquisitions. It cannot gain authority that way. An owner can
|
||||
terminate such sessions. Removing that capability needs a pooler or role policy
|
||||
(M8).
|
||||
|
||||
@@ -2534,3 +2534,49 @@ Only clean-to-v1 and exact-v1 restart are supported migration paths at this gate
|
||||
The migration test caught this one after the host-test correction.
|
||||
- Not executed: Docker, the Gitea CI, cluster runs, a separate database host, and
|
||||
host or storage loss (M8).
|
||||
|
||||
## M7.6–M7.9 review fixes (schema v9) - 2026-09-29
|
||||
|
||||
Review `.cache/reviews/M7_6-M7_9_REVIEW.md` found three issues. All three are
|
||||
fixed:
|
||||
- **[P1] Build retirement could race acquisition.** Schema v9 applies one
|
||||
upgrade-lock order across the barrier, acquisitions, lease checks, provisioning
|
||||
and the retired-build trigger.
|
||||
- **[P2] Clone-world classification could race.** Provisioning now takes a
|
||||
per-world lock and checks classification after it.
|
||||
- **[P2] The operations listener accepted without bound.** Each `serve()` now
|
||||
makes at most 4 accept attempts.
|
||||
|
||||
Evidence:
|
||||
- WSL Linux `checks` set (Debug, Release, analysis, ASan/UBSan): CTest 59/59
|
||||
each. The format check passed.
|
||||
- PostgreSQL ASan/UBSan: full suite passed. Analysis and Release were rerun green
|
||||
after fixing two clang-tidy findings in the new tests. Upgrade paths now end at
|
||||
1..9.
|
||||
- Race tests (database `evo_race`): each holds a transaction open, confirms the
|
||||
contender is blocked on a lock via `pg_stat_activity`, then commits.
|
||||
- A build-less acquisition queued behind a barrier is refused (EVL06).
|
||||
- A build-aware acquisition of the build being retired is refused, and the
|
||||
region stays unheld.
|
||||
- The next build acquires.
|
||||
- Racing barriers for the last two current builds: one returns 22023 and one
|
||||
build stays current.
|
||||
- Different-source clones into one world, and an original region racing a
|
||||
clone: the second is refused (22023).
|
||||
- Same-source clones into different regions both succeed.
|
||||
- The final origins are exactly `2/1<10, 4/1<10, 4/2<10`.
|
||||
- Accept budget:
|
||||
- With 16 queued connections, successive `serve()` calls give 4 open, then 8
|
||||
open with 0 refused, then 8 open with 4 refused.
|
||||
- Refusals never exceed 4 per call under replenished connections.
|
||||
- The first accepted probe got 200.
|
||||
- A real server acknowledged 60 ticks under continuous connect/close churn
|
||||
without losing readiness.
|
||||
- Mutations, each reverting one fix (restored and rebuilt). Each was caught:
|
||||
- an unlocked build-aware acquire;
|
||||
- an unlocked legacy acquire;
|
||||
- an unlocked barrier;
|
||||
- no per-world lock;
|
||||
- an accept budget of 1000.
|
||||
- Qualification was unchanged: SIGKILL to ready took 19.55–19.60 s.
|
||||
- Windows was not rerun. All changes are Linux PostgreSQL, server or doc files.
|
||||
|
||||
@@ -164,6 +164,13 @@ To upgrade the build:
|
||||
returns the number of leases fenced, and refuses to retire the last current
|
||||
build.
|
||||
|
||||
Schema v9 serializes all of this under one lock order (ADR-0068 amendment). The
|
||||
barrier takes a database-wide advisory lock (71420002) exclusively. Acquisitions,
|
||||
lease checks, provisioning and the retired-build trigger take it shared, and
|
||||
provisioning also takes a per-world lock (71420003). A barrier therefore briefly
|
||||
waits for in-flight commits and blocks new ones. Keep other advisory locks away
|
||||
from these two keys.
|
||||
|
||||
Old images then quarantine (exit 4) instead of serving. Rules and format changes
|
||||
are refused explicitly; there is no converter.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user