M12.4: retain read-only destination migration stages
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (ecs) (push) Successful in 2m25s
Foundation / checks (network) (push) Successful in 20m44s
Foundation / checks (client) (push) Successful in 21m57s
Foundation / checks (tsan) (push) Successful in 22m46s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (checks) (push) Canceled after 30m2s
Foundation / checks (postgres) (push) Canceled after 29m56s

This commit is contained in:
dingo
2026-10-18 09:32:50 -04:00
parent c625a4cb68
commit 0515702964
21 changed files with 1023 additions and 21 deletions
+17
View File
@@ -1,5 +1,22 @@
# Project status and next steps
## M12.4i durable read-only destination staging - 2026-10-18
ADR-0148/schema 48 stages the destination under current authority at its exact
captured head. A verified source capsule and complete destination ghost are
retained for replay, while the source owns all outgoing stock and neither head
moves. Both migration roles pause publication/admission and host readiness
across replacement. Explicit source abort releases both pointers atomically;
stage/abort races and exact retries cannot activate or reopen an aborted ghost.
Native Release passes 78/78; the private PostgreSQL staging suite, ASan/UBSan,
Clang analysis, format and shell syntax checks pass. The complete private
persistence/process/backup/DR/migration/permission suite passes in 501 s.
[Evidence and limits](engineering/VALIDATION.md)
retain atomic paired publication, its common immutable receipt, commit/abort
races and recovery/activation as the next M12.4 segment. All of M12 remains in
progress. No deployment, automatic movement or live region changed.
## M12.4h durable source freeze and explicit abort - 2026-10-18
ADR-0147/schema 47 retains exact biological migration preparation, replay capsules
@@ -0,0 +1,56 @@
# ADR-0148: Durable destination staging without activation
## Status
Accepted and locally qualified for scoped M12.4i implementation, 2026-10-18.
Executed evidence and limits are in [VALIDATION](../engineering/VALIDATION.md).
Paired result publication and activation remain subsequent
work. This selects no route, automatic migration or deployment.
## Decision
Extend ADR-0147 preparation with destination-authorized staging at the exact
captured destination head. Lock the two regional lease rows in region order,
check current destination authority and the retained source freeze, and refuse
pending destination work. Retain one immutable staging record and one bounded
destination pending pointer. A region cannot participate in two workflows or
simultaneously be a source and destination. Both biological heads stay unchanged.
Restore both verified predecessor payloads and replay the exact retained trace.
Copy the source predecessor into the destination object namespace, and upload
and read back the derived destination candidate there. Record its SHA-256 and
length. It is read-only evidence: it is never a committed checkpoint, owned
stock, active population, census entry or permission to run biology. Recovery
replays both cuts from the destination namespace and compares the complete
candidate bytes. Equal aggregate material alone is insufficient.
New checkpoints, profile changes, intervals and player reservations are refused
at a staged destination. Its host pauses readiness and advancement, retains exact
requests after unknown replies, renews authority and recovers the pause across
replacement. No lease expiry, drain or timeout releases it. Explicit source
abort atomically appends the existing immutable abort and clears both pending
pointers. It requires no destination authority because no stock has moved.
Stage and abort serialize on the source lease row: either staging wins then
abort removes both pointers, or abort wins and new staging is refused. An exact
retry of a previously staged, aborted request returns that retained outcome and
never reopens either pointer. Destination movement before first staging requires
explicit source abort and a new preparation, without silent rebasing.
Keep all staged objects and predecessor evidence after abort. Backup already
copies immutable namespace objects and shared metadata; this is not cellular
clone/restore qualification. The future paired publisher must recheck both
current fences, heads, staged object bindings and unsettled request, then commit
both results and the common receipt atomically before activation.
## Alternatives, risks and qualification
An in-memory ghost disappears on process loss. Activating the uploaded payload
duplicates source stock. Advancing a destination after staging invalidates the
paired candidate. Freeze the whole destination instead; the coarse pause and
extra immutable storage are explicit costs, not a final movement latency policy.
Require both representations, real player-origin wildlife, full ghost replay,
unchanged heads and stock, conflicting/duplicate requests, pending-work refusal,
stale destination authority, replacement/restart, missing/corrupt/semantically
false ghost evidence, ambiguous replies, process kills, stage/abort races and
host readiness. No coupled ecology, world-cut or multigeneration claim follows.
+3 -1
View File
@@ -1,6 +1,8 @@
# Architecture decision records
- [ADR-0147: Durable biological preparation and source freeze](0147-durable-cellular-migration-freeze.md) - schema 47 retains exact preparation/capsules and explicit abort; the source stays authoritative and paused across worker replacement. Destination staging and paired activation remain open.
- [ADR-0148: Durable destination staging without activation](0148-durable-cellular-migration-staging.md) - schema 48 freezes the destination at its exact cut, retains a verified read-only ghost and replays it on recovery; source abort releases both pauses. Paired publication/activation remain open.
- [ADR-0147: Durable biological preparation and source freeze](0147-durable-cellular-migration-freeze.md) - schema 47 retains exact preparation/capsules and explicit abort; the source stays authoritative and paused across worker replacement. Paired activation remains open.
- [ADR-0146: Durable boundary-accounting upgrade admission](0146-durable-boundary-accounting-upgrade.md) - schema 46 extends immutable version-upgrade evidence to same-profile bundle 9->10, full upgrade replay, population batch 2 and preserved representation/interval guards; paired migration escrow remains open.
@@ -16,8 +16,15 @@ uses both retained manifests and a verified destination capsule in the source's
object namespace. Both biological heads stay unchanged; destination inspection
is optimistic and grants no destination authority. Source publication and new
interval/player reservations are blocked until explicit settlement. See
[ADR-0147](../adr/0147-durable-cellular-migration-freeze.md). Destination staging,
atomic paired result publication and activation remain required.
[ADR-0147](../adr/0147-durable-cellular-migration-freeze.md).
M12.4i/schema 48 adds destination-authorized staging under ordered endpoint locks
and the exact retained preparation/heads. A verified source capsule and derived
destination ghost stay read-only in the destination namespace. Both roles pause
publication, interval/player admission and host readiness across replacement.
Explicit source abort releases both pointers atomically; stage/abort races cannot
activate or strand stock. See [ADR-0148](../adr/0148-durable-cellular-migration-staging.md).
Atomic paired result publication, its common receipt and activation remain required.
M7.1 now scopes the first implementation in [ADR-0058](../adr/0058-region-durability-contract.md)
and [REGION_DURABILITY.md](REGION_DURABILITY.md): full checkpoint batches, not
+7 -1
View File
@@ -70,7 +70,13 @@ destination predecessor capsule in the source namespace allows complete pair
replay without consulting destination live state. The owning host pauses
readiness, retains unknown requests and never treats timeout/drain as abort.
See [the freeze contract](../../schemas/ecology/cellular-migration-freeze-v1.md).
Destination staging, paired commit/abort races and activation remain open.
ADR-0148/schema 48 now stages a verified destination ghost and source replay
capsule under current destination authority and ordered endpoint locks. The
destination head remains unchanged and is durably paused alongside the source.
Recovery verifies the complete ghost without activating it; explicit source abort
releases both pointers atomically. Stage/abort races serialize on source authority.
See [the staging contract](../../schemas/ecology/cellular-migration-stage-v1.md).
Atomic paired publication, commit/abort races and activation remain open.
World/region/frame are nonzero durable IDs. No ID is a Pod name. The host records
its unique holder ID and monotonically increasing nonzero lease epoch. Exhaustion
@@ -140,3 +140,26 @@ and canonically encoded false member/material declarations fail complete replay.
See [the contract](../../schemas/ecology/cellular-migration-freeze-v1.md).
Destination staging, atomic paired head publication, commit/abort races and
activation remain the next segment; no route or automatic migration is enabled.
## M12.4i durable read-only destination staging
ADR-0148/schema 48 freezes the destination at its exact captured head under
current destination authority and ordered endpoint locks. The adapter restores
both cuts, replays the retained trace, copies the source capsule into the
destination namespace and uploads/reads back the derived destination ghost.
Its exact digest/length is immutable retry evidence, not an ownership receipt.
Recovery uses those local objects and compares the complete rederived ghost.
Neither biological head moves, and the source still owns all outgoing stock.
Both local migration roles are exclusive and block new publication, profile
changes, intervals and player reservations. Unknown staging retains exact bytes;
the owning destination pauses readiness/advancement through worker replacement.
Source abort serializes with staging and releases both pointers atomically.
Aborted-stage retries return their old outcome and never reopen a pause.
Missing/corrupt evidence and real but semantically false objects fail closed.
See [the staging contract](../../schemas/ecology/cellular-migration-stage-v1.md).
Atomic paired publication/common receipt, commit/abort races, recovery/activation
and linked ecology remain subsequent work. The ghost cannot run biology or
contribute owned census/extinction stock. No biological format, rule, route,
automatic movement, live region or deployment changes.
+6 -3
View File
@@ -64,9 +64,12 @@ or close source-freeze/destination-activation and escrow qualification.
ADR-0147/schema 47 now adds durable source preparation/freeze, retained replay
capsules, explicit abort and host pause/recovery. Source stock remains owned at
its exact head through worker replacement; no timeout or drain releases it.
Destination staging, atomic paired commit, commit/abort races and activation
remain open. The destination head is only checked optimistically at preparation
and must be rechecked before any future stage/commit.
ADR-0148/schema 48 now supplies destination-authorized staging at its exact
captured head, verified read-only ghost/capsule retention and full destination
recovery. Both endpoints stay paused; explicit source abort atomically releases
both pointers and races safely with staging. Atomic paired commit, its common
receipt, commit/abort races and activation remain open. Preparation remains an
optimistic destination observation; first staging rechecks both captured heads.
The transfer tests also expose the current rules-8 maturation gap: buds retain
their subadult realization while reproduction requires a fully realized parent.
Finite extra food alone does not close repeated wildlife generations. A versioned
+13
View File
@@ -32,6 +32,19 @@ the C++ decoder. The metric has one fixed unlabelled series and remains bounded.
These are preparation/abort checks, not proof of destination activation or atomic
paired publication; full historical persistence regression is also required.
M12.4i's private staging matrix covers both bundle-10 representations and actual
player-origin wildlife. It requires unchanged biological heads, complete ghost
replay, exact/conflicting retries, refusal of moved destination heads and pending
intervals, publication/profile/life-reservation guards and exclusive endpoint
roles. Stale destination epochs fail; replacement retains the read-only ghost.
Missing source capsules, missing/same-length corrupt ghosts and valid but false
ghost objects fail recovery. Real lock-timeout ambiguity, SIGKILL before/after
stage commit and independent-connection stage/abort races preserve one outcome.
The owning destination host retains exact uncertain work across restart and
resumes only its original biology after confirmed source abort. Full schema,
permission, backup/DR and historical regression remain required. These tests do
not qualify paired commit, destination activation or linked ecological movement.
Tests defend contracts, not directory coverage. All checks remain active in
Release builds. Current proof has no actual network, biology or distributed
recovery, so those categories below are milestone gates rather than passing
+48
View File
@@ -1,5 +1,53 @@
# Validation report
## 2026-10-18 - M12.4i durable read-only destination staging
ADR-0148/schema 48 retains a destination-authorized staging record and bounded
pending pointer at the exact captured head. The adapter verifies both cuts,
replays the trace and retains a source capsule plus verified destination ghost
in the destination namespace. The source owns all outgoing stock; neither head
moves. Source abort releases both pending pointers atomically, and exact stage
retries retain their original outcome. Both migration roles are exclusive.
Executed locally:
- Native MSVC Release **78/78 passed**, 12.62 s.
- Actual GCC 14 optional server/persistence targets rebuilt; final private
PostgreSQL population/staging suite passed, **11 s** (8 s test phase).
- Linux Clang 19 ASan/UBSan private population/staging suite passed, **33 s**
(30 s test phase), with no sanitizer reports.
- Explicit clang-tidy 19 on the adapter, owning host and PostgreSQL population
tests passed with no owned warnings. Repository clang-format 19 and both
changed shell scripts' syntax checks passed.
- Complete frozen-binary private PostgreSQL qualification passed with **exit 0**,
**501 s**: C++ integration/process/load/history (426 s), life reservations,
backup/restore, host-loss DR, upgrade, migration history/rollback, runtime
permission audit and checksum/future-version rejection. The detached runner
recorded its completion code.
Both representations use real funded player-origin wildlife and unchanged cuts.
Cases cover exact/conflicting retries, a moved destination, pending destination
intervals, source/destination role exclusivity, blocked publication/profile/life
reservations and stale destination authority. Replacement epochs retain staging.
Missing source capsules, missing and same-length corrupt ghosts fail strong
binding. An existing valid object with false destination biology fails complete
replay. Real lock-timeout ambiguity retains exact requests; SIGKILL before/after
stage commit leaves absence/a retained pause respectively. Independent database
connections race stage against source abort without stranded destination work.
The owning destination host pauses on unknown staging, survives drain/recreation
without adopting the ghost and resumes its identical original state after source
abort. The existing fixed freeze metric covers either role without new labels.
Limits: this is staging evidence, not a migration result or activation receipt.
Atomic paired publication, its common immutable outcome, commit/abort races and
recovery/activation remain open. Backup retains all immutable namespace objects;
existing backup/DR fixtures do not qualify cellular cloning or a new cellular
restore procedure. No elapsed-day, linked-geometry or multigeneration claim
follows. No live database, World0, deployment, dependency, biological rule, format
or golden changed. The earlier captured `Build_Errors.txt` and unrelated local
work remain untracked; remote CI confirmation of the previous runner repair
remains separate.
## 2026-10-18 - M12.4h durable migration source freeze and explicit abort
ADR-0147/schema 47 retains exact source-authorized preparation, both predecessor
@@ -42,4 +42,36 @@ abort_cellular_migration_freeze(Postgres&, RegionKey, const LeaseGrant&, const U
[[nodiscard]] std::expected<std::optional<CellularMigrationFreeze>, RecoveryFailure>
load_cellular_migration_freeze(Postgres&, const FilesystemObjectStore&, RegionKey,
const LeaseGrant&, std::chrono::milliseconds timeout);
struct CellularMigrationStagePrepared {
RegionKey key{}; // Destination region.
std::vector<std::uint8_t> trace;
VerifiedObject ghost;
};
struct CellularMigrationStage {
CellularMigrationFreeze freeze;
VerifiedObject ghost; // Read-only evidence, never an ownership receipt.
};
struct CellularMigrationStageFailure {
PublicationError kind{};
LeaseStoreFailure store{};
ObjectError object{};
CellularMigrationStagePrepared prepared;
};
// Replay the retained pair, retain a source capsule and verified destination
// ghost in the destination namespace, then freeze its unchanged exact head.
[[nodiscard]] std::expected<CellularMigrationStage, CellularMigrationStageFailure>
stage_cellular_migration(Postgres&, const FilesystemObjectStore& source_objects,
const FilesystemObjectStore& destination_objects,
const LeaseGrant& destination_lease, const CommittedCheckpoint& source,
const CommittedCheckpoint& destination,
const server::CellularMigrationTrace&, std::chrono::milliseconds timeout,
const PublishHooks& hooks = {});
[[nodiscard]] std::expected<CellularMigrationStage, CellularMigrationStageFailure>
retry_cellular_migration_stage(Postgres&, const LeaseGrant&, const CellularMigrationStagePrepared&,
std::chrono::milliseconds timeout);
// Fresh destination authority; complete local capsule and ghost replay.
[[nodiscard]] std::expected<std::optional<CellularMigrationStage>, RecoveryFailure>
load_cellular_migration_stage(Postgres&, const FilesystemObjectStore&, RegionKey, const LeaseGrant&,
std::chrono::milliseconds timeout);
} // namespace evo::persistence
@@ -100,6 +100,12 @@ class CellularRegionHost {
const CellularMigrationRequest&);
[[nodiscard]] std::expected<void, CellularPublishFailure>
abort_migration_freeze(const persistence::Uuid& request);
// Explicit destination staging. The ghost stays read-only until a future
// paired publication receipt; source abort releases this pause.
[[nodiscard]] std::expected<void, CellularPublishFailure>
stage_migration(const persistence::CommittedCheckpoint& source,
const persistence::FilesystemObjectStore& source_objects,
const CellularMigrationTrace&);
// Explicit no-tick fenced transition. Uncertain publication pauses the host.
[[nodiscard]] std::expected<void, CellularPublishFailure> set_population_mode(bool enabled);
[[nodiscard]] bool population_mode() const noexcept;
@@ -228,6 +234,7 @@ class CellularRegionHost {
bool migration_reload_{}, migration_frozen_{};
persistence::LeaseClock::time_point migration_poll_at_{};
std::optional<persistence::CellularMigrationPrepared> migration_request_;
std::optional<persistence::CellularMigrationStagePrepared> migration_stage_request_;
std::optional<persistence::Uuid> migration_abort_;
bool interval_reload_{};
std::optional<persistence::PopulationInterval> admitted_interval_;
@@ -0,0 +1,140 @@
-- ADR-0148: a staged destination is frozen evidence, never active ownership.
CREATE TABLE evolution.cellular_migration_stages (
request_id uuid PRIMARY KEY REFERENCES evolution.cellular_migration_preparations,
ghost_sha256 bytea NOT NULL CHECK (octet_length(ghost_sha256)=32 AND ghost_sha256<>decode(repeat('00',32),'hex')),
ghost_bytes evolution.u64 NOT NULL CHECK (ghost_bytes BETWEEN 1 AND 16781400),
holder uuid NOT NULL, epoch evolution.u64 NOT NULL CHECK (epoch>0)
);
CREATE TABLE evolution.cellular_migration_destination_pending (
world_id evolution.u64 NOT NULL, region_id evolution.u64 NOT NULL,
request_id uuid UNIQUE NOT NULL REFERENCES evolution.cellular_migration_stages,
PRIMARY KEY(world_id,region_id)
);
REVOKE ALL ON evolution.cellular_migration_stages,evolution.cellular_migration_destination_pending FROM PUBLIC;
CREATE TRIGGER cellular_migration_stages_immutable BEFORE UPDATE OR DELETE ON evolution.cellular_migration_stages
FOR EACH ROW EXECUTE FUNCTION evolution.refuse_history_rewrite();
CREATE TRIGGER cellular_migration_stages_no_truncate BEFORE TRUNCATE ON evolution.cellular_migration_stages
FOR EACH STATEMENT EXECUTE FUNCTION evolution.refuse_history_rewrite();
CREATE FUNCTION evolution.cellular_migration_stage_record(w evolution.u64,r evolution.u64,id uuid)
RETURNS TABLE(trace bytea,source_manifest bytea,destination_manifest bytea,aborted boolean,ghost_sha256 bytea,ghost_bytes evolution.u64)
LANGUAGE sql STABLE SET search_path = pg_catalog, pg_temp AS $$
SELECT p.trace,s.manifest,d.manifest,EXISTS(SELECT 1 FROM evolution.cellular_migration_aborts a WHERE a.request_id=p.request_id),
g.ghost_sha256,g.ghost_bytes
FROM evolution.cellular_migration_preparations p JOIN evolution.cellular_migration_stages g USING(request_id)
JOIN evolution.region_commits s ON s.commit_id=p.source_commit
JOIN evolution.region_commits d ON d.commit_id=p.destination_commit
WHERE p.world_id=w AND p.destination_region=r AND p.request_id=id
$$;
REVOKE ALL ON FUNCTION evolution.cellular_migration_stage_record(evolution.u64,evolution.u64,uuid) FROM PUBLIC;
CREATE FUNCTION evolution.stage_cellular_migration(w evolution.u64,r evolution.u64,holder uuid,epoch evolution.u64,t bytea,digest bytea,size evolution.u64)
RETURNS TABLE(trace bytea,source_manifest bytea,destination_manifest bytea,aborted boolean,ghost_sha256 bytea,ghost_bytes evolution.u64)
LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, pg_temp AS $$
DECLARE id uuid; p evolution.cellular_migration_preparations%ROWTYPE;
old evolution.cellular_migration_stages%ROWTYPE;
BEGIN
PERFORM evolution.check_cellular_migration_trace(t);
IF w IS NULL OR r IS NULL OR evolution.manifest_uint(t,16,8)<>w OR evolution.manifest_uint(t,136,8)<>r
OR digest IS NULL OR octet_length(digest)<>32 OR digest=decode(repeat('00',32),'hex')
OR size IS NULL OR size NOT BETWEEN 1 AND (CASE evolution.manifest_uint(t,12,4) WHEN 1 THEN 16777216 ELSE 16781400 END) THEN
RAISE EXCEPTION 'invalid destination stage' USING ERRCODE='22023';
END IF;
id := encode(substring(t FROM 49 FOR 16),'hex')::uuid;
SELECT * INTO p FROM evolution.cellular_migration_preparations x WHERE x.request_id=id;
IF NOT FOUND OR p.world_id<>w OR p.destination_region<>r OR p.trace<>t THEN
RAISE EXCEPTION 'stage differs from retained preparation' USING ERRCODE='EVL07';
END IF;
-- Shared order for future two-owner work. Source abort serializes on its row.
PERFORM 1 FROM evolution.region_leases x WHERE x.world_id=w AND x.region_id IN (p.source_region,r)
ORDER BY x.region_id FOR UPDATE;
PERFORM evolution.check_region_lease(w,r,holder,epoch);
SELECT * INTO old FROM evolution.cellular_migration_stages x WHERE x.request_id=id;
IF FOUND THEN
IF old.ghost_sha256<>digest OR old.ghost_bytes<>size THEN
RAISE EXCEPTION 'stage request reused' USING ERRCODE='EVL07';
END IF;
RETURN QUERY SELECT * FROM evolution.cellular_migration_stage_record(w,r,id);
RETURN;
END IF;
IF EXISTS(SELECT 1 FROM evolution.cellular_migration_aborts a WHERE a.request_id=id)
OR NOT EXISTS(SELECT 1 FROM evolution.cellular_migration_pending x WHERE x.world_id=w AND x.region_id=p.source_region AND x.request_id=id)
OR EXISTS(SELECT 1 FROM evolution.cellular_migration_pending x WHERE x.world_id=w AND x.region_id=r)
OR EXISTS(SELECT 1 FROM evolution.cellular_migration_destination_pending x WHERE x.world_id=w AND x.region_id=r)
OR EXISTS(SELECT 1 FROM evolution.population_interval_pending x WHERE x.world_id=w AND x.region_id=r)
OR EXISTS(SELECT 1 FROM evolution.cellular_mutation_reservations x WHERE x.world_id=w AND x.region_id=r
AND NOT EXISTS(SELECT 1 FROM evolution.cellular_mutation_action_publications a
WHERE a.world_id=x.world_id AND a.account_id=x.account_id AND a.action_id=x.action_id))
OR EXISTS(SELECT 1 FROM evolution.life_reservations x WHERE x.world_id=w AND x.region_id=r AND x.rejection=0 AND x.ended_commit IS NULL
AND NOT EXISTS(SELECT 1 FROM evolution.primitive_life_bindings b
WHERE b.world_id=x.world_id AND b.account_id=x.account_id AND b.request_id=x.request_id)) THEN
RAISE EXCEPTION 'migration endpoint has pending work or was aborted' USING ERRCODE='EVL18';
END IF;
IF NOT EXISTS(SELECT 1 FROM evolution.region_heads h WHERE h.world_id=w AND h.region_id=p.source_region AND h.commit_id=p.source_commit
AND h.digest=substring(t FROM 89 FOR 32) AND h.end_tick=evolution.manifest_uint(t,32,8))
OR NOT EXISTS(SELECT 1 FROM evolution.region_heads h WHERE h.world_id=w AND h.region_id=r AND h.commit_id=p.destination_commit
AND h.digest=substring(t FROM 161 FOR 32) AND h.end_tick=evolution.manifest_uint(t,32,8)) THEN
RAISE EXCEPTION 'staged migration head moved' USING ERRCODE='EVL03';
END IF;
INSERT INTO evolution.cellular_migration_stages VALUES(id,digest,size,holder,epoch);
INSERT INTO evolution.cellular_migration_destination_pending VALUES(w,r,id);
RETURN QUERY SELECT * FROM evolution.cellular_migration_stage_record(w,r,id);
END $$;
REVOKE ALL ON FUNCTION evolution.stage_cellular_migration(evolution.u64,evolution.u64,uuid,evolution.u64,bytea,bytea,evolution.u64) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION evolution.stage_cellular_migration(evolution.u64,evolution.u64,uuid,evolution.u64,bytea,bytea,evolution.u64) TO :"runtime_role";
CREATE FUNCTION evolution.load_cellular_migration_stage(w evolution.u64,r evolution.u64,holder uuid,epoch evolution.u64)
RETURNS TABLE(trace bytea,source_manifest bytea,destination_manifest bytea,aborted boolean,ghost_sha256 bytea,ghost_bytes evolution.u64)
LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, pg_temp AS $$
DECLARE id uuid; initial_id uuid; src evolution.u64;
BEGIN
SELECT x.request_id,p.source_region INTO id,src FROM evolution.cellular_migration_destination_pending x
JOIN evolution.cellular_migration_preparations p USING(request_id) WHERE x.world_id=w AND x.region_id=r;
IF FOUND THEN
initial_id := id;
PERFORM 1 FROM evolution.region_leases x WHERE x.world_id=w AND x.region_id IN (src,r) ORDER BY x.region_id FOR UPDATE;
END IF;
PERFORM evolution.check_region_lease(w,r,holder,epoch);
-- Abort may have won while we waited. Never turn its completed release into corruption.
SELECT x.request_id INTO id FROM evolution.cellular_migration_destination_pending x WHERE x.world_id=w AND x.region_id=r;
IF NOT FOUND THEN RETURN; END IF;
IF initial_id IS DISTINCT FROM id THEN
RAISE EXCEPTION 'destination workflow changed while waiting' USING ERRCODE='40001';
END IF;
IF NOT EXISTS(SELECT 1 FROM evolution.cellular_migration_preparations p JOIN evolution.region_heads h
ON h.world_id=p.world_id AND h.region_id=p.destination_region
WHERE p.request_id=id AND p.world_id=w AND p.destination_region=r AND h.commit_id=p.destination_commit
AND h.digest=substring(p.trace FROM 161 FOR 32) AND h.end_tick=evolution.manifest_uint(p.trace,32,8)
AND EXISTS(SELECT 1 FROM evolution.cellular_migration_pending x WHERE x.world_id=w AND x.region_id=p.source_region AND x.request_id=id)
AND NOT EXISTS(SELECT 1 FROM evolution.cellular_migration_aborts a WHERE a.request_id=id)) THEN
RAISE EXCEPTION 'migration destination freeze inconsistent' USING ERRCODE='XX000';
END IF;
RETURN QUERY SELECT * FROM evolution.cellular_migration_stage_record(w,r,id);
END $$;
REVOKE ALL ON FUNCTION evolution.load_cellular_migration_stage(evolution.u64,evolution.u64,uuid,evolution.u64) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION evolution.load_cellular_migration_stage(evolution.u64,evolution.u64,uuid,evolution.u64) TO :"runtime_role";
ALTER FUNCTION evolution.abort_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,uuid) RENAME TO abort_cellular_migration_freeze_v47;
REVOKE ALL ON FUNCTION evolution.abort_cellular_migration_freeze_v47(evolution.u64,evolution.u64,uuid,evolution.u64,uuid) FROM :"runtime_role",PUBLIC;
CREATE FUNCTION evolution.abort_cellular_migration_freeze(w evolution.u64,r evolution.u64,holder uuid,epoch evolution.u64,id uuid)
RETURNS TABLE(trace bytea,source_manifest bytea,destination_manifest bytea,aborted boolean)
LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, pg_temp AS $$
BEGIN
RETURN QUERY SELECT * FROM evolution.abort_cellular_migration_freeze_v47(w,r,holder,epoch,id);
DELETE FROM evolution.cellular_migration_destination_pending x WHERE x.world_id=w AND x.request_id=id;
END $$;
REVOKE ALL ON FUNCTION evolution.abort_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,uuid) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION evolution.abort_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,uuid) TO :"runtime_role";
CREATE OR REPLACE FUNCTION evolution.refuse_frozen_cellular_work() RETURNS trigger
LANGUAGE plpgsql SET search_path = pg_catalog, pg_temp AS $$
BEGIN
IF EXISTS(SELECT 1 FROM evolution.cellular_migration_pending p WHERE p.world_id=NEW.world_id AND p.region_id=NEW.region_id)
OR EXISTS(SELECT 1 FROM evolution.cellular_migration_destination_pending p WHERE p.world_id=NEW.world_id AND p.region_id=NEW.region_id) THEN
RAISE EXCEPTION 'cellular endpoint frozen for migration' USING ERRCODE='EVL18';
END IF;
RETURN NEW;
END $$;
CREATE TRIGGER source_migration_endpoint_exclusive BEFORE INSERT ON evolution.cellular_migration_pending
FOR EACH ROW EXECUTE FUNCTION evolution.refuse_frozen_cellular_work();
INSERT INTO evolution.schema_migrations(version,sha256) VALUES(48,:'migration_sha_48');
+14 -3
View File
@@ -123,6 +123,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
SELECT :'history_v44' || ',45:' || :'migration_sha_45' AS history_v45 \gset
SELECT :'history_v45' || ',46:' || :'migration_sha_46' AS history_v46 \gset
SELECT :'history_v46' || ',47:' || :'migration_sha_47' AS history_v47 \gset
SELECT :'history_v47' || ',48:' || :'migration_sha_48' AS history_v48 \gset
SELECT :'history' = :'history_v22' AS at_v22,
:'history' = :'history_v23' AS at_v23,
:'history' = :'history_v24' AS at_v24,
@@ -148,8 +149,10 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
:'history' = :'history_v44' AS at_v44,
:'history' = :'history_v45' AS at_v45,
:'history' = :'history_v46' AS at_v46,
:'history' = :'history_v47' AS at_v47 \gset
\if :at_v47
:'history' = :'history_v47' AS at_v47,
:'history' = :'history_v48' AS at_v48 \gset
\if :at_v48
\elif :at_v47
\elif :at_v46
\elif :at_v45
\elif :at_v44
@@ -537,7 +540,8 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24,
coalesce(max(version), 0) = 44 AS at_v44,
coalesce(max(version), 0) = 45 AS at_v45,
coalesce(max(version), 0) = 46 AS at_v46,
coalesce(max(version), 0) = 47 AS at_v47
coalesce(max(version), 0) = 47 AS at_v47,
coalesce(max(version), 0) = 48 AS at_v48
FROM evolution.schema_migrations \gset
\if :at_v24
\ir 0025_cellular_v2_bundle.sql
@@ -753,6 +757,7 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24,
\elif :at_v45
\elif :at_v46
\elif :at_v47
\elif :at_v48
\else
\echo 'Unknown or changed migration history after v43'
DO $$ BEGIN RAISE EXCEPTION 'unknown or changed migration history'; END $$;
@@ -773,6 +778,10 @@ SELECT coalesce(max(version), 0) = 46 AS needs_v47 FROM evolution.schema_migrati
\if :needs_v47
\ir 0047_cellular_migration_freeze.sql
\endif
SELECT coalesce(max(version), 0) = 47 AS needs_v48 FROM evolution.schema_migrations \gset
\if :needs_v48
\ir 0048_cellular_migration_staging.sql
\endif
-- Effective-privilege audit on every run, including no-ops (ADR-0060). Attributes
-- alone are insufficient: predefined roles such as pg_write_all_data, PUBLIC
-- grants or SET-able memberships would bypass the fenced functions. Checks the
@@ -792,6 +801,8 @@ BEGIN
UNION
SELECT m.member FROM pg_catalog.pg_auth_members m JOIN closure c ON m.roleid = c.oid
), fenced(oid) AS (VALUES
('evolution.stage_cellular_migration(evolution.u64,evolution.u64,uuid,evolution.u64,bytea,bytea,evolution.u64)'::regprocedure::oid),
('evolution.load_cellular_migration_stage(evolution.u64,evolution.u64,uuid,evolution.u64)'::regprocedure::oid),
('evolution.prepare_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,bytea)'::regprocedure::oid),
('evolution.abort_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,uuid)'::regprocedure::oid),
('evolution.load_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64)'::regprocedure::oid),
@@ -59,6 +59,8 @@ abort. The current backup copies all immutable namespace objects and the shared
database snapshot; no GC or pruning is authorized. This does not qualify cellular
history cloning/restore beyond the existing supported fixtures.
Destination ghost staging, atomic two-head publication, commit/abort races and
activation remain subsequent work. This segment adds no automatic migration,
Schema 48/ADR-0148 now adds [destination staging](cellular-migration-stage-v1.md)
and atomic release of both pending pointers on source abort. Atomic two-head
publication, commit/abort races and activation remain subsequent work.
This segment adds no automatic migration,
atlas, cross-edge movement, environmental coupling or census/extinction rule.
@@ -0,0 +1,54 @@
# Biological destination staging
Authority: ADR-0148, SQL schema 48. This extends
[source preparation](cellular-migration-freeze-v1.md) with destination-authorized
read-only evidence. Biological/checkpoint/trace versions retain their meanings.
There is no new active checkpoint profile or migration result receipt.
The caller supplies the exact retained migration trace, both predecessor
manifests and current destination authority. The adapter verifies manifests and
payload SHA-256/length, replays the complete pair, copies the source predecessor
into the destination namespace and uploads/reads back the derived destination
candidate. Its content address and byte length form the exact retry input with
the canonical trace. Unpublished uploads remain retained orphans on failure.
SQL independently checks the retained preparation's exact trace, current
destination lease, both exact heads and source pending pointer. Both regional
lease rows are locked in increasing region order. Destination pending intervals,
unpublished mutation choices, successful outstanding life reservations and any
other local migration role block first staging. Moving the destination before
staging refuses the request; no implicit rebase is performed.
Schema 48 retains immutable `cellular_migration_stages`, keyed by the preparation
request UUID, with ghost SHA-256, byte length and staging holder/epoch. One
`cellular_migration_destination_pending` pointer per region bounds active lookup.
The source/destination pending roles are exclusive. Existing indexed publication
and interval/player-reservation guards cover either role.
The source owns all outgoing organisms and material until a future paired
commit. The staged candidate owns nothing. Neither region head changes. No tick,
feeding, passive healing, movement, birth or death is performed by staging. In
particular, a ghost cannot supply an owned census or terminal extinction count.
Destination recovery uses its own namespace's two predecessor capsules and ghost.
It rederives the whole pair and compares the complete destination bytes, including
membership, policy, biology and history. Missing or same-length corrupt objects
fail strong binding; a valid object with false biology fails semantic replay.
There is no fallback to the live source or an older head.
Both first staging and replacement pause host readiness/advancement. Unknown
replies retain the exact trace/content address; reconciliation keeps the same
request and polls paused work at most once per second. Normal stepping has no
per-tick migration query. The existing fixed unlabelled migration-freeze metric
covers either role; no request/organism labels are introduced.
Explicit current-source abort appends the existing immutable outcome and clears
both pointers in the same transaction. It serializes with staging on the source
lease row. If abort wins first, new staging is refused. If stage wins first,
abort releases both pauses. Exact retries of a retained stage after abort return
`aborted` and do not reopen either role. Neither expiry, drain nor timeout implies
abort. Keep staged objects/capsules and journals after abort; no GC is authorized.
Atomic paired result publication, its immutable common receipt, commit/abort
races, recovery/activation and linked ecological geometry remain subsequent
work. This segment enables no automatic movement, route, atlas or deployment.
@@ -252,4 +252,179 @@ load_cellular_migration_freeze(Postgres& db, const FilesystemObjectStore& object
return fail(RecoveryError::history);
return result;
}
namespace {
constexpr auto stage_columns = "encode(trace,'hex'),encode(source_manifest,'hex'),"
"encode(destination_manifest,'hex'),aborted::text,"
"encode(ghost_sha256,'hex'),ghost_bytes::text";
struct StageRecord {
Record cut;
VerifiedObject ghost;
};
std::optional<StageRecord> parse_stage(const PgRows::value_type& row) {
if (row.size() != 6)
return std::nullopt;
auto cut = parse(PgRows::value_type(row.begin(), row.begin() + 4));
const auto digest = unhex(row[4].value_or(""), 32);
const auto size = row[5].value_or("");
std::uint64_t bytes{};
const auto [end, error] = std::from_chars(size.data(), size.data() + size.size(), bytes);
if (!cut || !digest || digest->size() != 32 || error != std::errc{} ||
end != size.data() + size.size() || bytes == 0)
return std::nullopt;
VerifiedObject ghost{{}, bytes};
std::ranges::copy(*digest, ghost.sha256.begin());
return StageRecord{std::move(*cut), ghost};
}
std::optional<CellularMigrationStage> checked_stage(const StageRecord& record, RegionKey key,
const LeaseGrant& lease) {
const auto trace = server::decode_migration_trace(record.cut.trace);
if (!trace || key != RegionKey{trace->world, trace->request.destination.region})
return std::nullopt;
const auto freeze =
checked_record(record.cut, {trace->world, trace->request.source.region}, lease);
const auto destination = decode_manifest(record.cut.destination);
if (!freeze || !destination || record.ghost.sha256 == Sha256Digest{} ||
record.ghost.bytes > max_payload_bytes(destination->profile))
return std::nullopt;
return CellularMigrationStage{*freeze, record.ghost};
}
std::optional<std::vector<std::uint8_t>>
ghost_candidate(const CheckpointManifest& s, const CheckpointManifest& d,
std::span<const std::uint8_t> source, std::span<const std::uint8_t> destination,
const server::CellularMigrationTrace& trace) {
if (s.profile == CheckpointProfile::cellular_lives_v1) {
auto src = restore_cellular_payload(source, s),
dst = restore_cellular_payload(destination, d);
if (!src || !dst)
return std::nullopt;
auto pair = server::replay_migration_trace(**src, **dst, trace.request, trace);
if (!pair)
return std::nullopt;
return pair->destination->snapshot();
}
auto src = restore_population_payload(source, s),
dst = restore_population_payload(destination, d);
if (!src || !dst)
return std::nullopt;
auto pair = server::replay_migration_trace(**src, **dst, trace.request, trace);
if (!pair)
return std::nullopt;
return pair->destination->snapshot();
}
CellularMigrationStageFailure stage_failed(PublicationError kind, LeaseStoreFailure store = {},
ObjectError object = {},
CellularMigrationStagePrepared prepared = {}) {
return {kind, store, object, std::move(prepared)};
}
} // namespace
std::expected<CellularMigrationStage, CellularMigrationStageFailure>
retry_cellular_migration_stage(Postgres& db, const LeaseGrant& lease,
const CellularMigrationStagePrepared& prepared,
milliseconds timeout) {
const auto trace = server::decode_migration_trace(prepared.trace);
if (!trace || prepared.key != RegionKey{trace->world, trace->request.destination.region} ||
lease.frame != trace->frame || prepared.ghost.sha256 == Sha256Digest{} ||
prepared.ghost.bytes == 0)
return std::unexpected(stage_failed(PublicationError::candidate));
const std::array<std::string, 7> args{std::to_string(prepared.key.world),
std::to_string(prepared.key.region),
uuid(lease.holder),
std::to_string(lease.epoch),
"\\x" + hex(prepared.trace),
"\\x" + hex(prepared.ghost.sha256),
std::to_string(prepared.ghost.bytes)};
auto rows = db.query(std::string("SELECT ") + stage_columns +
" FROM evolution.stage_cellular_migration($1::evolution.u64,"
"$2::evolution.u64,$3::uuid,$4::evolution.u64,$5::bytea,$6::bytea,"
"$7::evolution.u64)",
args, timeout);
if (!rows) {
const auto store = failure(rows.error());
return std::unexpected(stage_failed(store.kind == LeaseStoreError::unknown
? PublicationError::unknown
: PublicationError::rejected,
store, {}, prepared));
}
const auto record = rows->size() == 1 ? parse_stage(rows->front()) : std::nullopt;
const auto result = record ? checked_stage(*record, prepared.key, lease) : std::nullopt;
if (!result || record->cut.trace != prepared.trace || result->ghost != prepared.ghost)
return std::unexpected(stage_failed(PublicationError::unknown, {}, {}, prepared));
return *result;
}
std::expected<CellularMigrationStage, CellularMigrationStageFailure>
stage_cellular_migration(Postgres& db, const FilesystemObjectStore& source_objects,
const FilesystemObjectStore& destination_objects, const LeaseGrant& lease,
const CommittedCheckpoint& source, const CommittedCheckpoint& destination,
const server::CellularMigrationTrace& trace, milliseconds timeout,
const PublishHooks& hooks) {
const auto s = decode_manifest(source.manifest), d = decode_manifest(destination.manifest);
if (!s || !d || !bound(source, *s) || !bound(destination, *d) ||
!cuts(*s, *d, source.head.digest, destination.head.digest, trace) ||
lease.frame != d->frame)
return std::unexpected(stage_failed(PublicationError::predecessor));
auto src = source_objects.get({s->payload_sha256, s->payload_bytes});
auto dst = destination_objects.get({d->payload_sha256, d->payload_bytes});
if (!src || !dst)
return std::unexpected(
stage_failed(PublicationError::predecessor, {}, !src ? src.error() : dst.error()));
const auto candidate = ghost_candidate(*s, *d, *src, *dst, trace);
const auto encoded = server::encode_migration_trace(trace);
if (!candidate || !encoded)
return std::unexpected(stage_failed(PublicationError::candidate));
auto capsule = destination_objects.put(*src);
if (!capsule)
return std::unexpected(stage_failed(PublicationError::upload, {}, capsule.error()));
auto capsule_bytes = destination_objects.get(*capsule);
if (!capsule_bytes || *capsule_bytes != *src ||
*capsule != VerifiedObject{s->payload_sha256, s->payload_bytes})
return std::unexpected(
stage_failed(PublicationError::verify, {},
capsule_bytes ? ObjectError::corrupt : capsule_bytes.error()));
auto ghost = destination_objects.put(*candidate);
if (!ghost)
return std::unexpected(stage_failed(PublicationError::upload, {}, ghost.error()));
auto readback = destination_objects.get(*ghost);
if (!readback || *readback != *candidate)
return std::unexpected(stage_failed(PublicationError::verify, {},
readback ? ObjectError::corrupt : readback.error()));
if (hooks.before_transaction)
hooks.before_transaction();
return retry_cellular_migration_stage(db, lease, {d->key, *encoded, *ghost}, timeout);
}
std::expected<std::optional<CellularMigrationStage>, RecoveryFailure>
load_cellular_migration_stage(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
const LeaseGrant& lease, milliseconds timeout) {
const auto fail = [](RecoveryError kind) { return std::unexpected(RecoveryFailure{kind, {}}); };
const std::array<std::string, 4> args{std::to_string(key.world), std::to_string(key.region),
uuid(lease.holder), std::to_string(lease.epoch)};
auto rows = db.query(std::string("SELECT ") + stage_columns +
" FROM evolution.load_cellular_migration_stage($1::evolution.u64,"
"$2::evolution.u64,$3::uuid,$4::evolution.u64)",
args, timeout);
if (!rows)
return std::unexpected(RecoveryFailure{RecoveryError::store, failure(rows.error())});
if (rows->empty())
return std::optional<CellularMigrationStage>{};
const auto record = rows->size() == 1 ? parse_stage(rows->front()) : std::nullopt;
auto result = record ? checked_stage(*record, key, lease) : std::nullopt;
if (!result || result->freeze.aborted)
return fail(RecoveryError::history);
const auto s = decode_manifest(record->cut.source),
d = decode_manifest(record->cut.destination);
if (!s || !d)
return fail(RecoveryError::manifest);
auto src = objects.get({s->payload_sha256, s->payload_bytes});
auto dst = objects.get({d->payload_sha256, d->payload_bytes});
auto ghost = objects.get(result->ghost);
if (!src || !dst || !ghost)
return fail(RecoveryError::tail);
const auto candidate = ghost_candidate(*s, *d, *src, *dst, result->freeze.trace);
if (!candidate || *candidate != *ghost)
return fail(RecoveryError::history);
return result;
}
} // namespace evo::persistence
+86 -3
View File
@@ -416,7 +416,8 @@ void CellularRegionHost::refresh() {
status_.ready = status_.phase == HostPhase::running && state_ && committed_ && grant &&
authority_.may_originate(LeaseClock::now()) && !interval_reload_ &&
!interval_request_.has_value() && !migration_reload_ && !migration_frozen_ &&
!migration_request_.has_value() && !migration_abort_.has_value();
!migration_request_.has_value() && !migration_stage_request_.has_value() &&
!migration_abort_.has_value();
status_.migration_frozen = migration_frozen_;
const auto& interval = admitted_interval_;
status_.ecological_pending_ticks =
@@ -430,7 +431,8 @@ void CellularRegionHost::refresh() {
}
void CellularRegionHost::reconcile_migration_freeze() {
if ((!migration_reload_ && !migration_frozen_ && !migration_request_ && !migration_abort_) ||
if ((!migration_reload_ && !migration_frozen_ && !migration_request_ &&
!migration_stage_request_ && !migration_abort_) ||
LeaseClock::now() < migration_poll_at_)
return;
const auto grant = authority_.grant();
@@ -460,6 +462,30 @@ void CellularRegionHost::reconcile_migration_freeze() {
}
migration_request_.reset();
}
if (migration_stage_request_) {
const auto staged = persistence::retry_cellular_migration_stage(
*db_, *grant, *migration_stage_request_, config_.lease.commit_timeout);
if (!staged) {
if (staged.error().kind == persistence::PublicationError::unknown) {
db_.reset();
++status_.unknown_outcomes;
return;
}
// A concurrent source abort can win before the uncertain stage.
// Reload authoritative pointers before making the region ready.
if (staged.error().store.kind != LeaseStoreError::commit_conflict) {
authority_.fence();
status_.phase = staged.error().store.kind == LeaseStoreError::fenced ||
staged.error().store.kind == LeaseStoreError::head_mismatch
? HostPhase::acquiring
: HostPhase::quarantined;
state_.reset();
committed_.reset();
return;
}
}
migration_stage_request_.reset();
}
if (migration_abort_) {
auto result = persistence::abort_cellular_migration_freeze(
*db_, config_.key, *grant, *migration_abort_, config_.lease.commit_timeout);
@@ -492,9 +518,65 @@ void CellularRegionHost::reconcile_migration_freeze() {
return;
}
migration_frozen_ = frozen->has_value();
if (!migration_frozen_) {
const auto staged = persistence::load_cellular_migration_stage(
*db_, *objects_, config_.key, *grant, config_.lease.commit_timeout);
if (!staged) {
migration_reload_ = true;
if (staged.error().kind == persistence::RecoveryError::store) {
db_.reset();
++status_.store_failures;
} else {
authority_.fence();
status_.phase = HostPhase::quarantined;
}
return;
}
migration_frozen_ = staged->has_value();
}
migration_reload_ = false;
}
std::expected<void, CellularPublishFailure>
CellularRegionHost::stage_migration(const persistence::CommittedCheckpoint& source,
const persistence::FilesystemObjectStore& source_objects,
const CellularMigrationTrace& trace) {
const auto grant = authority_.grant();
if (!grant.has_value() || !status_.ready || pending_ || !state_ || !committed_ ||
!queued_births_.empty() || !queued_mutations_.empty() || mutations_uncertain_ ||
!pending_endings_.empty() || interval_request_ || admitted_interval_ || interval_reload_ ||
!population_interval_safe(*state_) || !authority_.may_begin_commit(LeaseClock::now()) ||
!database())
return std::unexpected(CellularPublishFailure{CellularHostError::not_ready, {}, {}, {}});
migration_reload_ = true;
refresh();
const auto staged =
persistence::stage_cellular_migration(*db_, source_objects, *objects_, *grant, source,
*committed_, trace, config_.lease.commit_timeout);
if (!staged) {
if (staged.error().kind == persistence::PublicationError::unknown) {
migration_stage_request_ = staged.error().prepared;
db_.reset();
++status_.unknown_outcomes;
} else if (staged.error().store.kind == LeaseStoreError::fenced ||
staged.error().store.kind == LeaseStoreError::head_mismatch) {
authority_.fence();
state_.reset();
committed_.reset();
status_.phase = HostPhase::acquiring;
}
migration_poll_at_ = {};
refresh();
return std::unexpected(CellularPublishFailure{CellularHostError::publication,
staged.error().kind, staged.error().store,
staged.error().object});
}
migration_poll_at_ = {};
reconcile_migration_freeze();
refresh();
return {};
}
std::expected<void, CellularPublishFailure> CellularRegionHost::prepare_migration_freeze(
const persistence::CommittedCheckpoint& destination,
const persistence::FilesystemObjectStore& destination_objects,
@@ -573,7 +655,8 @@ const HostStatus& CellularRegionHost::step() {
acquire();
if (!pending_ && state_ && status_.phase == HostPhase::running)
reconcile_migration_freeze();
if (migration_reload_ || migration_frozen_ || migration_request_ || migration_abort_) {
if (migration_reload_ || migration_frozen_ || migration_request_ || migration_stage_request_ ||
migration_abort_) {
refresh();
return status_;
}
+319
View File
@@ -5,6 +5,7 @@
#include "evo/server/cellular_region_host.hpp"
#include "fixtures/cellular_ecology.hpp"
#include <barrier>
#include <csignal>
#include <cstdlib>
#include <filesystem>
@@ -868,6 +869,155 @@ void migration_freezes(Fixture& f) {
!take(retry_cellular_migration_freeze(*f.db, f.lease, prepared, 5s), "duplicate freeze")
.aborted,
"duplicate freeze lost its state");
const auto stage = [&](const CellularMigrationTrace& trace,
std::chrono::milliseconds timeout = 5s) {
return stage_cellular_migration(*f.db, *f.objects, *destination_objects,
destination_lease, f.head, destination_head, trace,
timeout);
};
// The optimistic destination cut must still be exact at first staging.
take(f.db->query("BEGIN", {}, 5s), "move destination before stage transaction");
if (representation == 0)
take(migrate_cellular_to_population(*f.db, *destination_objects, destination_lease,
destination_head, f.id(), f.build, {}, 5s),
"temporarily move destination profile");
else
take(materialize_population_profile(*f.db, *destination_objects, destination_lease,
destination_head, f.id(), f.build, 5s),
"temporarily move population destination");
const auto moved = stage(frozen.trace);
take(f.db->query("ROLLBACK", {}, 5s), "rollback moved destination fixture");
require(!moved && moved.error().store.kind == LeaseStoreError::head_mismatch,
"stage silently rebased a moved destination");
if (representation == 1) {
take(f.db->query("BEGIN", {}, 5s), "stage pending interval transaction");
take(admit_population_interval(*f.db, destination_key, destination_lease,
destination_head.head,
destination_head.head.end_tick + 20, 5s),
"destination admitted work before staging");
const auto pending = stage(frozen.trace);
take(f.db->query("ROLLBACK", {}, 5s), "rollback destination interval fixture");
require(!pending && pending.error().store.kind == LeaseStoreError::commit_conflict,
"stage stranded destination interval work");
}
const auto staged = take(stage(frozen.trace), "durable destination staging");
const CellularMigrationStagePrepared stage_request{destination_key, prepared.trace,
staged.ghost};
require(
!staged.freeze.aborted &&
take(take(load_cellular_migration_stage(*f.db, *destination_objects,
destination_key, destination_lease, 5s),
"recover destination ghost"),
"missing staged destination")
.ghost == staged.ghost &&
take(load_region_checkpoint(*f.db, destination_key, 5s),
"staged destination still owns original head")
.head == destination_before.head &&
take(retry_cellular_migration_stage(*f.db, destination_lease, stage_request, 5s),
"duplicate stage")
.ghost == staged.ghost,
"staging activated a ghost or lost exact replay");
auto changed_stage = stage_request;
++changed_stage.ghost.bytes;
const auto changed =
retry_cellular_migration_stage(*f.db, destination_lease, changed_stage, 5s);
require(!changed && changed.error().store.kind == LeaseStoreError::commit_conflict,
"stage accepted conflicting immutable ghost bytes");
auto reverse = frozen.trace;
reverse.request.request = f.id();
std::swap(reverse.request.source, reverse.request.destination);
const auto overlapping = retry_cellular_migration_freeze(
*f.db, destination_lease,
{destination_key, take(encode_migration_trace(reverse), "overlapping source trace")},
5s);
require(!overlapping && overlapping.error().store.kind == LeaseStoreError::commit_conflict,
"staged destination simultaneously prepared an outgoing workflow");
take(f.db->query("BEGIN", {}, 5s), "staged destination life reservation transaction");
Uuid new_account{};
new_account.back() = 5;
const auto staged_control =
take(claim_account_control(*f.db, destination_key, destination_lease, new_account, 1,
9950, 60000ms, 5s),
"staged destination reservation current control");
const auto staged_life =
reserve_primitive_life(*f.db, destination_key, destination_lease, new_account,
staged_control.generation, 1, 1, 5s);
take(f.db->query("ROLLBACK", {}, 5s), "rollback staged life reservation fixture");
require(!staged_life &&
std::string(staged_life.error().cause.sqlstate.data(), 5) == "EVL18",
"staged destination admitted a new player life reservation");
if (representation == 0) {
CellularBatch inputs{{f.key.world, 2, 1, destination_head.head.commit,
destination_head.head.digest, destination->tick(),
destination->tick() + 1},
{{}}};
const auto advanced = take(replay_cellular_batch(*destination, inputs),
"detached staged destination future");
const auto denied =
publish_cellular_batch(*f.db, *destination_objects, destination_lease,
destination_head, inputs, *advanced, f.id(), f.build, 5s);
require(!denied &&
std::string(denied.error().store.cause.sqlstate.data(), 5) == "EVL18",
"staged destination published an ordinary future");
} else {
require(!admit_population_interval(*f.db, destination_key, destination_lease,
destination_head.head,
destination_head.head.end_tick + 20, 5s) &&
!materialize_population_profile(*f.db, *destination_objects,
destination_lease, destination_head, f.id(),
f.build, 5s),
"staged destination progressed or changed profile");
}
const auto ghost_path =
destination_objects->directory() / hexadecimal(staged.ghost.sha256, false);
const auto ghost_backup = ghost_path.string() + ".stage-test";
std::filesystem::rename(ghost_path, ghost_backup);
const auto missing_ghost = load_cellular_migration_stage(
*f.db, *destination_objects, destination_key, destination_lease, 5s);
std::filesystem::rename(ghost_backup, ghost_path);
require(!missing_ghost && missing_ghost.error().kind == RecoveryError::tail,
"missing ghost fell back to the active destination");
char ghost_first{};
{
std::fstream file(ghost_path, std::ios::binary | std::ios::in | std::ios::out);
file.get(ghost_first);
file.seekp(0);
file.put(static_cast<char>(static_cast<unsigned char>(ghost_first) ^ 1U));
file.flush();
require(file.good(), "same-length ghost corruption fixture");
}
const auto corrupt_ghost = load_cellular_migration_stage(
*f.db, *destination_objects, destination_key, destination_lease, 5s);
{
std::fstream file(ghost_path, std::ios::binary | std::ios::in | std::ios::out);
file.put(ghost_first);
file.flush();
require(file.good(), "restore corrupt ghost fixture");
}
require(!corrupt_ghost && corrupt_ghost.error().kind == RecoveryError::tail,
"same-length corrupt ghost bypassed digest verification");
const auto source_manifest = take(decode_manifest(f.head.manifest), "stage source capsule");
const auto source_capsule_path =
destination_objects->directory() / hexadecimal(source_manifest.payload_sha256, false);
const auto source_capsule_backup = source_capsule_path.string() + ".stage-test";
std::filesystem::rename(source_capsule_path, source_capsule_backup);
const auto missing_source = load_cellular_migration_stage(
*f.db, *destination_objects, destination_key, destination_lease, 5s);
std::filesystem::rename(source_capsule_backup, source_capsule_path);
require(!missing_source && missing_source.error().kind == RecoveryError::tail,
"destination recovery used the live source instead of its retained capsule");
const auto old_destination_lease = destination_lease;
take(release_region_lease(*f.db, destination_key, destination_lease, 5s),
"release staged destination owner");
destination_lease = take(acquire_region_lease(*f.db, destination_key, f.build, 60000ms, 5s),
"replacement staged destination owner");
require(
take(load_cellular_migration_stage(*f.db, *destination_objects, destination_key,
destination_lease, 5s),
"new epoch ghost replay")
.has_value() &&
!retry_cellular_migration_stage(*f.db, old_destination_lease, stage_request, 5s),
"replacement activated the ghost or stale destination staged it");
auto conflict_trace = frozen.trace;
++conflict_trace.transferred.energy.value;
auto conflict = retry_cellular_migration_freeze(
@@ -960,6 +1110,15 @@ void migration_freezes(Fixture& f) {
"aborted pointer load")
.has_value(),
"abort was not durable or preparation retry reopened it");
require(
!take(load_cellular_migration_stage(*f.db, *destination_objects, destination_key,
destination_lease, 5s),
"aborted destination pointer")
.has_value() &&
take(retry_cellular_migration_stage(*f.db, destination_lease, stage_request, 5s),
"retry aborted stage")
.freeze.aborted,
"source abort stranded the destination or stage retry reopened its pointer");
require(take(load_region_checkpoint(*f.db, f.key, 5s), "aborted source head").head ==
source_before.head &&
f.raw->snapshot() == source_bytes,
@@ -1004,6 +1163,76 @@ void migration_freezes(Fixture& f) {
5s),
"abort reconciled freeze");
const auto false_stage_freeze = take(prepare(request_for()), "false ghost preparation");
const auto original_destination =
take(decode_manifest(destination_head.manifest), "false ghost existing object");
const CellularMigrationStagePrepared false_ghost{
destination_key,
take(encode_migration_trace(false_stage_freeze.trace), "false ghost trace"),
{original_destination.payload_sha256, original_destination.payload_bytes}};
take(retry_cellular_migration_stage(*f.db, destination_lease, false_ghost, 5s),
"metadata-only false ghost stage");
const auto false_ghost_replay = load_cellular_migration_stage(
*f.db, *destination_objects, destination_key, destination_lease, 5s);
require(!false_ghost_replay && false_ghost_replay.error().kind == RecoveryError::history,
"stage recovery accepted a real object with the wrong complete biology");
take(abort_cellular_migration_freeze(*f.db, f.key, f.lease,
false_stage_freeze.trace.request.request, 5s),
"abort false ghost evidence");
const auto uncertain_stage_freeze =
take(prepare(request_for()), "unknown stage preparation");
take(locker->query("BEGIN", {}, 5s), "stage ambiguity lock");
take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND "
"region_id=2 FOR UPDATE",
{}, 5s),
"hold stage reply");
auto stage_timed = connection();
const auto stage_unknown = stage_cellular_migration(
*stage_timed, *f.objects, *destination_objects, destination_lease, f.head,
destination_head, uncertain_stage_freeze.trace, 300ms);
require(!stage_unknown && stage_unknown.error().kind == PublicationError::unknown &&
!stage_unknown.error().prepared.trace.empty(),
"stage ambiguity lost exact bytes");
take(locker->query("ROLLBACK", {}, 5s), "release stage ambiguity");
take(retry_cellular_migration_stage(*f.db, destination_lease,
stage_unknown.error().prepared, 5s),
"resolve unknown stage");
take(abort_cellular_migration_freeze(*f.db, f.key, f.lease,
uncertain_stage_freeze.trace.request.request, 5s),
"abort reconciled stage");
for (unsigned race = 0; race < 4; ++race) {
const auto race_freeze = take(prepare(request_for()), "stage race preparation");
auto stage_db = connection(), abort_db = connection();
std::barrier start(2);
std::optional<std::expected<CellularMigrationStage, CellularMigrationStageFailure>>
staged_race;
std::optional<std::expected<CellularMigrationFreeze, LeaseStoreFailure>> aborted_race;
std::thread staging([&] {
start.arrive_and_wait();
staged_race = stage_cellular_migration(*stage_db, *f.objects, *destination_objects,
destination_lease, f.head, destination_head,
race_freeze.trace, 5s);
});
std::thread aborting([&] {
start.arrive_and_wait();
aborted_race = abort_cellular_migration_freeze(
*abort_db, f.key, f.lease, race_freeze.trace.request.request, 5s);
});
staging.join();
aborting.join();
require(aborted_race.has_value() && aborted_race->has_value() &&
staged_race.has_value() &&
(staged_race->has_value() ||
staged_race->error().store.kind == LeaseStoreError::commit_conflict) &&
!take(load_cellular_migration_stage(*f.db, *destination_objects,
destination_key, destination_lease, 5s),
"stage race final pointer")
.has_value(),
"stage/abort race left an active ghost or stranded pause");
}
// Both sides of preparation's transaction boundary, with real process death.
for (unsigned point = 0; point < 2; ++point) {
const auto killed_request = request_for();
@@ -1037,6 +1266,38 @@ void migration_freezes(Fixture& f) {
5s),
"abort killed preparation");
}
for (unsigned point = 0; point < 2; ++point) {
const auto killed = take(prepare(request_for()), "stage process-loss preparation");
const auto child = ::fork();
require(child >= 0, "stage publisher fork");
if (child == 0) {
try {
auto db = connection();
PublishHooks hooks;
if (point == 0)
hooks.before_transaction = [] { ::kill(::getpid(), SIGKILL); };
if (!stage_cellular_migration(*db, *f.objects, *destination_objects,
destination_lease, f.head, destination_head,
killed.trace, 5s, hooks))
::_exit(44);
::kill(::getpid(), SIGKILL);
} catch (...) {
::_exit(45);
}
::_exit(46);
}
int status{};
require(::waitpid(child, &status, 0) == child && WIFSIGNALED(status) &&
WTERMSIG(status) == SIGKILL &&
take(load_cellular_migration_stage(*f.db, *destination_objects,
destination_key, destination_lease, 5s),
"recover killed stage")
.has_value() == (point == 1),
"stage process loss changed ownership or lost the durable pause");
take(abort_cellular_migration_freeze(*f.db, f.key, f.lease,
killed.trace.request.request, 5s),
"abort killed stage");
}
}
// The owning host must retain the freeze through drain/recreation with entry off.
take(release_region_lease(*f.db, f.key, f.lease, 5s), "release fixture to frozen host");
@@ -1107,6 +1368,64 @@ void migration_freezes(Fixture& f) {
host->drain();
f.lease = take(acquire_region_lease(*f.db, f.key, f.build, 60000ms, 5s),
"return freeze fixture authority");
const auto destination_host_freeze =
take(prepare_cellular_migration_freeze(*f.db, *f.objects, *destination_objects, f.lease,
f.head, destination_head, request_for(), 5s),
"destination host source preparation");
take(release_region_lease(*f.db, destination_key, destination_lease, 5s),
"release destination to owning host");
const auto make_destination_host = [&] {
auto objects = take(FilesystemObjectStore::open(environment("PGHOST"), destination_key,
max_cellular_population_bytes),
"destination host objects");
CellularHostConfig config{destination_key, 1, {}, f.build, {f.build}};
config.lease.commit_timeout = 300ms;
return take(
CellularRegionHost::create(
config,
[] { return std::expected<std::unique_ptr<Postgres>, PgFailure>{connection()}; },
std::move(objects)),
"destination staging host");
};
auto destination_host = make_destination_host();
require(destination_host->step().ready, "destination host before staging");
const auto destination_bytes = destination_host->state()->snapshot();
take(locker->query("BEGIN", {}, 5s), "host stage ambiguity transaction");
take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND "
"region_id=1 FOR UPDATE",
{}, 5s),
"hold owning destination stage");
const auto host_stage_unknown =
destination_host->stage_migration(f.head, *f.objects, destination_host_freeze.trace);
require(!host_stage_unknown && !destination_host->status().ready &&
destination_host->state()->snapshot() == destination_bytes,
"owning destination advanced during ambiguous staging");
take(locker->query("ROLLBACK", {}, 5s), "release owning stage reply");
for (unsigned attempt = 0; attempt < 200 && !destination_host->status().migration_frozen;
++attempt) {
destination_host->step();
std::this_thread::sleep_for(25ms);
}
require(destination_host->status().migration_frozen && !destination_host->status().ready &&
!destination_host->advance_unattended(64),
"owning destination activated its ghost");
destination_host->drain();
destination_host = make_destination_host();
require(!destination_host->step().ready && destination_host->status().migration_frozen &&
destination_host->state()->snapshot() == destination_bytes &&
!destination_host->authority().has_value(),
"replacement destination lost staging or adopted the uncommitted ghost");
take(abort_cellular_migration_freeze(*f.db, f.key, f.lease,
destination_host_freeze.trace.request.request, 5s),
"source-authorized abort releases destination host");
for (unsigned attempt = 0; attempt < 200 && !destination_host->step().ready; ++attempt)
std::this_thread::sleep_for(25ms);
require(destination_host->status().ready && !destination_host->status().migration_frozen &&
destination_host->state()->snapshot() == destination_bytes,
"destination failed to resume its unchanged original state after source abort");
destination_host->drain();
destination_lease = take(acquire_region_lease(*f.db, destination_key, f.build, 60000ms, 5s),
"return staged destination fixture authority");
f.head = take(
materialize_population_profile(*f.db, *f.objects, f.lease, f.head, f.id(), f.build, 5s),
"resume frozen population as raw owner");
+2 -1
View File
@@ -213,6 +213,7 @@ void upgrade_across_migration_and_barrier() {
" -v migration_sha_45=$(s 0045_owned_population_intervals.sql)"
" -v migration_sha_46=$(s 0046_boundary_accounting_upgrade.sql)"
" -v migration_sha_47=$(s 0047_cellular_migration_freeze.sql)"
" -v migration_sha_48=$(s 0048_cellular_migration_staging.sql)"
" -v runtime_role=evo_runtime_test -f migrations/apply.sql"
" > /dev/null 2>&1") != 0);
CHECK(history(*owner_db) == "1,2,3,4,5,6,7");
@@ -230,7 +231,7 @@ void upgrade_across_migration_and_barrier() {
"tools/migrate-postgres.sh\" > /dev/null") == 0);
CHECK(history(*owner_db) ==
"1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,"
"25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47");
"25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48");
CHECK(scalar(*owner_db, "SELECT (c.condeferrable AND c.condeferred AND "
"c.confrelid = 'evolution.region_commits'::regclass)::text "
"FROM pg_catalog.pg_constraint c WHERE c.conrelid = "
+5 -5
View File
@@ -129,7 +129,7 @@ PGDATABASE=upgrade_path psql -X -v ON_ERROR_STOP=1 -1 -v "migration_sha=$v1_sha"
-f "$root/migrations/0001_region_bootstrap.sql" > /dev/null
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',') FROM evolution.schema_migrations")" = 1
PGDATABASE=upgrade_path bash "$root/tools/migrate-postgres.sh"
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48
# v2 heads have no payload binding: v2->v3 fails closed and leaves history at v2.
v2_sha="$(sha256sum "$root/migrations/0002_region_fencing.sql")"
v2_sha="${v2_sha%% *}"
@@ -150,7 +150,7 @@ PGDATABASE=v3_commits psql -X -v ON_ERROR_STOP=1 \
-c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \
-c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '01000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32), 'hex'))" > /dev/null
PGDATABASE=v3_commits bash "$root/tools/migrate-postgres.sh"
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 47:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 48:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) FROM evolution.region_commits")" = 1
# Review P1: a v4 database holding a published (possibly terminal) checkpoint has
# no durable decisions for it. The v5 upgrade must refuse before changing anything.
@@ -204,7 +204,7 @@ PGDATABASE=v6_populated psql -X -v ON_ERROR_STOP=1 \
-c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \
-c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '02000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32) || repeat('00', 80), 'hex'))" > /dev/null
PGDATABASE=v6_populated bash "$root/tools/migrate-postgres.sh"
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT count(*) FROM evolution.region_commits c, LATERAL evolution.parse_checkpoint_manifest(c.manifest) m WHERE m.commit_sequence = 0 AND m.predecessor_commit IS NULL")" = 1
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(conname, ',') FROM pg_constraint WHERE conrelid = 'evolution.region_heads'::regclass AND contype = 'c' AND pg_get_constraintdef(oid) LIKE '%end_tick%'")" = region_heads_published_tick
phase permission-audit
@@ -257,9 +257,9 @@ expect_audit_failure 'REVOKE EXECUTE ON FUNCTION evolution.claim_account_control
bash "$root/tools/migrate-postgres.sh" # Restored configuration passes the audit again.
phase checksum-drift
# Unknown future versions and checksum drift must not be repaired silently.
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=48 WHERE version=47'
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=49 WHERE version=48'
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=47 WHERE version=48"
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=48 WHERE version=49"
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=14"
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=13"
+3
View File
@@ -106,6 +106,8 @@ digest_46="$(sha256sum "$root/migrations/0046_boundary_accounting_upgrade.sql")"
digest_46="${digest_46%% *}"
digest_47="$(sha256sum "$root/migrations/0047_cellular_migration_freeze.sql")"
digest_47="${digest_47%% *}"
digest_48="$(sha256sum "$root/migrations/0048_cellular_migration_staging.sql")"
digest_48="${digest_48%% *}"
psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \
--set="migration_sha_2=$digest_2" --set="migration_sha_3=$digest_3" \
--set="migration_sha_4=$digest_4" --set="migration_sha_5=$digest_5" \
@@ -146,5 +148,6 @@ psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \
--set="migration_sha_45=$digest_45" \
--set="migration_sha_46=$digest_46" \
--set="migration_sha_47=$digest_47" \
--set="migration_sha_48=$digest_48" \
--set="runtime_role=$runtime_role" \
--file="$root/migrations/apply.sql"