diff --git a/README.md b/README.md index c6a0b95..bb650b1 100644 --- a/README.md +++ b/README.md @@ -173,8 +173,10 @@ retains both representations and their full recovery evidence. [Rules-9 census](docs/adr/0159-maturing-cellular-census.md) observes exact growing stock through verified history and registered snapshots; mixed rule cuts fail. [Portable rules-9 transfers](schemas/cellular-migration-trace-v2.md) retain exact -growing bodies under a distinct replay version. Durable rules-9 migration and -autonomous ecological qualification remain separate gates. +growing bodies under a distinct replay version. +[Schema 54](docs/adr/0161-durable-maturing-wildlife-transfer.md) admits those exact +pairs through fenced freeze/stage/paired publication and verified recovery/census. +Autonomous ecological and deployment qualification remain open. The microscope client is the M11 playtest path. `evo::simulation` remains graphics-, network- and database-independent. diff --git a/deploy/kubernetes/region/dev-session-route/listener.yaml b/deploy/kubernetes/region/dev-session-route/listener.yaml index 703a80e..1ace09c 100644 --- a/deploy/kubernetes/region/dev-session-route/listener.yaml +++ b/deploy/kubernetes/region/dev-session-route/listener.yaml @@ -12,7 +12,7 @@ spec: env: - {name: REGION_SESSION, value: "dev"} - {name: REGION_SESSION_LISTEN, value: "0.0.0.0:7400"} - - {name: REGION_SESSION_WINDOW, value: "dev-20261019T151436-42953:1792433676"} + - {name: REGION_SESSION_WINDOW, value: "dev-20261019T155548-13dfba50:1792439748"} - {name: REGION_M10_MAX_GRANT_LIFETIME, value: "14400"} ports: # QUIC (ADR-0090): UDP only; nothing listens on TCP 7400. diff --git a/docs/STATUS.md b/docs/STATUS.md index d551267..b9e6640 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,5 +1,25 @@ # Project status and next steps +## Durable migration of maturing wildlife - 2026-10-19 + +[ADR-0161](adr/0161-durable-maturing-wildlife-transfer.md)/schema 54 admits exact +bundle-11/trace-2 migration alongside retained bundle 10/trace 1. Source freeze, +read-only staging, atomic paired publication, complete recovery and registered +pending-handoff census verification reuse existing journals and fences. Mixed +bundles and cross-rule traces fail. Settled juveniles continue funded growth. + +Native Release passes **81/81**, both complete private rule matrices pass in +**35 s**, and ASan/UBSan passes in **86 s**. Analysis and formatting pass; evidence +and qualification limits are in [validation](engineering/VALIDATION.md). The full +private persistence/process/TLS/backup/DR/history/SQL qualification passes in +**613 s**, with frozen executable hashes unchanged. No live +upgrade, automatic movement or biological rule change. Linked resources, +autonomous sustainability, multi-host drills, actual days and product gates remain +open; M12 is still in progress. + +The same commit refreshes the DEV session window to a new four-hour absolute +deadline; rendered deployment policy passes for all 17 overlays. + ## Portable transfer of maturing wildlife - 2026-10-19 [ADR-0160](adr/0160-portable-maturing-wildlife-transfer.md) qualifies exact diff --git a/docs/adr/0160-portable-maturing-wildlife-transfer.md b/docs/adr/0160-portable-maturing-wildlife-transfer.md index d1e4180..7bdbbc7 100644 --- a/docs/adr/0160-portable-maturing-wildlife-transfer.md +++ b/docs/adr/0160-portable-maturing-wildlife-transfer.md @@ -52,3 +52,7 @@ process replay/continuation. Run Release, ASan/UBSan, analysis and affected priv durable regressions. This does not qualify sustainable generations, linked geometry/environmental budgets, cross-host partitions, actual days or M12 product readiness. + +ADR-0161/schema 54 subsequently admits exact bundle-11/trace-2 pairs through the +existing durable workflow and complete recovery/census verification. It preserves +the trace-version and equal-rule contracts above; no automatic route is added. diff --git a/docs/adr/0161-durable-maturing-wildlife-transfer.md b/docs/adr/0161-durable-maturing-wildlife-transfer.md new file mode 100644 index 0000000..1103703 --- /dev/null +++ b/docs/adr/0161-durable-maturing-wildlife-transfer.md @@ -0,0 +1,55 @@ +# ADR-0161: Durable migration of funded maturing wildlife + +## Status and scope + +Accepted for the scoped M12 durable maturation-transfer gate, 2026-10-19. +Extends ADR-0147–0149/0156/0158/0160. No biological rule, route, automatic upgrade, +default provisioning, new state table or live deployment is selected. + +## Decision + +Schema 54 admits precisely two migration pairs: equal bundle-10 predecessors with +trace 1 (rules 8), or equal bundle-11 predecessors with trace 2 (rules 9). Require +matching representations, world/frame/tick/quantum and compatible geometry, plus +the existing complete portable biological replay. Mixed bundles, cross-rule +traces and unknown versions fail. Existing retained bytes and migration checksums +remain exact; no SQL normalization of traces, manifests or digests is used. + +Reuse the existing immutable preparation, staging, abort and paired receipt +journals and pending pointers. Source preparation retains all stock and freezes +its exact head. Destination staging retains verified capsules and a read-only +ghost at its exact captured head. Ghosts and nonresident source history own no +additional stock. Both roles remain paused across lease expiry/replacement; +timeouts do not abort or activate anything. All work/choice/birth guards remain. + +The existing paired publisher requires both current fences, retained staging, +both exact heads and supported result manifests preserving each predecessor's +bundle. Publish both results, the common receipt and pointer settlement in one +transaction. Failed second publication rolls back the entire pair. Abort and +commit remain exclusive. Exact retries retain complete original bytes even under +replacement authority. No no-tick transfer runs maturation or re-derives bodies. + +Recovery and registered pending-handoff census verification select biology from +the retained bundle/trace pair, replay both complete candidates, bind both result +digests and compare full local bytes. Missing or false evidence fails the entire +operation/query without zero, partial or old-head fallback. Settled hosts reload +and verify their committed head before resuming. Subsequent ordinary raw/cohort +ticks continue funded growth under the retained rules and environment. + +## Alternatives, risks and qualification + +A new journal duplicates existing atomic authority. Reusing trace 1 changes a +retained biological contract. Publishing endpoints separately or activating a +ghost duplicates ownership. Extend the exact allowlist and retain the existing +workflow instead. This adds another replay/admission matrix and retained-object +cost; it does not remove the shared database's availability requirement. + +Require both representations and real player-origin juveniles, complete paired +stock/history equality, continued growth, freeze/staging recovery, retry/conflict, +pending-work/upgrade refusal, unknown replies, process kills, failed second +insert, abort/commit races, stale/replacement owners and registered census +capture races. Retain the rules-8 matrix and add rules-9 coverage, version-mismatch +SQL refusal and old-history replay. Run Release, sanitizers, analysis and the full +private persistence/backup/DR/checksum/privilege qualification. Multi-host network +and storage failure, linked environmental budgets, autonomous sustainability, +actual elapsed days and product acceptance remain separate M12 gates. diff --git a/docs/architecture/PERSISTENCE_ARCHITECTURE.md b/docs/architecture/PERSISTENCE_ARCHITECTURE.md index 2837e4b..02323f4 100644 --- a/docs/architecture/PERSISTENCE_ARCHITECTURE.md +++ b/docs/architecture/PERSISTENCE_ARCHITECTURE.md @@ -4,10 +4,13 @@ ADR-0158/schema 53 admits the explicit same-profile bundle-10 -> 11 maturation upgrade through the existing immutable version journal and lease/head fence. Recovery rederives the complete retained transformation; raw trace 3 and population batch 2 keep their layouts. Profile changes preserve the tuple. No new genesis, -automatic upgrade or default change. Durable biological migration for rules 9 -remains unsupported. [ADR-0160](../adr/0160-portable-maturing-wildlife-transfer.md) -adds portable equal-rules-9 transfer candidates and separately identified trace 2; -the durable adapter still admits only bundle 10/trace 1. +automatic upgrade or default change. [ADR-0160](../adr/0160-portable-maturing-wildlife-transfer.md) +adds portable equal-rules-9 transfer candidates and separately identified trace 2. +[ADR-0161](../adr/0161-durable-maturing-wildlife-transfer.md)/schema 54 admits exact +bundle 10/trace 1 or bundle 11/trace 2 pairs through the existing preparation, +staging, paired receipt, recovery and pending-handoff census workflow. All current +fences, head bindings, immutable evidence and complete replay remain required. +Mixed pairs and cross-rule traces fail; no new table or biological rule is added. [ADR-0159](../adr/0159-maturing-cellular-census.md) admits its exact retained stocks to census projection 1 and the existing verified history and registered snapshot readers. Mixed rules-8/rules-9 regional cuts fail the complete diff --git a/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md b/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md index 16c1a90..4d9a22f 100644 --- a/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md +++ b/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md @@ -50,6 +50,12 @@ food; raw/cohort biology and recorded continuation agree. Rules-8 trace 1 retain its bytes and meaning. Durable bundle-11/trace-2 admission, pending freeze/staging, atomic paired publication and complete recovery remain a separate gate. +ADR-0161/schema 54 subsequently admits bundle 11/trace 2 through that existing +durable workflow and complete registered pending-handoff verification. It preserves +the bundle-10/trace-1 path, exact biological state, ordered fences and atomic paired +receipt. Mixed bundles and cross-rule replay fail. It does not select automatic +movement, linked environmental budgets, sustainable ecology or deployment. + This is not the durable escrow gate. No prepared candidate may become an active destination owner before a future fenced transaction commits both heads and immutable transfer evidence. The source remains the sole owner until that commit; diff --git a/docs/engineering/M12_REMAINING_GATES.md b/docs/engineering/M12_REMAINING_GATES.md index 797bb1d..cdd08d2 100644 --- a/docs/engineering/M12_REMAINING_GATES.md +++ b/docs/engineering/M12_REMAINING_GATES.md @@ -132,6 +132,14 @@ retains rules 8. The durable adapter still rejects trace 2/bundle 11: explicit freeze/stage/paired publication/recovery and registered pending-handoff census qualification are required before rules-9 durable migration is supported. +ADR-0161/schema 54 now supplies that explicit durable admission through the +existing journals, pending pointers and two-head transaction. Exact bundle-10/ +trace-1 and bundle-11/trace-2 pairs retain their respective biology. Mixed bundles +and cross-rule traces fail. Both representations retain complete freeze/stage/ +publication/recovery and registered census verification. Multi-host network and +storage drills, linked resources, sustainable generations and actual-days/product +qualification remain open; no live/default upgrade or M12 completion is inferred. + M12.6a's [regional census projection](../../schemas/ecology/regional-cellular-census-v1.md) independently implements local current-genome and descendant-closure queries. It keeps suspended stock and retained zero-carrier ancestry, separates exact cohort diff --git a/docs/engineering/VALIDATION.md b/docs/engineering/VALIDATION.md index 57b5e91..da44866 100644 --- a/docs/engineering/VALIDATION.md +++ b/docs/engineering/VALIDATION.md @@ -1,5 +1,75 @@ # Validation report +## 2026-10-19 - Durable migration of maturing wildlife + +[ADR-0161](../adr/0161-durable-maturing-wildlife-transfer.md)/schema 54 admits +exact bundle-10/trace-1 and bundle-11/trace-2 migration pairs. Existing journals, +pending pointers, ordered fences and the atomic two-head transaction remain. +The retained bundle/trace pair selects replay; mixed rules and cross-rule traces +fail. There is no new state table, biological rule, default provisioning or live +upgrade. Existing numbered migrations and golden meanings remain unchanged. + +Executed locally: + +- MSVC default Release full CTest **81/81 passed, 17.34 s**, with NDEBUG. +- The complete affected private PostgreSQL population/history/transfer matrix + passed **exit 0, 35 s** (31 s population), retaining all rules-8 cases and + repeating the complete raw/cohort migration matrix in a separate rules-9 world. +- The same Clang 19 ASan/UBSan private matrix passed **exit 0, 86 s** (84 s + population), without sanitizer reports. +- The full frozen-executable private persistence/process/backup/DR and SQL suite + passed **exit 0, 613 s**: C++ integration **451 s**, loopback TLS migration faults + **78 s**, life reservations **4 s**, backup/restore **31 s**, DR **8 s**, upgrade + **10 s**, migration history **6 s**, effective runtime permissions **14 s**, + checksum/future-version refusal **8 s**, plus setup/migration. The saved runner + recorded zero explicitly; all frozen executable SHA-256 values remained exact. +- clang-tidy 19 passed on both changed persistence implementations, the migration + integration test and population publication matrix. Repository clang-format 19, + changed-shell `bash -n` and `git diff --check` passed. + +Both representations cover source freeze and staged ghost recovery, exact and +conflicting retries, pending choices/intervals, abort/commit exclusivity, failed +second publication, process loss, uncertain receipts, stale/replacement owners, +host adoption and missing/corrupt/false replay objects. Existing registered census +prepare/stage/abort/commit capture races and complete settled history checks now +also exercise rules 9. Ghosts add no stock; transferred nonresident history is +excluded from owned living inventory. A settled imported juvenile funds further +growth in a durably admitted 64-tick interval and recovers byte-identically. + +SQL independently refuses equal-tick mixed bundle-10/11 predecessors and trace 1 +against two bundle-11 cuts, leaving no pending freeze. Changing a retained request +to the other supported trace version conflicts rather than replacing its original +evidence. Normal C++ complete replay and manifest binding remain required; SQL +shape validation alone is not biological proof. + +The first rules-9 run caught a missing no-tick recovery dispatch: paired receipts +were tried only for bundle 10. Adding bundle 11 made complete host adoption and +the subsequent full affected matrix pass; no evidence fallback was introduced. +An all-target build was started before the focused run finished. Both completed; +the later full qualification uses a separately frozen executable directory and +saved runner with an explicit completion status, preventing subsequent relinks. + +Schema 54 replaces three existing functions in place, preserving the fenced +runtime surface. Its checker accepts matching magic/version 1 or 2 with unchanged +FNV/bounds; preparation and paired publication bind the corresponding exact +manifest bundle. No committed trace, manifest, digest or retry byte is normalized. +The legacy pre-context backup fixture restores the actual three earlier function +definitions before creating its old-schema backup, in addition to undoing schema +53 wrappers. This preserves the meaning of the legacy restore qualification. + +The requested DEV session window is refreshed in the same scoped commit after +qualification, with a unique UTC activation and absolute expiry four hours ahead. +Rendered deployment policy passes **75 objects / 74 files, 138 rendered objects / +17 overlays, 0 failures** with the final window. The unchanged policy self-test +logic was not rerun; all 46 cases passed in the preceding gate. This is desired +Git configuration, not a live connectivity or Argo observation. + +No live database, deployment or World0 operation occurred. Local private database, +filesystem and loopback TLS evidence does not qualify multi-host/kernel-network +partitions, remote storage failure, sustainable generations, linked environmental +budgets, target-host cost, actual elapsed days or M12 product acceptance. TSan was +not rerun because shared-memory ownership/scheduling did not change. + ## 2026-10-19 - Portable transfer of maturing wildlife [ADR-0160](../adr/0160-portable-maturing-wildlife-transfer.md) admits exact diff --git a/include/evo/persistence/cellular_migration_store.hpp b/include/evo/persistence/cellular_migration_store.hpp index 6381e9a..007bb7c 100644 --- a/include/evo/persistence/cellular_migration_store.hpp +++ b/include/evo/persistence/cellular_migration_store.hpp @@ -25,7 +25,7 @@ struct CellularMigrationFailure { // Verify both complete cuts and the paired biology, retain a verified destination // replay capsule in the source namespace, then freeze the exact source head. // Neither candidate is published or activated by this operation. -// Durable migration remains bundle 10/trace 1; portable trace 2 is refused. +// ADR-0161/schema 54: exact bundle 10/trace 1 or bundle 11/trace 2 pairs only. [[nodiscard]] std::expected prepare_cellular_migration_freeze(Postgres&, const FilesystemObjectStore& source_objects, const FilesystemObjectStore& destination_objects, diff --git a/include/evo/server/lifecycle/cellular_migration_trace.hpp b/include/evo/server/lifecycle/cellular_migration_trace.hpp index d6111b7..dc3cd11 100644 --- a/include/evo/server/lifecycle/cellular_migration_trace.hpp +++ b/include/evo/server/lifecycle/cellular_migration_trace.hpp @@ -42,7 +42,7 @@ struct RecordedPopulationMigration { // Portable recorded pair, not a receipt or ownership grant. The I/O adapter must // establish the trusted request against BOTH immutable predecessor manifests and // digest-checked payloads, then fence/commit both results before activation. -// Portable trace 2 does not extend the durable bundle-10/trace-1 allowlist. +// Durable admission is separate (ADR-0161/schema 54); a trace grants no authority. [[nodiscard]] std::expected prepare_migration_trace(const CellularLives&, const CellularLives&, const CellularMigrationRequest&); diff --git a/migrations/0054_maturing_cellular_migration.sql b/migrations/0054_maturing_cellular_migration.sql new file mode 100644 index 0000000..75d1c27 --- /dev/null +++ b/migrations/0054_maturing_cellular_migration.sql @@ -0,0 +1,195 @@ +-- ADR-0161: exact bundle 10/trace 1 and bundle 11/trace 2 migration pairs. +-- No new table or normalized retained bytes. Existing fenced transactions remain. +CREATE OR REPLACE FUNCTION evolution.check_cellular_migration_trace(t bytea) RETURNS void +LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp AS $$ +DECLARE n integer; members numeric; previous numeric := 0; life numeric; + hash numeric := 14695981039346656037; ids bytea[]; +BEGIN + IF t IS NULL OR octet_length(t) < 272 OR octet_length(t) > 2312 THEN + RAISE EXCEPTION 'invalid migration trace length' USING ERRCODE = '22023'; + END IF; + n := octet_length(t); members := evolution.manifest_uint(t,248,4); + ids := ARRAY[substring(t FROM 73 FOR 16),substring(t FROM 121 FOR 16), + substring(t FROM 145 FOR 16),substring(t FROM 193 FOR 16)]; + IF NOT ((substring(t FROM 1 FOR 8) = '\x4145564d49473031'::bytea AND evolution.manifest_uint(t,8,4) = 1) + OR (substring(t FROM 1 FOR 8) = '\x4145564d49473032'::bytea AND evolution.manifest_uint(t,8,4) = 2)) + OR evolution.manifest_uint(t,12,4) NOT IN (1,2) OR evolution.manifest_uint(t,16,8) = 0 + OR evolution.manifest_uint(t,24,8) = 0 OR evolution.manifest_uint(t,32,8) = 0 + OR evolution.manifest_uint(t,40,8) = 0 OR substring(t FROM 49 FOR 16) = decode(repeat('00',16),'hex') + OR evolution.manifest_uint(t,64,8) NOT BETWEEN 1 AND 2147483647 + OR evolution.manifest_uint(t,136,8) NOT BETWEEN 1 AND 2147483647 + OR evolution.manifest_uint(t,64,8) = evolution.manifest_uint(t,136,8) + OR substring(t FROM 89 FOR 32) = decode(repeat('00',32),'hex') + OR substring(t FROM 161 FOR 32) = decode(repeat('00',32),'hex') + OR substring(t FROM 209 FOR 40) = decode(repeat('00',40),'hex') + OR members NOT BETWEEN 1 AND 256 OR n <> 264 + 8 * members + OR evolution.manifest_uint(t,252,4) <> 0 + OR (SELECT count(DISTINCT x) FROM unnest(ids) x) <> 4 + OR decode(repeat('00',16),'hex') = ANY(ids) THEN + RAISE EXCEPTION 'invalid migration trace shape' USING ERRCODE = '22023'; + END IF; + FOR i IN 0..members::integer-1 LOOP + life := evolution.manifest_uint(t,256+8*i,8); + IF life <= previous OR life < 9223372036854775808 + OR trunc((life-9223372036854775808)/4294967296) = 0 OR mod(life,4294967296) = 0 THEN + RAISE EXCEPTION 'invalid migration member' USING ERRCODE = '22023'; + END IF; + previous := life; + END LOOP; + -- Exact bounded FNV-1a64, corruption detection only; manifests/payloads use SHA-256. + FOR i IN 0..n-9 LOOP + hash := mod((hash-mod(hash,256)+(mod(hash,256)::integer # get_byte(t,i)))*1099511628211, + 18446744073709551616); + END LOOP; + IF hash <> evolution.manifest_uint(t,n-8,8) THEN + RAISE EXCEPTION 'migration trace checksum differs' USING ERRCODE = '22023'; + END IF; +END $$; +REVOKE ALL ON FUNCTION evolution.check_cellular_migration_trace(bytea) FROM PUBLIC, :"runtime_role"; + +CREATE OR REPLACE FUNCTION evolution.prepare_cellular_migration_freeze(w evolution.u64,r evolution.u64,holder uuid,epoch evolution.u64,t bytea) +RETURNS TABLE(trace bytea,source_manifest bytea,destination_manifest bytea,aborted boolean) +LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, pg_temp AS $$ +DECLARE id uuid; src uuid; dst uuid; destination evolution.u64; h evolution.region_heads%ROWTYPE; + sm bytea; dm bytea; s evolution.checkpoint_manifest; d evolution.checkpoint_manifest; + old evolution.cellular_migration_preparations%ROWTYPE; + bundle bytea; + bundle10 constant bytea := '\x080000000700000002000000010000000100000004000000030000000200000001000000'; +BEGIN + PERFORM evolution.check_cellular_migration_trace(t); + bundle := CASE evolution.manifest_uint(t,8,4) WHEN 1 THEN bundle10 ELSE '\x090000000700000002000000010000000100000004000000030000000200000001000000'::bytea END; + IF evolution.manifest_uint(t,16,8) <> w OR evolution.manifest_uint(t,64,8) <> r THEN + RAISE EXCEPTION 'foreign migration source' USING ERRCODE = '22023'; + END IF; + id := encode(substring(t FROM 49 FOR 16),'hex')::uuid; + src := encode(substring(t FROM 73 FOR 16),'hex')::uuid; + dst := encode(substring(t FROM 145 FOR 16),'hex')::uuid; + destination := evolution.manifest_uint(t,136,8); + PERFORM evolution.check_region_lease(w,r,holder,epoch); + SELECT * INTO old FROM evolution.cellular_migration_preparations p WHERE p.request_id=id; + IF FOUND THEN + IF old.world_id<>w OR old.source_region<>r OR old.trace<>t THEN + RAISE EXCEPTION 'migration request reused' USING ERRCODE = 'EVL07'; + END IF; + RETURN QUERY SELECT * FROM evolution.cellular_migration_record(w,r,id); + RETURN; + END IF; + SELECT * INTO h FROM evolution.region_heads x WHERE x.world_id=w AND x.region_id=r FOR UPDATE; + IF NOT FOUND OR h.commit_id<>src OR h.digest<>substring(t FROM 89 FOR 32) + OR h.end_tick<>evolution.manifest_uint(t,32,8) THEN + RAISE EXCEPTION 'migration source moved' USING ERRCODE = 'EVL03'; + END IF; + SELECT c.manifest INTO sm FROM evolution.region_commits c WHERE c.commit_id=src; + -- Destination remains unowned by this workflow. A later stage must recheck its head. + SELECT c.manifest INTO dm FROM evolution.region_heads x JOIN evolution.region_commits c ON c.commit_id=x.commit_id + WHERE x.world_id=w AND x.region_id=destination AND x.commit_id=dst AND x.digest=substring(t FROM 161 FOR 32); + IF dm IS NULL THEN RAISE EXCEPTION 'migration destination moved' USING ERRCODE = 'EVL03'; END IF; + s := evolution.parse_checkpoint_manifest(sm); d := evolution.parse_checkpoint_manifest(dm); + IF s.profile NOT IN (4,5) OR d.profile<>s.profile OR s.versions<>bundle OR d.versions<>bundle + OR s.commit_sequence=0 OR d.commit_sequence=0 OR s.frame_id<>d.frame_id + OR s.frame_id<>evolution.manifest_uint(t,24,8) OR s.end_tick<>d.end_tick + OR s.tick_microseconds<>d.tick_microseconds + OR s.profile<>(CASE evolution.manifest_uint(t,12,4) WHEN 1 THEN 4 ELSE 5 END) THEN + RAISE EXCEPTION 'incompatible migration cuts' USING ERRCODE = 'EVL06'; + END IF; + IF EXISTS(SELECT 1 FROM evolution.cellular_migration_pending p WHERE p.world_id=w AND p.region_id=r) + OR EXISTS(SELECT 1 FROM evolution.population_interval_pending p WHERE p.world_id=w AND p.region_id=r) + OR EXISTS(SELECT 1 FROM evolution.cellular_mutation_reservations x WHERE x.world_id=w AND x.region_id=r + AND NOT EXISTS(SELECT 1 FROM evolution.cellular_mutation_action_publications a + WHERE a.world_id=x.world_id AND a.account_id=x.account_id AND a.action_id=x.action_id)) + OR EXISTS(SELECT 1 FROM evolution.life_reservations x WHERE x.world_id=w AND x.region_id=r AND x.rejection=0 AND x.ended_commit IS NULL + AND NOT EXISTS(SELECT 1 FROM evolution.primitive_life_bindings b + WHERE b.world_id=x.world_id AND b.account_id=x.account_id AND b.request_id=x.request_id)) THEN + RAISE EXCEPTION 'migration source has pending work' USING ERRCODE = 'EVL18'; + END IF; + INSERT INTO evolution.cellular_migration_preparations VALUES(id,w,r,destination,src,dst,t,holder,epoch); + INSERT INTO evolution.cellular_migration_pending VALUES(w,r,id); + RETURN QUERY SELECT * FROM evolution.cellular_migration_record(w,r,id); +END $$; +REVOKE ALL ON FUNCTION evolution.prepare_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,bytea) FROM PUBLIC; +GRANT EXECUTE ON FUNCTION evolution.prepare_cellular_migration_freeze(evolution.u64,evolution.u64,uuid,evolution.u64,bytea) TO :"runtime_role"; + +CREATE OR REPLACE FUNCTION evolution.publish_cellular_migration(id uuid,sm bytea,dm bytea,sh uuid,se evolution.u64,dh uuid,de evolution.u64) +RETURNS TABLE(trace bytea,source_predecessor bytea,destination_predecessor bytea,source_result bytea,destination_result bytea,ghost_sha256 bytea,ghost_bytes evolution.u64) +LANGUAGE plpgsql SECURITY DEFINER SET search_path=pg_catalog,pg_temp AS $$ +DECLARE p evolution.cellular_migration_preparations%ROWTYPE; g evolution.cellular_migration_stages%ROWTYPE; + old evolution.cellular_migration_commits%ROWTYPE; + s evolution.checkpoint_manifest := evolution.parse_checkpoint_manifest(sm); + d evolution.checkpoint_manifest := evolution.parse_checkpoint_manifest(dm); + predecessor evolution.checkpoint_manifest; h evolution.region_heads%ROWTYPE; + m bytea; c evolution.checkpoint_manifest; result_digest bytea; + bundle bytea := '\x080000000700000002000000010000000100000004000000030000000200000001000000'; + zero32 constant bytea := decode(repeat('00',32),'hex'); +BEGIN + SELECT * INTO p FROM evolution.cellular_migration_preparations x WHERE x.request_id=id; + IF NOT FOUND THEN RAISE EXCEPTION 'unknown migration preparation' USING ERRCODE='EVL04'; END IF; + PERFORM evolution.check_cellular_migration_trace(p.trace); + IF evolution.manifest_uint(p.trace,8,4)=2 THEN bundle := '\x090000000700000002000000010000000100000004000000030000000200000001000000'::bytea; END IF; + IF s.world_id<>p.world_id OR d.world_id<>p.world_id OR s.region_id<>p.source_region OR d.region_id<>p.destination_region + OR s.commit_id<>encode(substring(p.trace FROM 121 FOR 16),'hex')::uuid + OR d.commit_id<>encode(substring(p.trace FROM 193 FOR 16),'hex')::uuid THEN + RAISE EXCEPTION 'foreign migration results' USING ERRCODE='22023'; + END IF; + PERFORM 1 FROM evolution.region_leases x WHERE x.world_id=p.world_id AND x.region_id IN (p.source_region,p.destination_region) + ORDER BY x.region_id FOR UPDATE; + PERFORM evolution.check_region_lease(p.world_id,p.source_region,sh,se); + PERFORM evolution.check_region_lease(p.world_id,p.destination_region,dh,de); + SELECT * INTO old FROM evolution.cellular_migration_commits x WHERE x.request_id=id; + IF FOUND THEN + IF NOT EXISTS(SELECT 1 FROM evolution.region_commits x WHERE x.commit_id=old.source_result AND x.manifest=sm) + OR NOT EXISTS(SELECT 1 FROM evolution.region_commits x WHERE x.commit_id=old.destination_result AND x.manifest=dm) THEN + RAISE EXCEPTION 'migration commit request reused' USING ERRCODE='EVL07'; + END IF; + RETURN QUERY SELECT * FROM evolution.cellular_migration_commit_record(id); + RETURN; + END IF; + SELECT * INTO g FROM evolution.cellular_migration_stages x WHERE x.request_id=id; + IF NOT FOUND OR EXISTS(SELECT 1 FROM evolution.cellular_migration_aborts x WHERE x.request_id=id) + OR NOT EXISTS(SELECT 1 FROM evolution.cellular_migration_pending x WHERE x.world_id=p.world_id AND x.region_id=p.source_region AND x.request_id=id) + OR NOT EXISTS(SELECT 1 FROM evolution.cellular_migration_destination_pending x WHERE x.world_id=p.world_id AND x.region_id=p.destination_region AND x.request_id=id) THEN + RAISE EXCEPTION 'migration not staged or was aborted' USING ERRCODE='EVL18'; + END IF; + IF s.holder<>sh OR s.epoch<>se OR d.holder<>dh OR d.epoch<>de + OR d.payload_sha256<>g.ghost_sha256 OR d.payload_bytes<>g.ghost_bytes THEN + RAISE EXCEPTION 'migration writer or staged ghost differs' USING ERRCODE='EVL07'; + END IF; + FOREACH m IN ARRAY ARRAY[sm,dm] LOOP + c := evolution.parse_checkpoint_manifest(m); + SELECT * INTO h FROM evolution.region_heads x WHERE x.world_id=c.world_id AND x.region_id=c.region_id FOR UPDATE; + IF NOT FOUND OR h.commit_sequence=18446744073709551615 OR c.commit_sequence<>h.commit_sequence+1 + OR c.predecessor_commit<>h.commit_id OR c.predecessor_digest<>h.digest OR c.begin_tick<>h.end_tick + OR c.end_tick<>h.end_tick OR c.end_tick<>evolution.manifest_uint(p.trace,32,8) + OR h.commit_id<>(CASE WHEN c.region_id=p.source_region THEN p.source_commit ELSE p.destination_commit END) THEN + RAISE EXCEPTION 'migration endpoint head moved' USING ERRCODE='EVL03'; + END IF; + predecessor := evolution.parse_checkpoint_manifest((SELECT x.manifest FROM evolution.region_commits x WHERE x.commit_id=h.commit_id)); + IF c.profile NOT IN (4,5) OR c.profile<>predecessor.profile OR c.profile<>s.profile OR c.versions<>bundle + OR c.versions<>predecessor.versions OR c.frame_id<>predecessor.frame_id + OR c.tick_microseconds<>predecessor.tick_microseconds OR c.frame_id<>evolution.manifest_uint(p.trace,24,8) + OR c.frame_id<>(SELECT x.frame_id FROM evolution.region_leases x WHERE x.world_id=c.world_id AND x.region_id=c.region_id) + OR evolution.manifest_uint(m,228,4)<>0 + OR evolution.manifest_uint(m,232,4)<>(CASE c.profile WHEN 4 THEN 4 ELSE 5 END) + OR evolution.manifest_uint(m,236,4)<>1 OR c.trace_bytes<>0 OR c.trace_sha256<>zero32 + OR c.decision_count<>0 OR c.decisions_sha256<>zero32 + OR NOT EXISTS(SELECT 1 FROM evolution.retained_builds x WHERE x.build_identity=c.build_identity) + OR EXISTS(SELECT 1 FROM evolution.population_interval_pending x WHERE x.world_id=c.world_id AND x.region_id=c.region_id) THEN + RAISE EXCEPTION 'unsupported paired migration result' USING ERRCODE='EVL06'; + END IF; + END LOOP; + INSERT INTO evolution.cellular_migration_commits VALUES(id,s.commit_id,d.commit_id); + DELETE FROM evolution.cellular_migration_pending x WHERE x.request_id=id; + DELETE FROM evolution.cellular_migration_destination_pending x WHERE x.request_id=id; + FOREACH m IN ARRAY ARRAY[sm,dm] LOOP + c := evolution.parse_checkpoint_manifest(m); result_digest := sha256(m); + INSERT INTO evolution.region_commits(world_id,region_id,commit_sequence,commit_id,commit_digest,manifest, + begin_tick,end_tick,published_epoch,tick_microseconds,versions,build_identity,payload_bytes,payload_sha256) + VALUES(c.world_id,c.region_id,c.commit_sequence,c.commit_id,result_digest,m,c.begin_tick,c.end_tick,c.epoch, + c.tick_microseconds,c.versions,c.build_identity,c.payload_bytes,c.payload_sha256); + UPDATE evolution.region_heads x SET commit_sequence=c.commit_sequence,commit_id=c.commit_id,digest=result_digest, + end_tick=c.end_tick,published_epoch=c.epoch WHERE x.world_id=c.world_id AND x.region_id=c.region_id; + END LOOP; + RETURN QUERY SELECT * FROM evolution.cellular_migration_commit_record(id); +END $$; +REVOKE ALL ON FUNCTION evolution.publish_cellular_migration(uuid,bytea,bytea,uuid,evolution.u64,uuid,evolution.u64) FROM PUBLIC; +GRANT EXECUTE ON FUNCTION evolution.publish_cellular_migration(uuid,bytea,bytea,uuid,evolution.u64,uuid,evolution.u64) TO :"runtime_role"; +INSERT INTO evolution.schema_migrations(version,sha256) VALUES(54,:'migration_sha_54'); diff --git a/migrations/apply.sql b/migrations/apply.sql index fdbf357..0befc2e 100644 --- a/migrations/apply.sql +++ b/migrations/apply.sql @@ -129,6 +129,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution') SELECT :'history_v50' || ',51:' || :'migration_sha_51' AS history_v51 \gset SELECT :'history_v51' || ',52:' || :'migration_sha_52' AS history_v52 \gset SELECT :'history_v52' || ',53:' || :'migration_sha_53' AS history_v53 \gset + SELECT :'history_v53' || ',54:' || :'migration_sha_54' AS history_v54 \gset SELECT :'history' = :'history_v22' AS at_v22, :'history' = :'history_v23' AS at_v23, :'history' = :'history_v24' AS at_v24, @@ -160,8 +161,10 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution') :'history' = :'history_v50' AS at_v50, :'history' = :'history_v51' AS at_v51, :'history' = :'history_v52' AS at_v52, - :'history' = :'history_v53' AS at_v53 \gset - \if :at_v53 + :'history' = :'history_v53' AS at_v53, + :'history' = :'history_v54' AS at_v54 \gset + \if :at_v54 + \elif :at_v53 \elif :at_v52 \elif :at_v51 \elif :at_v50 @@ -561,7 +564,8 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24, coalesce(max(version), 0) = 50 AS at_v50, coalesce(max(version), 0) = 51 AS at_v51, coalesce(max(version), 0) = 52 AS at_v52, - coalesce(max(version), 0) = 53 AS at_v53 + coalesce(max(version), 0) = 53 AS at_v53, + coalesce(max(version), 0) = 54 AS at_v54 FROM evolution.schema_migrations \gset \if :at_v24 \ir 0025_cellular_v2_bundle.sql @@ -783,6 +787,7 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24, \elif :at_v51 \elif :at_v52 \elif :at_v53 +\elif :at_v54 \else \echo 'Unknown or changed migration history after v43' DO $$ BEGIN RAISE EXCEPTION 'unknown or changed migration history'; END $$; @@ -827,6 +832,10 @@ SELECT coalesce(max(version), 0) = 52 AS needs_v53 FROM evolution.schema_migrati \if :needs_v53 \ir 0053_funded_cellular_maturation.sql \endif +SELECT coalesce(max(version), 0) = 53 AS needs_v54 FROM evolution.schema_migrations \gset +\if :needs_v54 + \ir 0054_maturing_cellular_migration.sql +\endif -- Effective-privilege audit on every run, including no-ops (ADR-0060). Attributes -- alone are insufficient: predefined roles such as pg_write_all_data, PUBLIC -- grants or SET-able memberships would bypass the fenced functions. Checks the diff --git a/schemas/cellular-maturation-upgrade.md b/schemas/cellular-maturation-upgrade.md index 99a585e..69966f9 100644 --- a/schemas/cellular-maturation-upgrade.md +++ b/schemas/cellular-maturation-upgrade.md @@ -38,8 +38,9 @@ allows only the existing fenced entry points. Earlier migration bytes are intact No live/default auto-upgrade or new provisioning mode. An explicitly upgraded owner can recover and continue raw/cohort biology, including interval admission, settlement and materialization before returning-player operations. Biological -durable migration remains bundle 10 only. ADR-0160 qualifies portable equal-rules-9 -candidates/trace 2 without extending that admission. ADR-0159 admits rules-9 retained +ADR-0160 qualifies portable equal-rules-9 candidates/trace 2. ADR-0161/schema 54 +admits bundle-11/trace-2 durable migration through the existing fenced workflow, +without mixed-bundle migration or any automatic upgrade. ADR-0159 admits rules-9 retained body/reserve to census projection 1 and the verified history/registered snapshot readers. Mixed rule cuts or unavailable evidence prevent a complete result and cannot prove zero life or terminal extinction. Ecology/environment/actual-days/product diff --git a/schemas/cellular-migration-trace-v2.md b/schemas/cellular-migration-trace-v2.md index fea9825..2a40780 100644 --- a/schemas/cellular-migration-trace-v2.md +++ b/schemas/cellular-migration-trace-v2.md @@ -1,8 +1,10 @@ # Exact paired biological transfer trace 2 Authority: [ADR-0160](../docs/adr/0160-portable-maturing-wildlife-transfer.md). -Portable replay only. Durable freeze/stage/paired publication remains bundle -10/trace 1 and explicitly refuses this version; no ownership grant is added. +This trace alone grants no authority. Follow-on +[ADR-0161](../docs/adr/0161-durable-maturing-wildlife-transfer.md)/schema 54 admits +bundle-11/trace-2 pairs through the existing fenced freeze/stage/paired publication +workflow and complete recovery. Bundle 10 still requires trace 1; mixed pairs fail. Use the exact bounded [trace-1 layout](cellular-migration-trace-v1.md), with these two changes: @@ -33,6 +35,6 @@ Newly spent construction energy becomes heat under ADR-0157, not migration flow. Qualification includes real growth before and after transfer, raw/cohort exact biology, repeated returns, stock/census closure, placement/capacity/eligibility refusals, codec tampering and independent-process raw/cohort pair and interval -continuation. A later durable gate must explicitly admit trace 2 with bundle 11, -source freeze, staged ghost, atomic paired receipt, recovery and pending-handoff -census evidence. Nothing here enables routing or claims sustainable ecology. +continuation. Schema 54's admission additionally requires source freeze, staged +ghost, atomic paired receipt, recovery and pending-handoff census evidence. +Nothing here enables routing or claims sustainable ecology. diff --git a/schemas/ecology/cellular-migration-commit-v1.md b/schemas/ecology/cellular-migration-commit-v1.md index 7a84d65..da2962f 100644 --- a/schemas/ecology/cellular-migration-commit-v1.md +++ b/schemas/ecology/cellular-migration-commit-v1.md @@ -7,12 +7,18 @@ formats retain their meanings. The publisher restores both digest-verified predecessor cuts, replays the exact retained trace and uploads/reads back both complete candidates. Two same-profile -bundle-10 no-tick manifests each advance sequence once and bind their own -predecessor commit/SHA, world/region/frame, completed tick, quantum and history. +bundle-10 (trace 1) or bundle-11 (trace 2) no-tick manifests each advance sequence +once and bind their own predecessor commit/SHA, world/region/frame, completed tick, +quantum and history. Ordinary batch evidence is zero: migration is neither a biological interval nor a profile/version conversion. Result IDs come from the retained trace; writer fields name the respective current endpoint authorities. +ADR-0161/schema 54 requires the exact retained bundle/trace pair at both +predecessors/results. Rules 9 continues funded growth only on subsequent ticks; +the transfer itself preserves every life field. Mixed bundles or cross-rule +traces fail. Existing rules-8 evidence and all transaction guards retain meaning. + Exact retry input is request UUID plus both canonical manifests. Both CURRENT lease grants are supplied separately. SQL locks endpoint lease rows in increasing region order and checks both grants, captured heads, pending pointers, stage and diff --git a/schemas/ecology/cellular-migration-freeze-v1.md b/schemas/ecology/cellular-migration-freeze-v1.md index fc1b319..55fe429 100644 --- a/schemas/ecology/cellular-migration-freeze-v1.md +++ b/schemas/ecology/cellular-migration-freeze-v1.md @@ -8,6 +8,9 @@ One source-authorized request binds canonical [migration trace 1](../cellular-migration-trace-v1.md), both retained predecessor commits/manifests and their exact payloads. Both cuts use bundle 10, the same representation, world/frame/completed tick and compatible geometry/biology. +ADR-0161/schema 54 additionally admits equal bundle-11 cuts with trace 2 (rules 9). +Bundle 10/trace 1 retains rules 8; mixed tuples, cross-rule traces and unknown +versions fail. Both paths use the same journals and source/head/eligibility guards. The source lease/head is fenced; the destination head is checked optimistically and remains free to advance. Later staging must check it again. Preparation does not establish a simultaneous world cut or a physical route. diff --git a/schemas/ecology/census-pending-migrations-v1.md b/schemas/ecology/census-pending-migrations-v1.md index 0dcf0b6..b35766c 100644 --- a/schemas/ecology/census-pending-migrations-v1.md +++ b/schemas/ecology/census-pending-migrations-v1.md @@ -4,6 +4,11 @@ Semantic adapter contract, schema 52; [ADR-0156](../../docs/adr/0156-census-pending-migration-evidence.md). No biological codec/rule change or client endpoint. +ADR-0161/schema 54 admits both exact bundle-10/trace-1 and bundle-11/trace-2 +handoffs. Verification uses the retained pair, never the latest head's rule. +Mixed pairs, unknown versions and unavailable replay evidence fail the whole +registered query. Ghosts add no stock under either rule. + - Capture source/destination pending pointer union, history domain, complete region registry and exact current heads in one read-only repeatable-read view. - Inventory order is ascending request UUID. Maximum 64 requests; a 65th response diff --git a/src/persistence/cellular_checkpoint_store.cpp b/src/persistence/cellular_checkpoint_store.cpp index 1d388e7..8b3b7f0 100644 --- a/src/persistence/cellular_checkpoint_store.cpp +++ b/src/persistence/cellular_checkpoint_store.cpp @@ -576,7 +576,8 @@ verify_cellular_no_tick_checkpoint(Postgres& db, const FilesystemObjectStore& ob m.frame != committed.head.frame || lease.frame != m.frame || m.sequence == 0 || m.begin_tick != m.end_tick || m.batch != BatchDigests{}) return fail(RecoveryError::manifest); - if (m.cellular_versions == cellular_v10_versions) { + if (m.cellular_versions == cellular_v10_versions || + m.cellular_versions == cellular_v11_versions) { auto paired = verify_cellular_migration_checkpoint(db, objects, lease, committed, payload, timeout); if (!paired) diff --git a/src/persistence/cellular_migration_store.cpp b/src/persistence/cellular_migration_store.cpp index 5813434..25b0bf4 100644 --- a/src/persistence/cellular_migration_store.cpp +++ b/src/persistence/cellular_migration_store.cpp @@ -86,9 +86,10 @@ bool cuts(const CheckpointManifest& s, const CheckpointManifest& d, const auto profile = t.representation == server::CellularMigrationRepresentation::individual ? CheckpointProfile::cellular_lives_v1 : CheckpointProfile::cellular_population_v1; - return t.format_version == 1 && supported(s) && supported(d) && encode_manifest(s) && - encode_manifest(d) && s.cellular_versions == cellular_v10_versions && - d.cellular_versions == cellular_v10_versions && s.profile == profile && + const bool version = (t.format_version == 1 && s.cellular_versions == cellular_v10_versions) || + (t.format_version == 2 && s.cellular_versions == cellular_v11_versions); + return version && supported(s) && supported(d) && encode_manifest(s) && encode_manifest(d) && + d.cellular_versions == s.cellular_versions && s.profile == profile && d.profile == profile && s.sequence > 0 && d.sequence > 0 && s.key == RegionKey{t.world, t.request.source.region} && d.key == RegionKey{t.world, t.request.destination.region} && s.frame == t.frame && @@ -145,8 +146,7 @@ std::expected retry_cellular_migration_freeze(Postgres& db, const LeaseGrant& lease, const CellularMigrationPrepared& prepared, milliseconds timeout) { const auto trace = server::decode_migration_trace(prepared.trace); - if (!trace || trace->format_version != 1 || - prepared.key != RegionKey{trace->world, trace->request.source.region} || + if (!trace || prepared.key != RegionKey{trace->world, trace->request.source.region} || lease.frame != trace->frame) return std::unexpected(failed(PublicationError::candidate)); const std::array args{std::to_string(prepared.key.world), @@ -341,8 +341,7 @@ retry_cellular_migration_stage(Postgres& db, const LeaseGrant& lease, const CellularMigrationStagePrepared& prepared, milliseconds timeout) { const auto trace = server::decode_migration_trace(prepared.trace); - if (!trace || trace->format_version != 1 || - prepared.key != RegionKey{trace->world, trace->request.destination.region} || + if (!trace || prepared.key != RegionKey{trace->world, trace->request.destination.region} || lease.frame != trace->frame || prepared.ghost.sha256 == Sha256Digest{} || prepared.ghost.bytes == 0) return std::unexpected(stage_failed(PublicationError::candidate)); @@ -521,8 +520,11 @@ constexpr auto commit_columns = std::optional migration_manifest(const CheckpointManifest& p, const Sha256Digest& digest, const Uuid& commit, const LeaseGrant& lease, const Sha256Digest& build, VerifiedObject object) { - if (!supported(p) || p.cellular_versions != cellular_v10_versions || p.sequence == 0 || - p.sequence == std::numeric_limits::max() || lease.frame != p.frame) + if (!supported(p) || + (p.cellular_versions != cellular_v10_versions && + p.cellular_versions != cellular_v11_versions) || + p.sequence == 0 || p.sequence == std::numeric_limits::max() || + lease.frame != p.frame) return std::nullopt; auto m = p; m.commit = commit; diff --git a/tests/population_publication_tests.cpp b/tests/population_publication_tests.cpp index e0ff77c..635235d 100644 --- a/tests/population_publication_tests.cpp +++ b/tests/population_publication_tests.cpp @@ -2228,8 +2228,10 @@ void migration_publication(Fixture& f, const RegionKey destination_key, if (direction == 0) { auto locker = connection(true), timed = connection(); take(locker->query("BEGIN", {}, 5s), "pair ambiguity transaction"); - take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND " - "region_id=" + + take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=" + + std::to_string(f.key.world) + + " AND " + "region_id=" + std::to_string(sk.region) + " FOR UPDATE", {}, 5s), "hold paired reply"); @@ -2587,7 +2589,7 @@ void migration_freezes(Fixture& f) { take(FilesystemObjectStore::open(environment("PGHOST"), destination_key, max_cellular_population_bytes), "migration destination objects"); - auto destination = opening(30001, true, 4, false, 2); + auto destination = opening(f.key.world - 200, true, 4, false, 2); auto destination_head = take(provision_cellular_region(*f.owner, *destination_objects, destination_key, 1, *destination, f.id(), f.build, 5s), @@ -2625,6 +2627,43 @@ void migration_freezes(Fixture& f) { destination_head, f.id(), f.build, 5s), "destination accounting upgrade"); destination = take(destination->migration_upgrade_candidate(), "destination upgraded oracle"); + if (f.raw->rules_version() == cellular_lives_maturation_rules) { + const auto member = std::ranges::find_if( + f.raw->lives(), [](const auto& life) { return life.account == LifeAccount{}; }); + require(member != f.raw->lives().end(), "rules-9 migration lacks real wildlife"); + auto upgraded = take(destination->maturation_upgrade_candidate(), "maturing peer oracle"); + const auto mismatch = [&](std::uint32_t version) { + const CellularMigrationRequest request{ + f.id(), + {1, f.head.head.commit, f.id(), f.head.head.digest}, + {2, destination_head.head.commit, f.id(), destination_head.head.digest}, + member->identity.genome}; + auto recorded = take(prepare_migration_trace(*f.raw, *upgraded, request), + "mismatched rule fixture"); + recorded.trace.format_version = version; + const std::array args{ + std::to_string(f.key.world), hexadecimal(f.lease.holder, false), + std::to_string(f.lease.epoch), + hexadecimal( + take(encode_migration_trace(recorded.trace), "mismatched trace encoding"))}; + const auto refused = + f.db->query("SELECT * FROM evolution.prepare_cellular_migration_freeze(" + "$1::evolution.u64,1,$2::uuid,$3::evolution.u64,$4::bytea)", + args, 5s); + require(!refused && std::string(refused.error().sqlstate.data(), 5) == "EVL06" && + !take(load_cellular_migration_freeze(*f.db, *f.objects, f.key, f.lease, 5s), + "no partial mismatch freeze") + .has_value(), + "SQL accepted incompatible biological cuts"); + }; + mismatch(2); // Actual retained cuts are rules 9 and rules 8, at equal ticks. + destination_head = + take(migrate_cellular_maturation_profile(*f.db, *destination_objects, destination_lease, + destination_head, f.id(), f.build, 5s), + "destination maturation admission"); + mismatch(1); // Both actual predecessors now retain rules 9, not trace 1. + destination = std::move(upgraded); + } const std::array builds{f.build}; const auto request_for = [&] { const auto member = std::ranges::find_if( @@ -2689,36 +2728,40 @@ void migration_freezes(Fixture& f) { const CellularMigrationPrepared prepared{ f.key, take(encode_migration_trace(frozen.trace), "frozen canonical trace")}; auto unsupported = frozen.trace; - unsupported.format_version = 2; + unsupported.format_version = frozen.trace.format_version == 1 ? 2 : 1; const auto unsupported_bytes = take(encode_migration_trace(unsupported), "portable rules-9 trace"); const auto refused_freeze = retry_cellular_migration_freeze(*f.db, f.lease, {f.key, unsupported_bytes}, 5s); const auto refused_stage = retry_cellular_migration_stage( *f.db, destination_lease, {destination_key, unsupported_bytes, {f.build, 1}}, 5s); - require(!refused_freeze && refused_freeze.error().kind == PublicationError::candidate && - !refused_stage && refused_stage.error().kind == PublicationError::candidate && + require(!refused_freeze && refused_freeze.error().kind == PublicationError::rejected && + !refused_stage && refused_stage.error().kind == PublicationError::rejected && !stage_cellular_migration(*f.db, *f.objects, *destination_objects, destination_lease, f.head, destination_head, unsupported, 5s), - "portable rules-9 trace expanded durable migration admission"); + "cross-rule trace replaced retained durable migration evidence"); const std::array unsupported_args{hexadecimal(f.lease.holder, false), std::to_string(f.lease.epoch), hexadecimal(unsupported_bytes)}; const auto refused_sql = - f.db->query("SELECT * FROM evolution.prepare_cellular_migration_freeze(30201," - "1,$1::uuid,$2::evolution.u64,$3::bytea)", + f.db->query("SELECT * FROM evolution.prepare_cellular_migration_freeze(" + + std::to_string(f.key.world) + + "," + "1,$1::uuid,$2::evolution.u64,$3::bytea)", unsupported_args, 5s); - require(!refused_sql && std::string(refused_sql.error().sqlstate.data(), 5) == "22023", - "SQL admitted portable trace 2 before duplicate lookup"); + require(!refused_sql && std::string(refused_sql.error().sqlstate.data(), 5) == "EVL07", + "SQL admitted a cross-rule trace under an existing request"); auto malformed = prepared.trace; malformed.back() ^= 1; const std::array malformed_args{hexadecimal(f.lease.holder, false), std::to_string(f.lease.epoch), hexadecimal(malformed)}; const auto malformed_sql = f.db->query("SELECT * FROM " - "evolution.prepare_cellular_migration_freeze(30201," - "1,$1::uuid,$2::evolution.u64,$3::bytea)", + "evolution.prepare_cellular_migration_freeze(" + + std::to_string(f.key.world) + + "," + "1,$1::uuid,$2::evolution.u64,$3::bytea)", malformed_args, 5s); require(!malformed_sql && std::string(malformed_sql.error().sqlstate.data(), 5) == "22023", "SQL accepted an invalid trace checksum before duplicate lookup"); @@ -2929,18 +2972,21 @@ void migration_freezes(Fixture& f) { } const auto capsule_manifest = take(decode_manifest(destination_head.manifest), "capsule manifest"); - const auto denied_upgrade_id = f.id(); - const auto denied_upgrade = migrate_cellular_maturation_profile( - *f.db, *f.objects, f.lease, f.head, denied_upgrade_id, f.build, 5s); - require( - !denied_upgrade && - std::string(denied_upgrade.error().store.cause.sqlstate.data(), 5) == "EVL18" && - !take(load_cellular_profile_migration(*f.db, f.key, f.lease, denied_upgrade_id, 5s), - "frozen maturation journal") - .has_value() && - take(load_region_checkpoint(*f.db, f.key, 5s), "frozen maturation head").head == - f.head.head, - "maturation escaped a frozen handoff or retained partial upgrade evidence"); + if (f.raw->rules_version() == cellular_lives_foraging_rules) { + const auto denied_upgrade_id = f.id(); + const auto denied_upgrade = migrate_cellular_maturation_profile( + *f.db, *f.objects, f.lease, f.head, denied_upgrade_id, f.build, 5s); + require( + !denied_upgrade && + std::string(denied_upgrade.error().store.cause.sqlstate.data(), 5) == "EVL18" && + !take(load_cellular_profile_migration(*f.db, f.key, f.lease, denied_upgrade_id, + 5s), + "frozen maturation journal") + .has_value() && + take(load_region_checkpoint(*f.db, f.key, 5s), "frozen maturation head").head == + f.head.head, + "maturation escaped a frozen handoff or retained partial upgrade evidence"); + } const auto capsule_path = f.objects->directory() / hexadecimal(capsule_manifest.payload_sha256, false); const auto backup = capsule_path.string() + ".freeze-test"; @@ -3038,8 +3084,10 @@ void migration_freezes(Fixture& f) { // A genuine unknown reply retains exactly the chosen cut and group. auto locker = connection(true); take(locker->query("BEGIN", {}, 5s), "freeze ambiguity lock"); - take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND " - "region_id=1 FOR UPDATE", + take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=" + + std::to_string(f.key.world) + + " AND " + "region_id=1 FOR UPDATE", {}, 5s), "hold freeze reply"); auto timed = connection(); @@ -3077,8 +3125,10 @@ void migration_freezes(Fixture& f) { const auto uncertain_stage_freeze = take(prepare(request_for()), "unknown stage preparation"); take(locker->query("BEGIN", {}, 5s), "stage ambiguity lock"); - take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND " - "region_id=2 FOR UPDATE", + take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=" + + std::to_string(f.key.world) + + " AND " + "region_id=2 FOR UPDATE", {}, 5s), "hold stage reply"); auto stage_timed = connection(); @@ -3228,11 +3278,10 @@ void migration_freezes(Fixture& f) { "host failed to resume the identical source after confirmed abort"); auto locker = connection(true); take(locker->query("BEGIN", {}, 5s), "host unknown freeze lock"); - take( - locker->query( - "SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND region_id=1 FOR UPDATE", - {}, 5s), - "hold host freeze reply"); + take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=" + + std::to_string(f.key.world) + " AND region_id=1 FOR UPDATE", + {}, 5s), + "hold host freeze reply"); const auto uncertain_request = request_for(); const auto uncertain = host->prepare_migration_freeze(destination_head, *destination_objects, uncertain_request); @@ -3246,11 +3295,10 @@ void migration_freezes(Fixture& f) { host->state()->snapshot() == before, "host chose different biology after unknown preparation"); take(locker->query("BEGIN", {}, 5s), "host unknown abort lock"); - take( - locker->query( - "SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND region_id=1 FOR UPDATE", - {}, 5s), - "hold host abort reply"); + take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=" + + std::to_string(f.key.world) + " AND region_id=1 FOR UPDATE", + {}, 5s), + "hold host abort reply"); const auto unknown_abort = host->abort_migration_freeze(uncertain_request.request); require(!unknown_abort && !host->status().ready, "unconfirmed abort resumed the source"); take(locker->query("ROLLBACK", {}, 5s), "release host abort reply"); @@ -3285,9 +3333,11 @@ void migration_freezes(Fixture& f) { require(destination_host->step().ready, "destination host before staging"); const auto destination_bytes = destination_host->state()->snapshot(); take(locker->query("BEGIN", {}, 5s), "host stage ambiguity transaction"); - take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 AND " - "region_id=1 FOR UPDATE", - {}, 5s), + take(locker->query( + "SELECT 1 FROM evolution.region_leases WHERE world_id=" + std::to_string(f.key.world) + + " AND " + "region_id=1 FOR UPDATE", + {}, 5s), "hold owning destination stage"); const auto host_stage_unknown = destination_host->stage_migration(f.head, *f.objects, destination_host_freeze.trace); @@ -3496,5 +3546,51 @@ void run_population_publication_tests() { boundary_upgrades(fixture, true); host_ownership_and_admission(fixture); maturing_census(); + Fixture maturing; + maturing.key = {30203, 1}; + maturing.next_id = 45000; + funded_source(maturing); + maturing.head = + take(migrate_cellular_boundary_profile(*maturing.db, *maturing.objects, maturing.lease, + maturing.head, maturing.id(), maturing.build, 5s), + "migration rules-9 prerequisite"); + maturing.head = + take(migrate_cellular_maturation_profile(*maturing.db, *maturing.objects, maturing.lease, + maturing.head, maturing.id(), maturing.build, 5s), + "migration rules-9 admission"); + auto source = take(recover_cellular_checkpoint(*maturing.db, *maturing.objects, maturing.key, + maturing.lease, + std::array{maturing.build}, 5s), + "migrating maturing source"); + maturing.raw = std::move(source.state); + migration_freezes(maturing); + require(maturing.raw->boundary_ledger().incoming != accounting::Inventory{}, + "rules-9 fixture did not retain a settled migration"); + maturing.head = take(migrate_cellular_to_population( + *maturing.db, *maturing.objects, maturing.lease, maturing.head, + maturing.id(), maturing.build, {true, 1200, 1800}, 5s), + "settled rules-9 population entry"); + recover(maturing); + auto grown = unattended(maturing); + bool grew = false; + for (const auto& life : grown.state->state().lives()) { + const auto previous = + std::ranges::find(maturing.population->state().lives(), life.identity.life, + [](const auto& old) { return old.identity.life; }); + grew |= previous != maturing.population->state().lives().end() && + life.growth.realized_per_thousand > previous->growth.realized_per_thousand; + } + require(grew, "settled migrated juvenile did not fund further growth"); + take(admit_population_interval(*maturing.db, maturing.key, maturing.lease, maturing.head.head, + grown.batch.header.end_tick, 5s), + "admit settled migration growth"); + maturing.head = take(publish_population_batch(*maturing.db, *maturing.objects, maturing.lease, + maturing.head, grown.batch, *grown.state, + maturing.id(), maturing.build, 5s), + "publish settled migration growth"); + recover(maturing); + require(maturing.population->snapshot() == grown.state->snapshot(), + "settled migration growth replay differs"); + std::puts("rules-9 durable migration and post-settlement funded growth passed"); std::puts("population publication, exact recovery, process loss and fencing passed"); } diff --git a/tests/region_migration_tests.cpp b/tests/region_migration_tests.cpp index 8bc8834..9f8d8af 100644 --- a/tests/region_migration_tests.cpp +++ b/tests/region_migration_tests.cpp @@ -219,6 +219,7 @@ void upgrade_across_migration_and_barrier() { " -v migration_sha_51=$(s 0051_database_history_domain.sql)" " -v migration_sha_52=$(s 0052_census_pending_migrations.sql)" " -v migration_sha_53=$(s 0053_funded_cellular_maturation.sql)" + " -v migration_sha_54=$(s 0054_maturing_cellular_migration.sql)" " -v runtime_role=evo_runtime_test -f migrations/apply.sql" " > /dev/null 2>&1") != 0); CHECK(history(*owner_db) == "1,2,3,4,5,6,7"); @@ -234,9 +235,9 @@ void upgrade_across_migration_and_barrier() { CHECK(shell("PGDATABASE=" + database + " EVO_PG_RUNTIME_ROLE=evo_runtime_test bash \"$EVO_SOURCE_ROOT/" "tools/migrate-postgres.sh\" > /dev/null") == 0); - CHECK(history(*owner_db) == - "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24," - "25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53"); + CHECK(history(*owner_db) == "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24," + "25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46," + "47,48,49,50,51,52,53,54"); CHECK(scalar(*owner_db, "SELECT (c.condeferrable AND c.condeferred AND " "c.confrelid = 'evolution.region_commits'::regclass)::text " "FROM pg_catalog.pg_constraint c WHERE c.conrelid = " diff --git a/tools/ci/history-domain-check.sh b/tools/ci/history-domain-check.sh index c14b414..f395aa3 100644 --- a/tools/ci/history-domain-check.sh +++ b/tools/ci/history-domain-check.sh @@ -58,6 +58,11 @@ for invalid in NULL "'00000000-0000-0000-0000-000000000000'"; do done # Reconstruct the exact pre-context metadata schema in this disposable database. # Restore the resulting real legacy backup through both supported workflows. +# Undo schema 54's three replacements as well as schema 53 below; the legacy +# backup must contain the actual pre-context function contracts and privileges. +awk '/^CREATE FUNCTION evolution\.(check_cellular_migration_trace|prepare_cellular_migration_freeze|publish_cellular_migration)\(/ {capture=1; sub(/^CREATE FUNCTION/,"CREATE OR REPLACE FUNCTION")} capture {print} capture && /^END \$\$;/ {capture=0}' \ + "$root/migrations/0047_cellular_migration_freeze.sql" \ + "$root/migrations/0049_cellular_migration_publication.sql" | sql > /dev/null sql -v runtime_role="$EVO_PG_RUNTIME_ROLE" > /dev/null <<'SQL' DROP FUNCTION evolution.parse_checkpoint_manifest(bytea); ALTER FUNCTION evolution.parse_checkpoint_manifest_v52(bytea) RENAME TO parse_checkpoint_manifest; diff --git a/tools/ci/postgres-check.sh b/tools/ci/postgres-check.sh index 6c463a8..d0ea285 100644 --- a/tools/ci/postgres-check.sh +++ b/tools/ci/postgres-check.sh @@ -154,7 +154,7 @@ PGDATABASE=upgrade_path psql -X -v ON_ERROR_STOP=1 -1 -v "migration_sha=$v1_sha" -f "$root/migrations/0001_region_bootstrap.sql" > /dev/null test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',') FROM evolution.schema_migrations")" = 1 PGDATABASE=upgrade_path bash "$root/tools/migrate-postgres.sh" -test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53 +test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54 # v2 heads have no payload binding: v2->v3 fails closed and leaves history at v2. v2_sha="$(sha256sum "$root/migrations/0002_region_fencing.sql")" v2_sha="${v2_sha%% *}" @@ -175,7 +175,7 @@ PGDATABASE=v3_commits psql -X -v ON_ERROR_STOP=1 \ -c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \ -c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '01000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32), 'hex'))" > /dev/null PGDATABASE=v3_commits bash "$root/tools/migrate-postgres.sh" -test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 53:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53 +test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 54:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54 test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) FROM evolution.region_commits")" = 1 # Review P1: a v4 database holding a published (possibly terminal) checkpoint has # no durable decisions for it. The v5 upgrade must refuse before changing anything. @@ -229,7 +229,7 @@ PGDATABASE=v6_populated psql -X -v ON_ERROR_STOP=1 \ -c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \ -c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '02000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32) || repeat('00', 80), 'hex'))" > /dev/null PGDATABASE=v6_populated bash "$root/tools/migrate-postgres.sh" -test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53 +test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54 test "$(PGDATABASE=v6_populated psql -XAtc "SELECT count(*) FROM evolution.region_commits c, LATERAL evolution.parse_checkpoint_manifest(c.manifest) m WHERE m.commit_sequence = 0 AND m.predecessor_commit IS NULL")" = 1 test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(conname, ',') FROM pg_constraint WHERE conrelid = 'evolution.region_heads'::regclass AND contype = 'c' AND pg_get_constraintdef(oid) LIKE '%end_tick%'")" = region_heads_published_tick phase permission-audit @@ -287,9 +287,9 @@ expect_audit_failure 'REVOKE EXECUTE ON FUNCTION evolution.load_world_cellular_m expect_audit_failure 'GRANT EXECUTE ON FUNCTION evolution.fork_restored_history_v1(uuid) TO evo_runtime_test' 'REVOKE EXECUTE ON FUNCTION evolution.fork_restored_history_v1(uuid) FROM evo_runtime_test' phase checksum-drift # Unknown future versions and checksum drift must not be repaired silently. -psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=54 WHERE version=53' +psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=55 WHERE version=54' if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi -psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=53 WHERE version=54" +psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=54 WHERE version=55" psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=14" if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=13" diff --git a/tools/migrate-postgres.sh b/tools/migrate-postgres.sh index a732f5f..eaa4a9f 100644 --- a/tools/migrate-postgres.sh +++ b/tools/migrate-postgres.sh @@ -118,6 +118,8 @@ digest_52="$(sha256sum "$root/migrations/0052_census_pending_migrations.sql")" digest_52="${digest_52%% *}" digest_53="$(sha256sum "$root/migrations/0053_funded_cellular_maturation.sql")" digest_53="${digest_53%% *}" +digest_54="$(sha256sum "$root/migrations/0054_maturing_cellular_migration.sql")" +digest_54="${digest_54%% *}" psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \ --set="migration_sha_2=$digest_2" --set="migration_sha_3=$digest_3" \ --set="migration_sha_4=$digest_4" --set="migration_sha_5=$digest_5" \ @@ -163,6 +165,6 @@ psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \ --set="migration_sha_50=$digest_50" \ --set="migration_sha_51=$digest_51" \ --set="migration_sha_52=$digest_52" \ - --set="migration_sha_53=$digest_53" \ + --set="migration_sha_53=$digest_53" --set="migration_sha_54=$digest_54" \ --set="runtime_role=$runtime_role" \ --file="$root/migrations/apply.sql"