diff --git a/docs/STATUS.md b/docs/STATUS.md index fe03731..12643bb 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,5 +1,20 @@ # Project status and next steps +## M12.4g durable accounting upgrade - 2026-10-18 + +ADR-0146/schema 46 extends the immutable version-upgrade journal to same-profile +bundle-9 to 10 upgrades. Full source/result replay protects biology and cohort +policy/membership/sequence; pending intervals and current lease/head fences remain +enforced. Batch 2 is admitted only under bundle 10, and representation changes +preserve either tuple. No worker upgrades automatically. + +Native Release passes 78/78; Clang analysis/format and ASan/UBSan upgrade checks +pass. The complete private PostgreSQL/process/backup/DR/upgrade/permission/history +suite passes in 503 s. [Evidence and limits](engineering/VALIDATION.md) retain the +next M12.4 work: durable paired biological escrow, source freeze, destination +activation and linked-boundary contracts. All of M12 remains in progress; no +deployment or live region changed. + ## M12.4f explicit portable accounting upgrade - 2026-10-17 ADR-0145 declares checkpoint bundle 10 for owner 7/population envelope 2, with an diff --git a/docs/adr/0146-durable-boundary-accounting-upgrade.md b/docs/adr/0146-durable-boundary-accounting-upgrade.md new file mode 100644 index 0000000..ea42f6e --- /dev/null +++ b/docs/adr/0146-durable-boundary-accounting-upgrade.md @@ -0,0 +1,56 @@ +# ADR-0146: Durable boundary-accounting upgrade admission + +## Status + +Accepted for scoped M12.4g implementation, 2026-10-17; locally qualified +2026-10-18, including the full private persistence suite. See +[validation](../engineering/VALIDATION.md). No deployment, automatic upgrade or +World0 change. Paired biological escrow remains open. + +## Decision + +Admit ADR-0145's bundle 10 through SQL schema 46. Extend the existing immutable +`cellular_profile_migrations` journal: it already retains both sequences, commits, +manifest/payload digests and exact version tuples. A new table or independent +publication service would duplicate that authority. Old upgrade meanings remain. + +Only an explicit bundle-9 to 10, same-profile (4 or 5), same-tick upgrade may +change the tuple. The existing fenced version-migration publisher verifies the +exact current head, lease/frame and retained build, inserts the complete upgrade +evidence and result, and advances the head in one transaction. An unsettled +population interval blocks a new upgrade. Exact retries return their original +receipt under current authority; changed bytes conflict. Unknown outcomes retain +the exact prepared request. An explicit adapter verifies source SHA-256, derives +the complete upgrade, uploads and reads back its object before publication. + +Recovery under a fresh current lease verifies the immutable journal, both complete +manifests and objects, rederives the exact upgrade, and requires byte equality. +No fallback genesis, older head, opening-stock reset or version reinterpretation +is allowed. Version upgrades and profile changes retain distinct journals. A +no-tick recovery selects the recorded operation, never assumes its kind solely +from the target bundle. + +For bundle 10, population profile 5 requires envelope 2/batch 2. Retained bundle +9 continues to require envelope 1/batch 1. Profile 4's unchanged trace 3 carries +the same rules-8 inputs. Representation transitions preserve the tuple. Interval +admission/settlement, pending player-choice guards and materialization before +player operations retain their existing requirements. Generic publication cannot +change versions or bypass dedicated evidence. No worker automatically upgrades. + +This admits the accounting format only. It does not admit paired migration trace +1, publish two heads, freeze a source, activate a destination ghost, choose an atlas +or resolve multigeneration growth/environment/census gates. + +## Alternatives and qualification + +Reusing representation-transition evidence for a same-profile version change +would erase the distinction needed by recovery. Fresh provisioning would lose +the retained opening history. Keep the existing version-upgrade journal and +extend its exact supported pair instead. + +Require both profiles, full-state replay/continuation, retained-format refusal, +pending interval refusal, stale/current leases, duplicate/conflicting requests, +ambiguous outcomes, publisher process loss, missing/corrupt objects, migration +rollback/history checks and least-privilege audits on private PostgreSQL. Portable +success alone is insufficient. Actual paired escrow and production costs remain +separate gates. diff --git a/docs/adr/README.md b/docs/adr/README.md index 9b3429c..3199737 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -1,5 +1,7 @@ # Architecture decision records +- [ADR-0146: Durable boundary-accounting upgrade admission](0146-durable-boundary-accounting-upgrade.md) - schema 46 extends immutable version-upgrade evidence to same-profile bundle 9->10, full upgrade replay, population batch 2 and preserved representation/interval guards; paired migration escrow remains open. + - [ADR-0145: Explicit boundary-accounting checkpoint upgrade](0145-boundary-accounting-checkpoint-upgrade.md) - portable bundle 10, exact no-tick retained-source upgrade and representation-preserving continuation; SQL admission and durable upgrade evidence remain open. - [ADR-0138: Current cellular population reference instrument](0138-cellular-population-reference-instrument.md) - M12.2c headless rules-8 control, encoded replay partitions and offline closure; no aggregation qualification. diff --git a/docs/architecture/PERSISTENCE_ARCHITECTURE.md b/docs/architecture/PERSISTENCE_ARCHITECTURE.md index 8d6d704..ef75f29 100644 --- a/docs/architecture/PERSISTENCE_ARCHITECTURE.md +++ b/docs/architecture/PERSISTENCE_ARCHITECTURE.md @@ -1,11 +1,14 @@ # Persistence and region authority -M12.4f defines portable cellular bundle 10 for owner-7 boundary accounting, with -an explicit same-profile, no-tick bundle-9 upgrade candidate. Profile transitions -preserve either tuple. Portable reader support grants no publication authority; -schema 45 still refuses bundle 10 and no worker selects it. Dedicated fenced -upgrade evidence/recovery and atomic paired transfer publication remain required. -See [ADR-0145](../adr/0145-boundary-accounting-checkpoint-upgrade.md). +M12.4f defines cellular bundle 10 for owner-7 boundary accounting, with an explicit +same-profile, no-tick bundle-9 upgrade. M12.4g/schema 46 admits it through the +existing immutable version-upgrade journal and lease/head fence. Recovery +rederives the complete source/target transformation. Profile transitions preserve +either tuple; bundle 9 requires population batch 1 and bundle 10 requires batch 2. +Portable reader support grants no publication authority and no worker selects an +upgrade automatically. Atomic paired transfer publication remains required. +See [ADR-0145](../adr/0145-boundary-accounting-checkpoint-upgrade.md) and +[ADR-0146](../adr/0146-durable-boundary-accounting-upgrade.md). M7.1 now scopes the first implementation in [ADR-0058](../adr/0058-region-durability-contract.md) and [REGION_DURABILITY.md](REGION_DURABILITY.md): full checkpoint batches, not diff --git a/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md b/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md index 49b7b49..64ed4d7 100644 --- a/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md +++ b/docs/engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md @@ -97,3 +97,26 @@ confusion is refused in both directions. Portable reader support does not imply database admission: schema 45 and the population prepared validator still reject the tuple. Dedicated upgrade publication/evidence/recovery is the next segment; paired escrow, geometry and remaining biological gates stay open. + +## M12.4g durable accounting upgrade admission + +[ADR-0146](../adr/0146-durable-boundary-accounting-upgrade.md)/schema 46 extends +the existing immutable version-upgrade journal to an explicit same-profile +bundle-9 to 10 publication. Both profiles retain complete source/result manifests, +payload digests and exact version tuples under the current lease and head fence. +An unsettled population interval blocks a new upgrade. The adapter restores the +verified source, derives the exact candidate, uploads/reads back its object and +retains the same prepared bytes after ambiguity. No worker upgrades automatically. + +Recovery selects the recorded no-tick operation: a version upgrade is rederived +from its full retained source; a representation change uses its separate journal. +Both require full byte equality. Matching aggregate inventory cannot admit a +different cohort policy or membership. Bundle 10 requires envelope/batch 2; +bundle 9 retains envelope/batch 1. Raw biological trace 3 and rules 8 are unchanged. +Representation changes preserve either tuple and interval/player-operation +guards remain in place. + +This admits the accounting format and its ordinary continuation. Paired migration +trace 1 still has no SQL publisher, source freeze, staged ghost or activation +rights. Durable two-head escrow, abort/partition/duplicate settlement, linked +geometry and ecology remain open. No deployment, live upgrade or golden changed. diff --git a/docs/engineering/M12_REMAINING_GATES.md b/docs/engineering/M12_REMAINING_GATES.md index bb64493..9cb4dae 100644 --- a/docs/engineering/M12_REMAINING_GATES.md +++ b/docs/engineering/M12_REMAINING_GATES.md @@ -56,8 +56,11 @@ cross-version replay is refused and a new process reproduces a post-transfer by bundle 9 and schema 45. ADR-0145 now defines the portable bundle-10 declaration and explicit same-profile, no-tick source upgrade, with complete payload preservation and no new genesis. -SQL admission, immutable upgrade evidence and replayed recovery remain required -before those candidates can become durable region heads. +ADR-0146/schema 46 supplies SQL admission through the existing immutable +version-upgrade evidence and complete replayed recovery. Both profile upgrades, +population batch 2 and bundle-preserving profile changes are explicit; no worker +upgrades automatically. This format admission does not publish a migration pair +or close source-freeze/destination-activation and escrow qualification. The transfer tests also expose the current rules-8 maturation gap: buds retain their subadult realization while reproduction requires a fully realized parent. Finite extra food alone does not close repeated wildlife generations. A versioned diff --git a/docs/engineering/TEST_STRATEGY.md b/docs/engineering/TEST_STRATEGY.md index 06bef00..75755e6 100644 --- a/docs/engineering/TEST_STRATEGY.md +++ b/docs/engineering/TEST_STRATEGY.md @@ -10,6 +10,16 @@ continue refusing these portable candidates through schema 45 and existing prepared/generic publication paths without advancing the head. Durable admission and upgrade crash/retry recovery remain separate gates. +M12.4g's private PostgreSQL suite upgrades raw/profile-5 bundle-9 sources using +schema 46. It rehearses raw upgrade/trace-3 continuation and rollback, retains a +nonempty cohort, refuses pending-interval upgrades and wrong publication paths, +and recovers after a pre-transaction SIGKILL and an ambiguous exact retry. Changed +requests and stale writers fail. Missing/corrupt objects quarantine; a canonical +but unequal cohort with matching biological totals fails full upgrade replay. +Batch-1/2 cross-binding is refused in both directions. The process-loss and +host-recreation/materialization matrix also runs with batch 2. Full historical +migration/backup/DR/permission regression remains required for schema admission. + Tests defend contracts, not directory coverage. All checks remain active in Release builds. Current proof has no actual network, biology or distributed recovery, so those categories below are milestone gates rather than passing diff --git a/docs/engineering/VALIDATION.md b/docs/engineering/VALIDATION.md index 4dc2036..cb4fe37 100644 --- a/docs/engineering/VALIDATION.md +++ b/docs/engineering/VALIDATION.md @@ -1,5 +1,51 @@ # Validation report +## 2026-10-18 - M12.4g durable boundary-accounting upgrade + +ADR-0146/schema 46 admits the explicit same-profile bundle-9 to 10 upgrade through +the existing immutable version-upgrade journal. Recovery rederives the complete +source/result transformation; representation changes retain separate evidence. +Both profiles preserve their exact tuple through ordinary continuation; bundle +10 requires population batch 2 while bundle 9 retains batch 1. No worker performs +an automatic upgrade. Biology/rules 8, goldens, dependencies and live state are +unchanged. + +Executed locally: + +- Native MSVC Release **78/78 passed**, 12.30 s. +- Actual GCC 14 optional server/PostgreSQL targets rebuilt; final focused private + PostgreSQL population/upgrade suite passed, 6 s total. +- Linux Clang 19 ASan/UBSan final private population/upgrade suite passed, 26 s + total (22 s test phase), with no reports. +- Explicit clang-tidy 19 on both affected adapters and both PostgreSQL test + sources passed, with no owned warnings. Repository clang-format 19 and shell + syntax checks passed. +- Complete frozen-binary private PostgreSQL qualification passed with **exit 0**, + **503 s**: C++ integration/process/load/history (423 s), life reservations, + backup, DR, upgrade, migration history/rollback, runtime permission audit and + checksum/future-version rejection. Earlier interrupted runs are not counted. + +New cases exercise raw upgrade/trace-3 continuation and rollback, nonempty cohort +policy/sequence retention, pending-interval refusal, wrong publication paths, +pre-transaction SIGKILL, ambiguous exact retry, conflicting bytes and stale +writers. Missing/corrupt objects quarantine. A canonical cohort object with equal +biological totals but different policy/membership fails complete upgrade replay. +Batch-1/2 cross-binding is refused in both directions. The publication process +kill and actual host-recreation/materialization matrix also runs with batch 2. + +An initial new-migration CASE expression failed parsing and was corrected before +the successful runs. The first full fixture omitted sibling load/session-probe +tools; a corrected run was then interrupted by a hangup. The final detached run +used every required frozen sibling executable and recorded its completion code. + +Limits: these backup/DR/clone checks retain their existing supported fixtures; +they do not qualify cellular cloning or a new world-wide migration cut. Paired +biological transfer trace 1 remains unadmitted; no durable source freeze, staged +ghost activation, two-head escrow, linked atlas/environment or multi-generation/ +elapsed-day ecological claim follows. No live database, World0 or deployment was +touched. Remote CI/container confirmation of the earlier build-runner repair +remains separate; `Build_Errors.txt` and unrelated local work remain untracked. + ## 2026-10-17 - M12.4f portable boundary-accounting checkpoint upgrade ADR-0145 adds portable bundle 10, explicit bundle-9 no-tick upgrade candidates diff --git a/include/evo/persistence/checkpoint_store.hpp b/include/evo/persistence/checkpoint_store.hpp index d422315..c08b5e7 100644 --- a/include/evo/persistence/checkpoint_store.hpp +++ b/include/evo/persistence/checkpoint_store.hpp @@ -254,6 +254,21 @@ migrate_cellular_profile_to_v3(Postgres& db, const FilesystemObjectStore& object [[nodiscard]] std::expected retry_cellular_profile_migration(Postgres& db, const CellularProfileMigrationPrepared& prepared, std::chrono::milliseconds timeout); +// Explicit same-profile bundle-9 -> 10 upgrade. No worker selects it implicitly. +// Uses the immutable version-upgrade journal, including for population profile 5. +[[nodiscard]] std::expected +migrate_cellular_boundary_profile(Postgres&, const FilesystemObjectStore&, const LeaseGrant&, + const CommittedCheckpoint&, Uuid commit, + const Sha256Digest& build, std::chrono::milliseconds timeout, + const PublishHooks& hooks = {}); +// Verify the recorded no-tick operation under current authority, replaying its +// entire source/target payload. Version upgrades and profile transitions retain +// separate evidence; neither is inferred solely from the target tuple. +[[nodiscard]] std::expected +verify_cellular_no_tick_checkpoint(Postgres&, const FilesystemObjectStore&, const LeaseGrant&, + const CommittedCheckpoint&, const CheckpointManifest&, + std::span payload, + std::chrono::milliseconds timeout); // After an `unknown` publication: resolve exactly the prepared bytes. A committed // result yields the final batch; absent may be retried with the same prepared bytes diff --git a/include/evo/persistence/region_lease_store.hpp b/include/evo/persistence/region_lease_store.hpp index 9e4e545..0e9fc80 100644 --- a/include/evo/persistence/region_lease_store.hpp +++ b/include/evo/persistence/region_lease_store.hpp @@ -95,6 +95,8 @@ publish_cellular_checkpoint(Postgres& db, std::span manifest std::span decisions, std::span actions, std::span trace, std::chrono::milliseconds timeout); +// Explicit version upgrades: retained raw v1->v2/v2->v3, or same-profile +// bundle 9->10 for raw/population owners. Immutable journal and lease/head fence. [[nodiscard]] std::expected publish_cellular_profile_migration(Postgres& db, std::span manifest, std::chrono::milliseconds timeout); diff --git a/migrations/0046_boundary_accounting_upgrade.sql b/migrations/0046_boundary_accounting_upgrade.sql new file mode 100644 index 0000000..5f3bb64 --- /dev/null +++ b/migrations/0046_boundary_accounting_upgrade.sql @@ -0,0 +1,244 @@ +-- ADR-0146: explicit same-profile bundle 9 -> 10 upgrades; retained formats keep their meanings. +ALTER FUNCTION evolution.parse_checkpoint_manifest(bytea) RENAME TO parse_checkpoint_manifest_v45; +CREATE FUNCTION evolution.parse_checkpoint_manifest(m bytea) +RETURNS evolution.checkpoint_manifest LANGUAGE plpgsql IMMUTABLE +SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + c evolution.checkpoint_manifest; + bundle9 constant bytea := '\x080000000600000002000000010000000100000004000000030000000200000001000000'; + bundle10 constant bytea := '\x080000000700000002000000010000000100000004000000030000000200000001000000'; +BEGIN + IF m IS NOT NULL AND octet_length(m) = 360 AND evolution.manifest_uint(m,12,4) IN (4,5) + AND substring(m FROM 161 FOR 36) = bundle10 THEN + IF evolution.manifest_uint(m,56,8) = 0 THEN + RAISE EXCEPTION 'accounting bundle requires retained upgrade' USING ERRCODE = '22023'; + END IF; + c := evolution.parse_checkpoint_manifest_v45(overlay(m PLACING bundle9 FROM 161 FOR 36)); + c.versions := bundle10; + RETURN c; + END IF; + RETURN evolution.parse_checkpoint_manifest_v45(m); +END $$; +REVOKE ALL ON FUNCTION evolution.parse_checkpoint_manifest(bytea) FROM PUBLIC; + +ALTER FUNCTION evolution.cellular_births(bytea,bytea) RENAME TO cellular_births_v45; +REVOKE ALL ON FUNCTION evolution.cellular_births_v45(bytea,bytea) FROM PUBLIC, :"runtime_role"; +CREATE FUNCTION evolution.cellular_births(t bytea,m bytea) +RETURNS TABLE(account_id uuid,request_id evolution.u64,generation evolution.u64,identity evolution.u64,born evolution.u64) +LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp AS $$ +DECLARE c evolution.checkpoint_manifest := evolution.parse_checkpoint_manifest(m); +BEGIN + IF c.versions = '\x080000000700000002000000010000000100000004000000030000000200000001000000'::bytea THEN + m := overlay(m PLACING '\x080000000600000002000000010000000100000004000000030000000200000001000000'::bytea FROM 161 FOR 36); + END IF; + RETURN QUERY SELECT * FROM evolution.cellular_births_v45(t,m); +END $$; +REVOKE ALL ON FUNCTION evolution.cellular_births(bytea,bytea) FROM PUBLIC; + +ALTER FUNCTION evolution.check_population_trace(bytea,bytea) RENAME TO check_population_trace_v45; +REVOKE ALL ON FUNCTION evolution.check_population_trace_v45(bytea,bytea) FROM PUBLIC, :"runtime_role"; +CREATE FUNCTION evolution.check_population_trace(m bytea,t bytea) RETURNS void +LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp AS $$ +DECLARE c evolution.checkpoint_manifest := evolution.parse_checkpoint_manifest(m); n bytea; local_trace bytea; +BEGIN + IF c.versions = '\x080000000700000002000000010000000100000004000000030000000200000001000000'::bytea THEN + IF t IS NULL OR octet_length(t) < 152 OR octet_length(t) <> c.trace_bytes OR sha256(t) <> c.trace_sha256 + OR substring(t FROM 1 FOR 8) <> '\x4145564f50425432'::bytea OR evolution.manifest_uint(t,8,4) <> 2 THEN + RAISE EXCEPTION 'invalid accounting population trace binding' USING ERRCODE = '22023'; + END IF; + -- Normalize ONLY parser-local copies after checking the actual strong binding. + -- Committed manifest/trace, object digests and retry identity remain exact. + local_trace := overlay(t PLACING '\x4145564f50425431'::bytea FROM 1 FOR 8); + local_trace := overlay(local_trace PLACING '\x01000000'::bytea FROM 9 FOR 4); + n := overlay(m PLACING '\x080000000600000002000000010000000100000004000000030000000200000001000000'::bytea FROM 161 FOR 36); + n := overlay(n PLACING sha256(local_trace) FROM 289 FOR 32); + PERFORM evolution.check_population_trace_v45(n,local_trace); + ELSE + PERFORM evolution.check_population_trace_v45(m,t); + END IF; +END $$; +REVOKE ALL ON FUNCTION evolution.check_population_trace(bytea,bytea) FROM PUBLIC; + +ALTER FUNCTION evolution.publish_cellular_profile_migration(bytea) RENAME TO publish_cellular_profile_migration_v45; +REVOKE ALL ON FUNCTION evolution.publish_cellular_profile_migration_v45(bytea) FROM PUBLIC, :"runtime_role"; + +CREATE OR REPLACE FUNCTION evolution.publish_cellular_profile_migration(p_manifest bytea) +RETURNS TABLE(region_frame evolution.u64, head_sequence evolution.u64, head_commit uuid, + head_digest text, head_tick evolution.u64) +LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + c evolution.checkpoint_manifest := evolution.parse_checkpoint_manifest(p_manifest); + s evolution.checkpoint_manifest; + h evolution.region_heads%ROWTYPE; + p evolution.region_commits%ROWTYPE; + old evolution.region_commits%ROWTYPE; + frame evolution.u64; + d bytea := sha256(p_manifest); + v9_bundle constant bytea := '\x080000000600000002000000010000000100000004000000030000000200000001000000'; + v10_bundle constant bytea := '\x080000000700000002000000010000000100000004000000030000000200000001000000'; + zero32 constant bytea := decode(repeat('00', 32), 'hex'); +BEGIN + IF c.versions <> v10_bundle THEN + RETURN QUERY SELECT * FROM evolution.publish_cellular_profile_migration_v45(p_manifest); + RETURN; + END IF; + IF c.profile NOT IN (4,5) OR c.commit_sequence = 0 OR c.begin_tick <> c.end_tick + OR c.end_tick = 0 OR c.versions <> v10_bundle + OR evolution.manifest_uint(p_manifest, 228, 4) <> 0 + OR evolution.manifest_uint(p_manifest, 232, 4) <> (CASE c.profile WHEN 4 THEN 4 ELSE 5 END) + OR evolution.manifest_uint(p_manifest, 236, 4) <> 1 + OR c.trace_bytes <> 0 OR c.trace_sha256 <> zero32 + OR c.decision_count <> 0 OR c.decisions_sha256 <> zero32 THEN + RAISE EXCEPTION 'invalid boundary-accounting profile migration' USING ERRCODE = '22023'; + END IF; + PERFORM evolution.check_region_lease(c.world_id, c.region_id, c.holder, c.epoch); + SELECT * INTO old FROM evolution.region_commits r WHERE r.commit_id = c.commit_id; + IF FOUND THEN + IF old.manifest <> p_manifest THEN + RAISE EXCEPTION 'cellular profile migration identity reused' USING ERRCODE = 'EVL07'; + END IF; + SELECT * INTO p FROM evolution.region_commits r + WHERE r.world_id = old.world_id AND r.region_id = old.region_id + AND r.commit_sequence = old.commit_sequence - 1; + IF NOT FOUND OR NOT EXISTS ( + SELECT 1 FROM evolution.cellular_profile_migrations m + WHERE m.target_commit = c.commit_id AND m.world_id = c.world_id + AND m.region_id = c.region_id AND m.source_sequence = p.commit_sequence + AND m.source_commit = p.commit_id AND m.source_manifest_sha256 = p.commit_digest + AND m.source_payload_sha256 = p.payload_sha256 AND m.source_versions = p.versions + AND m.target_sequence = old.commit_sequence AND m.target_manifest_sha256 = d + AND m.target_payload_sha256 = old.payload_sha256 AND m.target_versions = old.versions + AND m.source_commit = c.predecessor_commit + AND m.source_manifest_sha256 = c.predecessor_digest) THEN + RAISE EXCEPTION 'cellular profile migration identity reused' USING ERRCODE = 'EVL07'; + END IF; + SELECT l.frame_id INTO frame FROM evolution.region_leases l + WHERE l.world_id = c.world_id AND l.region_id = c.region_id; + RETURN QUERY SELECT frame, old.commit_sequence, old.commit_id, + encode(old.commit_digest, 'hex'), old.end_tick; + RETURN; + END IF; + IF EXISTS (SELECT 1 FROM evolution.population_interval_pending x + WHERE x.world_id = c.world_id AND x.region_id = c.region_id) THEN + RAISE EXCEPTION 'accounting upgrade waits for interval' USING ERRCODE = 'EVL18'; + END IF; + SELECT * INTO h FROM evolution.region_heads r + WHERE r.world_id = c.world_id AND r.region_id = c.region_id FOR UPDATE; + IF NOT FOUND OR h.commit_sequence = 18446744073709551615 + OR c.commit_sequence <> h.commit_sequence + 1 OR c.predecessor_commit <> h.commit_id + OR c.predecessor_digest <> h.digest OR c.begin_tick <> h.end_tick THEN + RAISE EXCEPTION 'region head moved' USING ERRCODE = 'EVL03'; + END IF; + SELECT * INTO p FROM evolution.region_commits r + WHERE r.world_id = c.world_id AND r.region_id = c.region_id + AND r.commit_sequence = h.commit_sequence; + IF NOT FOUND THEN + RAISE EXCEPTION 'missing migration predecessor' USING ERRCODE = 'XX000'; + END IF; + s := evolution.parse_checkpoint_manifest(p.manifest); + IF s.profile <> c.profile OR s.versions <> v9_bundle OR p.versions <> v9_bundle + OR c.frame_id <> s.frame_id OR c.tick_microseconds <> s.tick_microseconds + OR s.commit_id <> c.predecessor_commit OR s.end_tick <> c.end_tick + OR evolution.manifest_uint(p.manifest, 232, 4) <> (CASE c.profile WHEN 4 THEN 4 ELSE 5 END) + OR NOT EXISTS (SELECT 1 FROM evolution.retained_builds b + WHERE b.build_identity = c.build_identity) THEN + RAISE EXCEPTION 'unsupported boundary-accounting profile migration' USING ERRCODE = 'EVL06'; + END IF; + SELECT l.frame_id INTO frame FROM evolution.region_leases l + WHERE l.world_id = c.world_id AND l.region_id = c.region_id; + IF c.frame_id <> frame THEN + RAISE EXCEPTION 'foreign frame' USING ERRCODE = '22023'; + END IF; + INSERT INTO evolution.cellular_profile_migrations(target_commit, world_id, region_id, + source_sequence, source_commit, source_manifest_sha256, source_payload_sha256, + source_versions, target_sequence, target_manifest_sha256, target_payload_sha256, + target_versions) + VALUES (c.commit_id, c.world_id, c.region_id, p.commit_sequence, p.commit_id, + p.commit_digest, p.payload_sha256, p.versions, c.commit_sequence, d, + c.payload_sha256, c.versions); + INSERT INTO evolution.region_commits(world_id, region_id, commit_sequence, commit_id, + commit_digest, manifest, begin_tick, end_tick, published_epoch, tick_microseconds, + versions, build_identity, payload_bytes, payload_sha256) + VALUES (c.world_id, c.region_id, c.commit_sequence, c.commit_id, d, p_manifest, + c.begin_tick, c.end_tick, c.epoch, c.tick_microseconds, c.versions, + c.build_identity, c.payload_bytes, c.payload_sha256); + UPDATE evolution.region_heads r + SET commit_sequence = c.commit_sequence, commit_id = c.commit_id, digest = d, + end_tick = c.end_tick, published_epoch = c.epoch + WHERE r.world_id = c.world_id AND r.region_id = c.region_id; + RETURN QUERY SELECT frame, c.commit_sequence, c.commit_id, encode(d, 'hex'), c.end_tick; +END $$; +REVOKE ALL ON FUNCTION evolution.publish_cellular_profile_migration(bytea) FROM PUBLIC; +GRANT EXECUTE ON FUNCTION evolution.publish_cellular_profile_migration(bytea) TO :"runtime_role"; + + +CREATE OR REPLACE FUNCTION evolution.publish_population_profile_transition_v44(m bytea) +RETURNS TABLE(region_frame evolution.u64,head_sequence evolution.u64,head_commit uuid, + head_digest text,head_tick evolution.u64) +LANGUAGE plpgsql SECURITY DEFINER SET search_path = pg_catalog, pg_temp AS $$ +DECLARE + c evolution.checkpoint_manifest := evolution.parse_checkpoint_manifest(m); + s evolution.checkpoint_manifest; p evolution.region_commits%ROWTYPE; + h evolution.region_heads%ROWTYPE; old evolution.region_commits%ROWTYPE; + d bytea := sha256(m); frame evolution.u64; + bundle9 constant bytea := '\x080000000600000002000000010000000100000004000000030000000200000001000000'; +BEGIN + IF c.profile NOT IN (4,5) OR c.versions NOT IN (bundle9,'\x080000000700000002000000010000000100000004000000030000000200000001000000'::bytea) OR c.commit_sequence = 0 + OR c.begin_tick <> c.end_tick OR c.end_tick = 0 OR c.trace_bytes <> 0 THEN + RAISE EXCEPTION 'invalid population profile transition' USING ERRCODE = '22023'; + END IF; + PERFORM evolution.check_region_lease(c.world_id,c.region_id,c.holder,c.epoch); + SELECT * INTO old FROM evolution.region_commits r WHERE r.commit_id = c.commit_id; + IF FOUND THEN + IF old.manifest <> m OR NOT EXISTS (SELECT 1 FROM evolution.population_profile_transitions x + WHERE x.target_commit = c.commit_id AND x.source_commit = c.predecessor_commit + AND x.source_manifest_sha256 = c.predecessor_digest AND x.target_profile = c.profile) THEN + RAISE EXCEPTION 'population transition identity reused' USING ERRCODE = 'EVL07'; + END IF; + SELECT l.frame_id INTO frame FROM evolution.region_leases l WHERE l.world_id = c.world_id AND l.region_id = c.region_id; + RETURN QUERY SELECT frame,old.commit_sequence,old.commit_id,encode(old.commit_digest,'hex'),old.end_tick; + RETURN; + END IF; + SELECT * INTO h FROM evolution.region_heads r WHERE r.world_id = c.world_id AND r.region_id = c.region_id FOR UPDATE; + IF NOT FOUND OR h.commit_sequence = 18446744073709551615 OR c.commit_sequence <> h.commit_sequence + 1 + OR c.predecessor_commit <> h.commit_id OR c.predecessor_digest <> h.digest OR c.begin_tick <> h.end_tick THEN + RAISE EXCEPTION 'population transition head moved' USING ERRCODE = 'EVL03'; + END IF; + SELECT * INTO p FROM evolution.region_commits r WHERE r.commit_id = h.commit_id; + IF NOT FOUND THEN RAISE EXCEPTION 'missing population source' USING ERRCODE = 'XX000'; END IF; + s := evolution.parse_checkpoint_manifest(p.manifest); + IF c.profile = 5 AND EXISTS (SELECT 1 FROM evolution.cellular_mutation_reservations x + WHERE x.world_id = c.world_id AND x.region_id = c.region_id AND NOT EXISTS + (SELECT 1 FROM evolution.cellular_mutation_action_publications a + WHERE a.world_id = x.world_id AND a.account_id = x.account_id AND a.action_id = x.action_id)) THEN + RAISE EXCEPTION 'pending player mutation blocks population transition' USING ERRCODE = 'EVL18'; + END IF; + IF s.profile NOT IN (4,5) OR s.profile = c.profile OR s.versions <> c.versions + OR s.frame_id <> c.frame_id OR s.tick_microseconds <> c.tick_microseconds + OR s.end_tick <> c.end_tick OR NOT EXISTS (SELECT 1 FROM evolution.retained_builds b WHERE b.build_identity = c.build_identity) THEN + RAISE EXCEPTION 'unsupported population source' USING ERRCODE = 'EVL06'; + END IF; + INSERT INTO evolution.population_profile_transitions VALUES(c.commit_id,p.commit_id,p.commit_digest,p.payload_sha256,s.profile,c.profile); + INSERT INTO evolution.region_commits(world_id,region_id,commit_sequence,commit_id,commit_digest,manifest, + begin_tick,end_tick,published_epoch,tick_microseconds,versions,build_identity,payload_bytes,payload_sha256) + VALUES(c.world_id,c.region_id,c.commit_sequence,c.commit_id,d,m,c.begin_tick,c.end_tick,c.epoch, + c.tick_microseconds,c.versions,c.build_identity,c.payload_bytes,c.payload_sha256); + UPDATE evolution.region_heads r SET commit_sequence = c.commit_sequence,commit_id = c.commit_id, + digest = d,end_tick = c.end_tick,published_epoch = c.epoch WHERE r.world_id = c.world_id AND r.region_id = c.region_id; + RETURN QUERY SELECT c.frame_id,c.commit_sequence,c.commit_id,encode(d,'hex'),c.end_tick; +END $$; +CREATE OR REPLACE FUNCTION evolution.refuse_unpublished_population() RETURNS trigger +LANGUAGE plpgsql SECURITY INVOKER SET search_path = pg_catalog, pg_temp AS $$ +BEGIN + IF evolution.manifest_uint(NEW.manifest,12,4) = 5 AND + (NEW.commit_sequence = 0 OR + (NOT EXISTS (SELECT 1 FROM evolution.population_publications p WHERE p.commit_id = NEW.commit_id) + AND NOT EXISTS (SELECT 1 FROM evolution.population_profile_transitions p + WHERE p.target_commit = NEW.commit_id AND p.target_profile = 5) + AND NOT EXISTS (SELECT 1 FROM evolution.cellular_profile_migrations p WHERE p.target_commit = NEW.commit_id))) THEN + RAISE EXCEPTION 'population commit requires fenced publisher' USING ERRCODE = 'EVL06'; + END IF; + RETURN NEW; +END $$; +REVOKE ALL ON FUNCTION evolution.refuse_unpublished_population() FROM PUBLIC; +INSERT INTO evolution.schema_migrations(version,sha256) VALUES(46,:'migration_sha_46'); diff --git a/migrations/apply.sql b/migrations/apply.sql index 2d3c9e0..ac03d4c 100644 --- a/migrations/apply.sql +++ b/migrations/apply.sql @@ -121,6 +121,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution') SELECT :'history_v42' || ',43:' || :'migration_sha_43' AS history_v43 \gset SELECT :'history_v43' || ',44:' || :'migration_sha_44' AS history_v44 \gset SELECT :'history_v44' || ',45:' || :'migration_sha_45' AS history_v45 \gset + SELECT :'history_v45' || ',46:' || :'migration_sha_46' AS history_v46 \gset SELECT :'history' = :'history_v22' AS at_v22, :'history' = :'history_v23' AS at_v23, :'history' = :'history_v24' AS at_v24, @@ -144,8 +145,10 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution') :'history' = :'history_v42' AS at_v42, :'history' = :'history_v43' AS at_v43, :'history' = :'history_v44' AS at_v44, - :'history' = :'history_v45' AS at_v45 \gset - \if :at_v45 + :'history' = :'history_v45' AS at_v45, + :'history' = :'history_v46' AS at_v46 \gset + \if :at_v46 + \elif :at_v45 \elif :at_v44 \elif :at_v43 \elif :at_v42 @@ -529,7 +532,8 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24, coalesce(max(version), 0) = 42 AS at_v42, coalesce(max(version), 0) = 43 AS at_v43, coalesce(max(version), 0) = 44 AS at_v44, - coalesce(max(version), 0) = 45 AS at_v45 + coalesce(max(version), 0) = 45 AS at_v45, + coalesce(max(version), 0) = 46 AS at_v46 FROM evolution.schema_migrations \gset \if :at_v24 \ir 0025_cellular_v2_bundle.sql @@ -743,6 +747,7 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24, \elif :at_v43 \elif :at_v44 \elif :at_v45 +\elif :at_v46 \else \echo 'Unknown or changed migration history after v43' DO $$ BEGIN RAISE EXCEPTION 'unknown or changed migration history'; END $$; @@ -755,6 +760,10 @@ SELECT coalesce(max(version), 0) = 44 AS needs_v45 FROM evolution.schema_migrati \if :needs_v45 \ir 0045_owned_population_intervals.sql \endif +SELECT coalesce(max(version), 0) = 45 AS needs_v46 FROM evolution.schema_migrations \gset +\if :needs_v46 + \ir 0046_boundary_accounting_upgrade.sql +\endif -- Effective-privilege audit on every run, including no-ops (ADR-0060). Attributes -- alone are insufficient: predefined roles such as pg_write_all_data, PUBLIC -- grants or SET-able memberships would bypass the fenced functions. Checks the diff --git a/schemas/cellular-population-batch.md b/schemas/cellular-population-batch.md index 4f6f8e4..05b18a0 100644 --- a/schemas/cellular-population-batch.md +++ b/schemas/cellular-population-batch.md @@ -63,3 +63,11 @@ Batch 2 is portable replay support only. Durable bundle 9's C++ prepared-commit validator and schema 45's SQL trace checker admit only batch 1. A new durable bundle/upgrade and exact history/recovery contract remain required before any publication, scheduling or transfer activation can use the new representation. + +M12.4g/schema 46 adds the dedicated bundle-9 to 10 upgrade and full recovery +contract. Bundle 10 admits batch 2 for ordinary intervals and manual conversion; +bundle 9 still refuses it and remains bound to batch 1. Actual SHA-256 and wire +magic/version bindings are checked before SQL's parser-local normalization; +stored bytes and retry identities are unchanged. Interval admission/settlement +and player-operation materialization keep their existing meaning. Paired transfer +trace 1 remains unadmitted and this adds no destination activation rights. diff --git a/schemas/cellular-population.md b/schemas/cellular-population.md index 23fc275..00db8fd 100644 --- a/schemas/cellular-population.md +++ b/schemas/cellular-population.md @@ -51,3 +51,9 @@ materialization or new automatic-policy evaluation. Untagged members stay untagg Envelope 2 is portable only in this segment. Existing durable trace 1 refuses it; schema 45/profile 5 does not admit the new nested owner tuple. No live flag or default entry enables migration or this new representation. + +M12.4g/ADR-0146 subsequently admits envelope 2 as profile 5 with bundle 10 and +history 5 in schema 46. An explicit no-tick version upgrade retains membership, +policy and conversion sequence through the immutable version-upgrade journal. +Ordinary publication requires population batch 2; profile transitions preserve +bundle 10. No worker upgrades automatically and no paired migration is published. diff --git a/schemas/region-commit.md b/schemas/region-commit.md index 4437690..12b6353 100644 --- a/schemas/region-commit.md +++ b/schemas/region-commit.md @@ -10,6 +10,15 @@ transitions preserve the exact bundle. Retained bundle 9 keeps envelope/batch 1. These portable readers and candidate constructors grant no database authority: SQL schema 45 still rejects bundle 10. See [ADR-0145](../docs/adr/0145-boundary-accounting-checkpoint-upgrade.md). +M12.4g/schema 46 subsequently admits that tuple through the fenced version-upgrade +publisher and its immutable `cellular_profile_migrations` evidence, for either +profile. The profile is unchanged across an upgrade; profile transitions retain +their separate journal and require the same tuple on both ends. Pending intervals +block a new upgrade. Bundle 10 binds population trace 2; bundle 9 still binds +trace 1. Recovery rederives the entire source/target upgrade, not merely its stock +totals. No paired biological migration trace is admitted by this schema. See +[ADR-0146](../docs/adr/0146-durable-boundary-accounting-upgrade.md). + M11.2 adds the portable [primitive creation profile 3](primitive-checkpoint-v1.md) using manifest v2. M11.3 adds profile 4, the [cellular lives profile v1](cellular-lives-v1.md), with its own version bundle, diff --git a/src/persistence/cellular_checkpoint_store.cpp b/src/persistence/cellular_checkpoint_store.cpp index 9ccc274..d2b3f56 100644 --- a/src/persistence/cellular_checkpoint_store.cpp +++ b/src/persistence/cellular_checkpoint_store.cpp @@ -236,9 +236,13 @@ retry_cellular_profile_migration(Postgres& db, const CellularProfileMigrationPre milliseconds timeout) { const auto manifest = decode_manifest(prepared.manifest); if (!manifest || manifest->key != prepared.key || manifest->commit != prepared.commit || - manifest->profile != CheckpointProfile::cellular_lives_v1 || - (manifest->cellular_versions != cellular_v2_versions && - manifest->cellular_versions != cellular_v3_versions) || + !supported(*manifest) || !encode_manifest(*manifest) || + !((manifest->profile == CheckpointProfile::cellular_lives_v1 && + (manifest->cellular_versions == cellular_v2_versions || + manifest->cellular_versions == cellular_v3_versions)) || + ((manifest->profile == CheckpointProfile::cellular_lives_v1 || + manifest->profile == CheckpointProfile::cellular_population_v1) && + manifest->cellular_versions == cellular_v10_versions)) || manifest->begin_tick != manifest->end_tick || manifest->batch != BatchDigests{}) return std::unexpected{migration_failure(PublicationError::candidate, {}, {}, prepared)}; @@ -271,6 +275,43 @@ retry_cellular_profile_migration(Postgres& db, const CellularProfileMigrationPre migration_failure(PublicationError::rejected, {}, published.error(), prepared)}; } +std::expected +migrate_cellular_boundary_profile(Postgres& db, const FilesystemObjectStore& objects, + const LeaseGrant& lease, const CommittedCheckpoint& predecessor, + Uuid commit, const Sha256Digest& build, milliseconds timeout, + const PublishHooks& hooks) { + const auto previous = decode_manifest(predecessor.manifest); + const auto previous_digest = sha256(predecessor.manifest); + if (!previous || !previous_digest || predecessor.head.digest != *previous_digest || + predecessor.head.sequence != previous->sequence || + predecessor.head.commit != previous->commit || + predecessor.head.end_tick != previous->end_tick || + predecessor.head.frame != previous->frame) + return std::unexpected(migration_failure(PublicationError::predecessor)); + auto source = objects.get({previous->payload_sha256, previous->payload_bytes}); + if (!source) + return std::unexpected(migration_failure(PublicationError::predecessor, source.error())); + auto payload = cellular_boundary_upgrade_payload(*source, *previous); + if (!payload) + return std::unexpected(migration_failure(PublicationError::candidate)); + const auto digest = sha256(*payload); + if (!digest) + return std::unexpected(migration_failure(PublicationError::digest)); + auto manifest = cellular_boundary_upgrade_manifest(*previous, *previous_digest, commit, lease, + build, payload->size(), *digest); + if (!manifest) + return std::unexpected(migration_failure(PublicationError::candidate)); + auto encoded = encode_manifest(*manifest); + if (!encoded) + return std::unexpected(migration_failure(PublicationError::candidate)); + if (auto stored = put_verified(objects, *payload, {*digest, payload->size()}); !stored) + return std::unexpected(migration_failure(stored.error().kind, stored.error().object)); + if (hooks.before_transaction) + hooks.before_transaction(); + return retry_cellular_profile_migration(db, {previous->key, commit, std::move(*encoded)}, + timeout); +} + std::expected migrate_cellular_profile_to_v2(Postgres& db, const FilesystemObjectStore& objects, const LeaseGrant& lease, const CommittedCheckpoint& predecessor, @@ -500,6 +541,80 @@ publish_cellular_batch(Postgres& db, const FilesystemObjectStore& objects, const return PublishedCellularBatch{std::move(*committed), std::move(published_results)}; } +std::expected +verify_cellular_no_tick_checkpoint(Postgres& db, const FilesystemObjectStore& objects, + const LeaseGrant& lease, const CommittedCheckpoint& committed, + const CheckpointManifest& m, + std::span payload, milliseconds timeout) { + const auto fail = [](RecoveryError kind) { return std::unexpected(RecoveryFailure{kind, {}}); }; + const auto digest = sha256(committed.manifest); + if (!digest || *digest != committed.head.digest || m.commit != committed.head.commit || + m.sequence != committed.head.sequence || m.end_tick != committed.head.end_tick || + m.frame != committed.head.frame || lease.frame != m.frame || m.sequence == 0 || + m.begin_tick != m.end_tick || m.batch != BatchDigests{}) + return fail(RecoveryError::manifest); + auto evidence = load_cellular_profile_migration(db, m.key, lease, m.commit, timeout); + if (!evidence) + return std::unexpected(RecoveryFailure{RecoveryError::store, evidence.error()}); + const auto optional_record = std::move(*evidence); + if (!optional_record.has_value()) + return verify_population_profile_transition(db, objects, lease, committed, m, payload, + timeout); + const auto& record = *optional_record; + auto predecessor = load_region_commit(db, m.key, m.sequence - 1, timeout); + if (!predecessor) + return std::unexpected(RecoveryFailure{RecoveryError::tail, predecessor.error()}); + const auto source_digest = sha256(predecessor->manifest); + const auto source = decode_manifest(predecessor->manifest); + const auto target_bytes = encode_manifest(m); + const auto payload_digest = sha256(payload); + if (!source || !source_digest || !target_bytes || *target_bytes != committed.manifest || + payload.size() != m.payload_bytes || !payload_digest || + *payload_digest != m.payload_sha256 || *source_digest != predecessor->head.digest || + source->commit != predecessor->head.commit || + source->sequence != predecessor->head.sequence || + source->end_tick != predecessor->head.end_tick || + source->frame != predecessor->head.frame || record.source_sequence != source->sequence || + record.source_commit != source->commit || record.source_manifest_sha256 != *source_digest || + record.source_payload_sha256 != source->payload_sha256 || + record.target_sequence != m.sequence || record.target_commit != m.commit || + record.target_manifest_sha256 != *digest || + record.target_payload_sha256 != m.payload_sha256 || + !std::ranges::equal(record.source_versions, + std::span{predecessor->manifest}.subspan(160, 36)) || + !std::ranges::equal(record.target_versions, std::span{*target_bytes}.subspan(160, 36))) + return fail(RecoveryError::history); + // Reconstruct with the recorded writer's lease fields. Loading the immutable + // evidence above is fenced by the CURRENT lease, which may be a newer epoch. + const LeaseGrant recorded{m.holder, m.epoch, m.frame, milliseconds{0}}; + const bool boundary = m.cellular_versions == cellular_v10_versions; + auto expected = + boundary ? cellular_boundary_upgrade_manifest(*source, *source_digest, m.commit, recorded, + m.build, m.payload_bytes, m.payload_sha256) + : cellular_profile_migration_manifest(*source, *source_digest, m.commit, recorded, + m.build, m.payload_bytes, m.payload_sha256); + if (!expected || *expected != m) + return fail(RecoveryError::history); + auto bytes = objects.get({source->payload_sha256, source->payload_bytes}); + if (!bytes) + return fail(RecoveryError::tail); + if (boundary) { + auto upgraded = cellular_boundary_upgrade_payload(*bytes, *source); + if (!upgraded || !std::ranges::equal(*upgraded, payload)) + return fail(RecoveryError::history); + } else { + auto state = restore_cellular_payload(*bytes, *source); + if (!state) + return fail(RecoveryError::tail); + auto upgraded = source->cellular_versions == cellular_v1_versions + ? (*state)->migrate_to_v2() + : (*state)->migrate_to_v3(); + if (!upgraded || !std::ranges::equal((*upgraded)->snapshot(), payload)) + return fail(RecoveryError::history); + } + return {}; +} + std::expected recover_cellular_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key, const LeaseGrant& lease, std::span retained_builds, @@ -538,75 +653,13 @@ recover_cellular_checkpoint(Postgres& db, const FilesystemObjectStore& objects, std::vector actions; std::vector mutation_actions_recovered; std::vector mutation_results_recovered; - if (manifest->sequence != 0 && manifest->begin_tick == manifest->end_tick && - manifest->cellular_versions == cellular_v9_versions) { - auto verified = verify_population_profile_transition(db, objects, lease, *committed, - *manifest, *payload, timeout); + if (manifest->sequence != 0 && manifest->begin_tick == manifest->end_tick) { + auto verified = verify_cellular_no_tick_checkpoint(db, objects, lease, *committed, + *manifest, *payload, timeout); if (!verified) return std::unexpected(verified.error()); return RecoveredCellularCheckpoint{ std::move(*committed), *manifest, std::move(*state), {}, {}, {}}; - } else if (manifest->sequence != 0 && manifest->begin_tick == manifest->end_tick) { - auto migration = load_cellular_profile_migration(db, key, lease, manifest->commit, timeout); - if (!migration) - return std::unexpected{RecoveryFailure{RecoveryError::store, migration.error()}}; - const auto migration_record = migration->value_or(CellularProfileMigrationRecord{}); - if (!*migration || migration_record.target_sequence != manifest->sequence || - migration_record.target_commit != manifest->commit || - migration_record.target_manifest_sha256 != *digest || - migration_record.target_payload_sha256 != manifest->payload_sha256 || - migration_record.target_sequence == 0) - return fail_with(RecoveryError::history); - auto predecessor = load_region_commit(db, key, manifest->sequence - 1, timeout); - if (!predecessor) - return std::unexpected{RecoveryFailure{RecoveryError::tail, predecessor.error()}}; - const auto source_manifest_digest = sha256(predecessor->manifest); - auto source_manifest = decode_manifest(predecessor->manifest); - const auto source_manifest_bytes = - encode_manifest(source_manifest ? *source_manifest : CheckpointManifest{}); - const auto target_manifest_bytes = encode_manifest(*manifest); - if (!source_manifest_digest || !source_manifest || !source_manifest_bytes || - !target_manifest_bytes || *source_manifest_digest != manifest->predecessor_digest || - predecessor->head.digest != *source_manifest_digest || - predecessor->head.commit != manifest->predecessor_commit || - predecessor->head.sequence != manifest->sequence - 1 || - source_manifest->profile != CheckpointProfile::cellular_lives_v1 || - (source_manifest->cellular_versions != cellular_v1_versions && - source_manifest->cellular_versions != cellular_v2_versions) || - ((source_manifest->cellular_versions == cellular_v1_versions) != - (manifest->cellular_versions == cellular_v2_versions)) || - source_manifest->end_tick != manifest->end_tick || - migration_record.source_sequence != source_manifest->sequence || - migration_record.source_commit != source_manifest->commit || - migration_record.source_manifest_sha256 != *source_manifest_digest || - migration_record.source_payload_sha256 != source_manifest->payload_sha256 || - migration_record.source_versions.size() != 36 || - !std::ranges::equal(migration_record.source_versions, - std::span{*source_manifest_bytes}.subspan(160, 36)) || - migration_record.target_versions.size() != 36 || - !std::ranges::equal(migration_record.target_versions, - std::span{*target_manifest_bytes}.subspan(160, 36))) - return fail_with(RecoveryError::history); - auto source_bytes = - objects.get({source_manifest->payload_sha256, source_manifest->payload_bytes}); - if (!source_bytes) - return fail_with(source_bytes.error() == ObjectError::io ? RecoveryError::store - : RecoveryError::tail); - const auto source_payload_digest = sha256(*source_bytes); - if (!source_payload_digest || *source_payload_digest != source_manifest->payload_sha256) - return fail_with(RecoveryError::tail); - auto source_state = restore_cellular_payload(*source_bytes, *source_manifest); - if (!source_state || - (*source_state)->format_version() != source_manifest->cellular_versions.envelope) - return fail_with(RecoveryError::tail); - auto migrated = source_manifest->cellular_versions == cellular_v1_versions - ? (*source_state)->migrate_to_v2() - : (*source_state)->migrate_to_v3(); - if (!migrated || (*migrated)->tick() != manifest->end_tick || - (*migrated)->snapshot() != *payload) - return fail_with(RecoveryError::history); - return RecoveredCellularCheckpoint{ - std::move(*committed), *manifest, std::move(*state), {}, {}, {}}; } if (manifest->sequence != 0) { auto action_bytes = diff --git a/src/persistence/population_checkpoint_store.cpp b/src/persistence/population_checkpoint_store.cpp index 20da44c..594c912 100644 --- a/src/persistence/population_checkpoint_store.cpp +++ b/src/persistence/population_checkpoint_store.cpp @@ -20,7 +20,7 @@ bool bound(const CommittedCheckpoint& checkpoint, const CheckpointManifest& m) { } bool canonical(const PopulationPreparedCommit& prepared, const CheckpointManifest& m) { if (!supported(m) || m.key != prepared.key || m.commit != prepared.commit || m.sequence == 0 || - m.cellular_versions != cellular_v9_versions || !encode_manifest(m)) + population_envelope_version(m.cellular_versions) == 0 || !encode_manifest(m)) return false; if (prepared.trace.empty()) return (m.profile == CheckpointProfile::cellular_lives_v1 || @@ -28,7 +28,8 @@ bool canonical(const PopulationPreparedCommit& prepared, const CheckpointManifes m.begin_tick == m.end_tick && m.batch == BatchDigests{}; const auto trace = server::decode_population_batch(prepared.trace); const auto digest = sha256(prepared.trace); - if (!trace || !digest || trace->format_version != 1 || + if (!trace || !digest || + trace->format_version != population_envelope_version(m.cellular_versions) || m.profile != CheckpointProfile::cellular_population_v1 || m.batch != BatchDigests{prepared.trace.size(), *digest, 0, {}}) return false; @@ -62,7 +63,7 @@ transition(Postgres& db, const FilesystemObjectStore& objects, const LeaseGrant& if (!previous || !previous_digest || !bound(predecessor, *previous) || previous->profile != (to_population ? CheckpointProfile::cellular_lives_v1 : CheckpointProfile::cellular_population_v1) || - previous->cellular_versions != cellular_v9_versions) + population_envelope_version(previous->cellular_versions) == 0) return std::unexpected(failure(PublicationError::predecessor)); auto source = objects.get({previous->payload_sha256, previous->payload_bytes}); if (!source) @@ -218,8 +219,9 @@ verify_population_profile_transition(Postgres& db, const FilesystemObjectStore& source->commit != m.predecessor_commit || predecessor->head.digest != m.predecessor_digest || source->key != m.key || source->frame != m.frame || source->end_tick != m.end_tick || - source->cellular_versions != cellular_v9_versions || - m.cellular_versions != cellular_v9_versions || record.source_commit != source->commit || + population_envelope_version(source->cellular_versions) == 0 || + m.cellular_versions != source->cellular_versions || + record.source_commit != source->commit || record.source_manifest_sha256 != predecessor->head.digest || record.source_payload_sha256 != source->payload_sha256) return fail(RecoveryError::tail); @@ -274,8 +276,8 @@ recover_population_checkpoint(Postgres& db, const FilesystemObjectStore& objects if (!population) return fail(RecoveryError::payload); if (m->batch == BatchDigests{}) { - if (auto verified = verify_population_profile_transition(db, objects, lease, *committed, *m, - *bytes, timeout); + if (auto verified = verify_cellular_no_tick_checkpoint(db, objects, lease, *committed, *m, + *bytes, timeout); !verified) return std::unexpected(verified.error()); } else { diff --git a/tests/population_publication_tests.cpp b/tests/population_publication_tests.cpp index e7eedd3..4b2c706 100644 --- a/tests/population_publication_tests.cpp +++ b/tests/population_publication_tests.cpp @@ -290,12 +290,14 @@ void conversions_and_retries(Fixture& f) { take(encode_manifest(boundary_manifest), "boundary candidate encoding"), {}}; auto rejected_boundary = retry_population_prepared(*f.db, boundary, 5s); - require(!rejected_boundary && rejected_boundary.error().kind == PublicationError::candidate && - !resolve_population_prepared(*f.db, boundary, 5s), - "portable bundle 10 bypassed prepared publication admission"); + require(!rejected_boundary && rejected_boundary.error().kind == PublicationError::rejected && + take(resolve_population_prepared(*f.db, boundary, 5s), + "upgrade through wrong journal resolution") + .status == Resolution::absent, + "accounting upgrade bypassed its dedicated version journal"); auto rejected_upgrade = publish_population_profile_transition(*f.db, boundary.manifest, 5s); - require(!rejected_upgrade && rejected_upgrade.error().kind == LeaseStoreError::invalid, - "schema 45 admitted portable accounting upgrade without dedicated evidence"); + require(!rejected_upgrade, + "representation publisher admitted accounting upgrade without dedicated evidence"); require(!publish_region_checkpoint(*f.db, boundary.manifest, {}, 5s), "generic publisher admitted boundary upgrade"); require(take(load_region_checkpoint(*f.db, f.key, 5s), "head after future rejection").head == @@ -543,6 +545,206 @@ void missing_and_corrupt_objects(Fixture& f) { "corrupt population payload did not fail closed"); recover(f); } +void boundary_upgrades(Fixture& f) { + const std::array builds{f.build}; + const auto original = f.head; + const auto expected_raw = + take(f.raw->migration_upgrade_candidate(), "raw upgrade oracle")->snapshot(); + // Rehearse the raw upgrade, recovery and ordinary trace-3 publication in one + // transaction, then roll it back to retain a real bundle-9 source for profile 5. + take(f.db->query("BEGIN", {}, 5s), "raw upgrade rollback rehearsal"); + auto raw_head = take( + migrate_cellular_boundary_profile(*f.db, *f.objects, f.lease, f.head, f.id(), f.build, 5s), + "raw accounting upgrade"); + auto raw = take(recover_cellular_checkpoint(*f.db, *f.objects, f.key, f.lease, builds, 5s), + "raw accounting recovery"); + require(raw.state->snapshot() == expected_raw && + raw.manifest.cellular_versions == cellular_v10_versions && + raw_head.head.end_tick == original.head.end_tick, + "raw accounting upgrade changed biology or time"); + const CellularBatchHeader h{f.key.world, + f.key.region, + 1, + raw_head.head.commit, + raw_head.head.digest, + raw_head.head.end_tick, + raw_head.head.end_tick + 1}; + const auto inputs = take(unattended_population_inputs(*raw.state, h), "upgraded raw inputs"); + auto advanced = take(replay_cellular_batch(*raw.state, inputs), "upgraded raw oracle"); + take(publish_cellular_batch(*f.db, *f.objects, f.lease, raw_head, inputs, *advanced, f.id(), + f.build, 5s), + "upgraded raw ordinary publication"); + require(take(recover_cellular_checkpoint(*f.db, *f.objects, f.key, f.lease, builds, 5s), + "upgraded raw ordinary recovery") + .state->snapshot() == advanced->snapshot(), + "raw accounting continuation differs"); + take(f.db->query("ROLLBACK", {}, 5s), "roll back raw upgrade rehearsal"); + require(take(load_region_checkpoint(*f.db, f.key, 5s), "raw rollback head").head == + original.head, + "rolled-back upgrade moved the durable head"); + + f.head = take(migrate_cellular_to_population(*f.db, *f.objects, f.lease, f.head, f.id(), + f.build, {true, 1200, 1800}, 5s), + "population accounting source entry"); + recover(f); + const auto member = std::ranges::find_if(f.population->state().lives(), [](const auto& life) { + return life.account == LifeAccount{}; + }); + require(member != f.population->state().lives().end(), "accounting source has no wildlife"); + auto collapsed = take( + prepare_population_conversion(*f.population, f.header(0), {member->identity.genome, true}), + "accounting source cohort"); + f.head = take(publish_population_batch(*f.db, *f.objects, f.lease, f.head, collapsed.batch, + *collapsed.state, f.id(), f.build, 5s), + "accounting source membership publication"); + recover(f); + require(!f.population->members().empty() && f.population->conversion_sequence() != 0, + "accounting source lost cohort history"); + auto interval = unattended(f); + take(admit_population_interval(*f.db, f.key, f.lease, f.head.head, + interval.batch.header.end_tick, 5s), + "upgrade pending interval"); + auto pending = + migrate_cellular_boundary_profile(*f.db, *f.objects, f.lease, f.head, f.id(), f.build, 5s); + require(!pending && std::string(pending.error().store.cause.sqlstate.data(), 5) == "EVL18", + "accounting upgrade discarded admitted work"); + f.head = take(publish_population_batch(*f.db, *f.objects, f.lease, f.head, interval.batch, + *interval.state, f.id(), f.build, 5s), + "settle pre-upgrade work"); + recover(f); + const auto before = f.head; + const auto source = take(decode_manifest(before.manifest), "accounting source manifest"); + const auto upgraded_population = + take(f.population->migration_upgrade_candidate(), "cohort upgrade oracle"); + const auto expected = upgraded_population->snapshot(); + const auto altered = + take(CellularPopulation::create(restore(upgraded_population->state()), {false, 1200, 1800}), + "valid unequal upgrade target"); + require(altered->state().current_total() == upgraded_population->state().current_total() && + altered->snapshot() != expected, + "unequal upgrade fixture must retain biological totals"); + const auto altered_object = take(f.objects->put(altered->snapshot()), "unequal upgrade object"); + const auto altered_manifest = take( + cellular_boundary_upgrade_manifest(source, before.head.digest, f.id(), f.lease, f.build, + altered_object.bytes, altered_object.sha256), + "unequal upgrade manifest"); + take(f.db->query("BEGIN", {}, 5s), "unequal upgrade recovery rehearsal"); + take(publish_cellular_profile_migration( + *f.db, take(encode_manifest(altered_manifest), "unequal upgrade encoding"), 5s), + "metadata-only unequal upgrade publication"); + const auto unequal = + recover_population_checkpoint(*f.db, *f.objects, f.key, f.lease, builds, 5s); + take(f.db->query("ROLLBACK", {}, 5s), "undo unequal upgrade rehearsal"); + require(!unequal && unequal.error().kind == RecoveryError::history && + take(load_region_checkpoint(*f.db, f.key, 5s), "head after unequal upgrade").head == + before.head, + "upgrade recovery compared only aggregate stock or rewrote the retained source"); + const auto child = ::fork(); + require(child >= 0, "upgrade pre-transaction fork"); + if (child == 0) { + try { + auto db = connection(); + PublishHooks hooks; + hooks.before_transaction = [] { ::kill(::getpid(), SIGKILL); }; + [[maybe_unused]] const auto result = migrate_cellular_boundary_profile( + *db, *f.objects, f.lease, before, identity(60000), f.build, 5s, hooks); + } catch (...) { + ::_exit(31); + } + ::_exit(32); + } + int status{}; + require(::waitpid(child, &status, 0) == child && WIFSIGNALED(status) && + WTERMSIG(status) == SIGKILL, + "upgrade child missed pre-transaction kill"); + recover(f); + require(f.head.head == before.head, "unpublished upgrade became authoritative"); + + auto locker = connection(true); + take(locker->query("BEGIN", {}, 5s), "accounting ambiguity lock transaction"); + take(locker->query("SELECT 1 FROM evolution.region_leases WHERE world_id=30201 " + "AND region_id=1 FOR UPDATE", + {}, 5s), + "hold accounting reply"); + auto timed = connection(); + auto unknown = migrate_cellular_boundary_profile(*timed, *f.objects, f.lease, before, f.id(), + f.build, 300ms); + require(!unknown && unknown.error().kind == PublicationError::unknown && + !unknown.error().prepared.manifest.empty(), + "uncertain accounting upgrade lost its exact request"); + take(locker->query("ROLLBACK", {}, 5s), "release accounting reply"); + auto resolver = connection(); + f.head = take(retry_cellular_profile_migration(*resolver, unknown.error().prepared, 5s), + "retry exact accounting upgrade"); + require(take(retry_cellular_profile_migration(*resolver, unknown.error().prepared, 5s), + "duplicate accounting upgrade") + .head == f.head.head, + "duplicate accounting upgrade advanced head"); + recover(f); + require(f.population->snapshot() == expected && f.head.head.end_tick == before.head.end_tick && + take(decode_manifest(f.head.manifest), "upgraded population manifest") + .cellular_versions == cellular_v10_versions, + "accounting upgrade changed biology, cohort history or time"); + auto changed = unknown.error().prepared; + auto changed_manifest = take(decode_manifest(changed.manifest), "changed accounting manifest"); + changed_manifest.payload_sha256[0] ^= 1; + changed.manifest = take(encode_manifest(changed_manifest), "changed accounting bytes"); + auto conflict = retry_cellular_profile_migration(*resolver, changed, 5s); + require(!conflict && conflict.error().store.kind == LeaseStoreError::commit_conflict, + "accounting retry accepted changed bytes"); + require( + !migrate_cellular_boundary_profile(*f.db, *f.objects, f.lease, f.head, f.id(), f.build, 5s), + "accounting upgrade applied twice"); + const auto path = f.objects->directory() / hexadecimal(source.payload_sha256, false); + const auto backup = path.string() + ".upgrade-test"; + std::filesystem::rename(path, backup); + const auto missing = + recover_population_checkpoint(*f.db, *f.objects, f.key, f.lease, builds, 5s); + std::filesystem::rename(backup, path); + require(!missing && missing.error().kind == RecoveryError::tail, + "accounting recovery trusted aggregate stock without its retained source"); + missing_and_corrupt_objects(f); + const auto stale = f.lease; + take(release_region_lease(*f.db, f.key, f.lease, 5s), "release accounting writer"); + f.lease = + take(acquire_region_lease(*f.db, f.key, f.build, 60000ms, 5s), "replace accounting writer"); + auto stale_retry = retry_cellular_profile_migration(*f.db, unknown.error().prepared, 5s); + require(!stale_retry && stale_retry.error().store.kind == LeaseStoreError::fenced && + !recover_population_checkpoint(*f.db, *f.objects, f.key, stale, builds, 5s), + "old accounting receipt granted current authority"); + recover(f); + require(f.population->snapshot() == expected, "new epoch changed accounting upgrade"); + auto future_interval = unattended(f); + auto old_trace = future_interval.batch; + old_trace.format_version = 1; + const auto old_bytes = take(encode_population_batch(old_trace), "wrong old trace fixture"); + const auto upgraded_manifest = take(decode_manifest(f.head.manifest), "new accounting source"); + auto wrong_trace_manifest = + take(population_successor_manifest( + upgraded_manifest, f.head.head.digest, f.id(), f.lease, old_trace.header.end_tick, + f.build, future_interval.state->snapshot().size(), + take(sha256(future_interval.state->snapshot()), "new payload SHA"), + {old_bytes.size(), take(sha256(old_bytes), "old trace SHA"), 0, {}}), + "wrong old trace manifest fixture"); + const PopulationPreparedCommit wrong_trace{ + f.key, wrong_trace_manifest.commit, + take(encode_manifest(wrong_trace_manifest), "wrong old trace manifest"), old_bytes}; + auto refused_old = retry_population_prepared(*f.db, wrong_trace, 5s); + auto refused_old_sql = + publish_population_checkpoint(*f.db, wrong_trace.manifest, old_bytes, 5s); + require(!refused_old && refused_old.error().kind == PublicationError::candidate && + !refused_old_sql && refused_old_sql.error().kind == LeaseStoreError::invalid && + take(load_region_checkpoint(*f.db, f.key, 5s), "head after old trace").head == + f.head.head, + "bundle 10 reinterpreted retained population trace 1"); + // Exercise the existing publication/process-loss matrix with batch 2 as well. + process_loss(f, 0); + process_loss(f, 1); + process_loss(f, 2); + isolation_and_reverse(f); + require(f.raw->format_version() == cellular_lives_migration_format, + "reverse transition downgraded accounting owner"); +} void host_ownership_and_admission(Fixture& f) { take(release_region_lease(*f.db, f.key, f.lease, 5s), "release fixture host authority"); const auto make_host = [&](bool enabled) { @@ -683,6 +885,7 @@ void run_population_publication_tests() { process_loss(fixture, 2); missing_and_corrupt_objects(fixture); isolation_and_reverse(fixture); + boundary_upgrades(fixture); host_ownership_and_admission(fixture); std::puts("population publication, exact recovery, process loss and fencing passed"); } diff --git a/tests/region_migration_tests.cpp b/tests/region_migration_tests.cpp index ba31cf3..bb22b2a 100644 --- a/tests/region_migration_tests.cpp +++ b/tests/region_migration_tests.cpp @@ -211,6 +211,7 @@ void upgrade_across_migration_and_barrier() { " -v migration_sha_43=$(s 0043_cellular_v9_foraging_bundle.sql)" " -v migration_sha_44=$(s 0044_exact_population_publication.sql)" " -v migration_sha_45=$(s 0045_owned_population_intervals.sql)" + " -v migration_sha_46=$(s 0046_boundary_accounting_upgrade.sql)" " -v runtime_role=evo_runtime_test -f migrations/apply.sql" " > /dev/null 2>&1") != 0); CHECK(history(*owner_db) == "1,2,3,4,5,6,7"); @@ -226,8 +227,9 @@ void upgrade_across_migration_and_barrier() { CHECK(shell("PGDATABASE=" + database + " EVO_PG_RUNTIME_ROLE=evo_runtime_test bash \"$EVO_SOURCE_ROOT/" "tools/migrate-postgres.sh\" > /dev/null") == 0); - CHECK(history(*owner_db) == "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24," - "25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45"); + CHECK(history(*owner_db) == + "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24," + "25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46"); CHECK(scalar(*owner_db, "SELECT (c.condeferrable AND c.condeferred AND " "c.confrelid = 'evolution.region_commits'::regclass)::text " "FROM pg_catalog.pg_constraint c WHERE c.conrelid = " diff --git a/tools/ci/postgres-check.sh b/tools/ci/postgres-check.sh index 53ae93e..cf7b845 100644 --- a/tools/ci/postgres-check.sh +++ b/tools/ci/postgres-check.sh @@ -129,7 +129,7 @@ PGDATABASE=upgrade_path psql -X -v ON_ERROR_STOP=1 -1 -v "migration_sha=$v1_sha" -f "$root/migrations/0001_region_bootstrap.sql" > /dev/null test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',') FROM evolution.schema_migrations")" = 1 PGDATABASE=upgrade_path bash "$root/tools/migrate-postgres.sh" -test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45 +test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46 # v2 heads have no payload binding: v2->v3 fails closed and leaves history at v2. v2_sha="$(sha256sum "$root/migrations/0002_region_fencing.sql")" v2_sha="${v2_sha%% *}" @@ -150,7 +150,7 @@ PGDATABASE=v3_commits psql -X -v ON_ERROR_STOP=1 \ -c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \ -c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '01000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32), 'hex'))" > /dev/null PGDATABASE=v3_commits bash "$root/tools/migrate-postgres.sh" -test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 45:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45 +test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 46:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46 test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) FROM evolution.region_commits")" = 1 # Review P1: a v4 database holding a published (possibly terminal) checkpoint has # no durable decisions for it. The v5 upgrade must refuse before changing anything. @@ -204,7 +204,7 @@ PGDATABASE=v6_populated psql -X -v ON_ERROR_STOP=1 \ -c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \ -c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '02000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32) || repeat('00', 80), 'hex'))" > /dev/null PGDATABASE=v6_populated bash "$root/tools/migrate-postgres.sh" -test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45 +test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46 test "$(PGDATABASE=v6_populated psql -XAtc "SELECT count(*) FROM evolution.region_commits c, LATERAL evolution.parse_checkpoint_manifest(c.manifest) m WHERE m.commit_sequence = 0 AND m.predecessor_commit IS NULL")" = 1 test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(conname, ',') FROM pg_constraint WHERE conrelid = 'evolution.region_heads'::regclass AND contype = 'c' AND pg_get_constraintdef(oid) LIKE '%end_tick%'")" = region_heads_published_tick phase permission-audit @@ -257,9 +257,9 @@ expect_audit_failure 'REVOKE EXECUTE ON FUNCTION evolution.claim_account_control bash "$root/tools/migrate-postgres.sh" # Restored configuration passes the audit again. phase checksum-drift # Unknown future versions and checksum drift must not be repaired silently. -psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=46 WHERE version=45' +psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=47 WHERE version=46' if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi -psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=45 WHERE version=46" +psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=46 WHERE version=47" psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=14" if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=13" diff --git a/tools/migrate-postgres.sh b/tools/migrate-postgres.sh index 2217f1a..ffcf6f1 100644 --- a/tools/migrate-postgres.sh +++ b/tools/migrate-postgres.sh @@ -102,6 +102,8 @@ digest_44="$(sha256sum "$root/migrations/0044_exact_population_publication.sql") digest_44="${digest_44%% *}" digest_45="$(sha256sum "$root/migrations/0045_owned_population_intervals.sql")" digest_45="${digest_45%% *}" +digest_46="$(sha256sum "$root/migrations/0046_boundary_accounting_upgrade.sql")" +digest_46="${digest_46%% *}" psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \ --set="migration_sha_2=$digest_2" --set="migration_sha_3=$digest_3" \ --set="migration_sha_4=$digest_4" --set="migration_sha_5=$digest_5" \ @@ -140,5 +142,6 @@ psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \ --set="migration_sha_43=$digest_43" \ --set="migration_sha_44=$digest_44" \ --set="migration_sha_45=$digest_45" \ + --set="migration_sha_46=$digest_46" \ --set="runtime_role=$runtime_role" \ --file="$root/migrations/apply.sql"