M12.7: require registered regions in census snapshots
Foundation / checks (deploy) (push) Successful in 58s
Foundation / checks (ecs) (push) Successful in 2m36s
Foundation / checks (network) (push) Successful in 20m56s
Foundation / checks (client) (push) Successful in 22m19s
Foundation / checks (tsan) (push) Successful in 25m18s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (checks) (push) Canceled after 44m49s
Foundation / checks (postgres) (push) Canceled after 44m54s
Foundation / checks (deploy) (push) Successful in 58s
Foundation / checks (ecs) (push) Successful in 2m36s
Foundation / checks (network) (push) Successful in 20m56s
Foundation / checks (client) (push) Successful in 22m19s
Foundation / checks (tsan) (push) Successful in 25m18s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (checks) (push) Canceled after 44m49s
Foundation / checks (postgres) (push) Canceled after 44m54s
This commit is contained in:
@@ -140,7 +140,9 @@ disjoint resident identities before summing.
|
||||
[Consistent scoped snapshots](schemas/ecology/consistent-cellular-census-snapshot-v1.md)
|
||||
capture declared current heads in one short database snapshot, then fully replay
|
||||
the immutable cuts; concurrent atomic migration cannot split a selected pair.
|
||||
Complete world/branch reconciliation,
|
||||
Schema 50's [registered-region coverage](schemas/ecology/registered-world-census-coverage-v1.md)
|
||||
requires every region in the durable authority registry, including unavailable
|
||||
owners, within the same selected database view. Complete world/branch reconciliation,
|
||||
client queries and terminal extinction remain later gates.
|
||||
`apps/evo-sim` owns CLI/file I/O
|
||||
and reporting. `tests` holds contract,
|
||||
|
||||
@@ -1,5 +1,23 @@
|
||||
# Project status and next steps
|
||||
|
||||
## M12.7a registered-world census coverage - 2026-10-18
|
||||
|
||||
Schema 50 now uses the durable authority registry as the required regional set
|
||||
for consistent census snapshots. Every registered region must have matching
|
||||
resources and recoverable evidence, including inactive/expired owners. Missing
|
||||
heads, unsupported profiles, unknown worlds and more than 64 regions fail the
|
||||
query. Catalog selection and head capture use the same database snapshot, with
|
||||
complete immutable replay afterward; no duplicate catalog/enrollment write.
|
||||
|
||||
Native Release passes **79/79**; registered coverage/enrollment and raw/cohort
|
||||
migration races, ASan/UBSan TLS/storage faults, Clang analysis and formatting pass.
|
||||
Complete private persistence/process/backup/DR, schema upgrade, permissions and
|
||||
checksum qualification passes in **551 s** after correcting a stale schema-history
|
||||
test expectation. [Evidence and limits](engineering/VALIDATION.md) retain external
|
||||
branch identity, other viable stores, durable accepted reconciliation records,
|
||||
client access and the ecology/soak/product gates. This segment accepts no terminal
|
||||
extinction, deployment or World0 promotion.
|
||||
|
||||
## M12.6d consistent scoped census snapshots - 2026-10-18
|
||||
|
||||
Scoped census reads now select all declared regional heads from one short
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# ADR-0154: Registered-world coverage for cellular census snapshots
|
||||
|
||||
## Status
|
||||
|
||||
Accepted for scoped M12.7a, 2026-10-18. Implements database-registered regional
|
||||
coverage under ADR-0133/0153, not terminal extinction or external branch identity.
|
||||
|
||||
## Context and decision
|
||||
|
||||
A consistent snapshot of a caller-declared list can still omit an owner. The
|
||||
existing durable `region_leases` registry is the source of region identity and
|
||||
frame for all supported provisioning paths. Duplicating it in a separate census
|
||||
catalog would create another enrollment truth and a failure window.
|
||||
|
||||
Schema 50 adds a least-privilege read function listing registered regions of one
|
||||
world, ordered by region ID, with frame IDs. Read from the registry itself, without
|
||||
joining/filtering on heads, profile, active holder, expiry, activity or living count.
|
||||
A missing checkpoint or unsupported region must prevent evidence completion,
|
||||
not disappear from the required list. Return at most 65 rows: the 65th detects
|
||||
overflow of the existing 64-region query bound. This is not a selected world size.
|
||||
No new table/index, registration write or biological codec/rule.
|
||||
|
||||
Add `load_registered_cellular_census_snapshot`. In ADR-0153's owned short read-only
|
||||
repeatable-read transaction, read this registry before selecting any heads. Require
|
||||
an exact match to the supplied region resources/grants, including frames. Empty
|
||||
registry means unknown world coverage, not zero life. Over-limit worlds fail;
|
||||
never truncate/paginate a subset and label it complete. Read every selected head
|
||||
in the same database snapshot, close the metadata transaction, then use unchanged
|
||||
exact-cut replay and scoped census validation. Return registry evidence alongside
|
||||
the exact cuts and census. Existing caller-declared snapshots keep their semantics.
|
||||
|
||||
Concurrent enrollment after the registry read belongs to a later snapshot. The
|
||||
current result covers the registered set as of its captured database view, not
|
||||
the latest state at return. Uncommitted enrollment is not included. A registered
|
||||
row without a recoverable head fails the whole result. No fallback or zero filling.
|
||||
|
||||
## Alternatives, benefits, costs and risks
|
||||
|
||||
Trusting a supplied list cannot prove coverage. Listing only checkpoint-bearing,
|
||||
cellular, active or unexpired regions hides unavailable or unsupported owners.
|
||||
A duplicate mutable catalog can disagree with the authority registry. Read the
|
||||
existing durable registry through its `(world_id, region_id)` primary-key prefix;
|
||||
the 65-row limit bounds listing work and detects oversized sets explicitly.
|
||||
|
||||
The new security-definer read function exposes only region/frame metadata to the
|
||||
existing trusted runtime group. Fixed search path, PUBLIC revocation and the full
|
||||
runtime privilege audit remain mandatory. It grants no table access, provisioning,
|
||||
new client permission or cross-world biological authority. Old schemas fail the
|
||||
new query instead of falling back to caller-declared coverage.
|
||||
|
||||
## Consequences and qualification
|
||||
|
||||
Test exact sets, omitted/extra/duplicate resources, unknown worlds, wrong frames,
|
||||
expired/released holders, missing heads, unsupported profiles and 64/65 bounds.
|
||||
Race committed and uncommitted enrollment with snapshot capture; current cuts
|
||||
must retain their selected catalog and later reads must see new enrollment.
|
||||
Repeat real raw/cohort atomic migration races under registered coverage. Retain
|
||||
failure cleanup, TLS faults, object verification and no-partial-result behavior.
|
||||
Run migration/no-op/checksum/permissions, full private persistence regression,
|
||||
sanitizers, Clang analysis/format and default-build checks. Never alter prior
|
||||
migration bytes or biological goldens.
|
||||
|
||||
This covers registered regions in one connected database view. A rewind can reuse
|
||||
the world ID; external branch identity is still missing. Outside/unsupported stores,
|
||||
viable reserves, durable accepted reconciliation records and global freshness remain
|
||||
M12.7 work. No terminal-extinction outcome, client endpoint, live database,
|
||||
deployment or World0 action.
|
||||
@@ -1,5 +1,7 @@
|
||||
# Architecture decision records
|
||||
|
||||
- [ADR-0154: Registered-world census coverage](0154-registered-world-census-coverage.md) - schema 50 uses the existing durable authority registry as the required regional set in a consistent snapshot; omitted/unavailable/over-bound owners fail without a new catalog table or extinction authority.
|
||||
|
||||
- [ADR-0153: Consistent scoped cellular census snapshots](0153-consistent-cellular-census-snapshots.md) - short read-only repeatable-read head capture followed by exact immutable replay; atomic pairs cannot straddle a declared-region cut, without claiming world/branch completeness.
|
||||
|
||||
- [ADR-0152: Scoped multi-region cellular census queries](0152-scoped-cellular-census-sets.md) - bounded declared coverage, exact boundaries, complete shared-genome equality and disjoint resident identities; no world-completeness or extinction authority.
|
||||
|
||||
@@ -1,5 +1,13 @@
|
||||
# Persistence and region authority
|
||||
|
||||
M12.7a/schema 50 adds [registered-world coverage](../../schemas/ecology/registered-world-census-coverage-v1.md).
|
||||
The existing authority registry supplies every required region/frame, without
|
||||
activity, expiry, profile or checkpoint filters. The bounded read function and
|
||||
head selection share one database snapshot; resources must match the entire
|
||||
registered set. Missing/unusable owners and a 65th region fail the query. There
|
||||
is no new catalog table or registration write. External branch identity, other
|
||||
viable stores and accepted global extinction reconciliation remain separate.
|
||||
|
||||
M12.6d captures declared regional heads in one short read-only repeatable-read
|
||||
transaction, closes it, then fully replays each exact immutable cut before
|
||||
constructing a scoped census. Atomic migration commits cannot split the selected
|
||||
|
||||
@@ -29,6 +29,13 @@ when both regions are declared. Durable required-world coverage, outside transfe
|
||||
and reserve reconciliation, external branch identity and extinction acceptance
|
||||
remain separate; this is no complete world evidence envelope.
|
||||
|
||||
M12.7a's [registered-region coverage](../../schemas/ecology/registered-world-census-coverage-v1.md)
|
||||
now checks supplied resources against every durable registered region in that
|
||||
same database view. It includes released/expired, empty, unsupported and
|
||||
unavailable owners; no missing-as-zero or caller omission can complete the result.
|
||||
This still lacks external branch identity, other viable-store reconciliation and
|
||||
a durable accepted global extinction record. It supplies no terminal conclusion.
|
||||
|
||||
## Identity and classification
|
||||
|
||||
An organism has a permanent identity and terminal death. An immutable genome has
|
||||
|
||||
@@ -14,7 +14,7 @@ Commit and push each validated scoped segment separately.
|
||||
| 12.4 | Owned intervals and durable paired migration/fault qualification implemented in scoped segments through 12.4l | ADR-0143–0149/schema 49; bounded admitted work, source freeze, staged ghost, atomic pair and exact recovery, local TLS and storage faults. Multi-host/orchestrator, linked ecology and integrated stock audits remain open. |
|
||||
| 12.5 | Blocked on 12.2-12.4 | Finite M6 process fixtures do not qualify coupled persistent cellular/cohort atmosphere/water budgets. |
|
||||
| 12.6 | Regional queries, replayed history, scoped multi-region queries and consistent database head snapshots implemented (12.6a-d) | ADR-0150–0153; short read-only snapshot capture prevents atomic pairs from straddling declared head sets, with exact immutable replay. Durable world/branch coverage, outside transfer/viability reconciliation, target-host query/index costs and client path remain required. |
|
||||
| 12.7 | Evidence policy defined; reconciler open | Complete durable world watermark and every viable/in-flight store remain required. |
|
||||
| 12.7 | Evidence policy and registered-region coverage implemented (12.7a) | ADR-0154/schema 50 requires the durable authority registry in one consistent database cut. External branch identity, other viable/in-flight stores and durable accepted global reconciliation remain required. |
|
||||
| 12.8 | Not qualified | Actual multi-day integrated ecology/restart/clone evidence remains required. |
|
||||
| 12.9 | Not accepted | Returning-player product evidence and expansion review remain required; World0 stays separately blocked. |
|
||||
|
||||
|
||||
@@ -27,8 +27,8 @@ or authorize production enablement.
|
||||
| 12.3 | Dedicated population profile/trace/store/owning host; exact recovery and private PostgreSQL fault tests | Population-specific executable/network fault drills and target-host publication/capacity qualification. |
|
||||
| 12.4 | Owned intervals; exact transfer/accounting; durable atomic pair/recovery; independent loopback TLS path loss and owner expiry/replacement/heal; local object sync/read-failure matrix | Multi-host/kernel-network, storage-host and orchestrator drills, linked-boundary contracts and integrated ownership/stock audits at scale. |
|
||||
| 12.5 | Finite cellular contact feeding/decay and separate M6 producer/consumer resource owner | One jointly owned cellular/environment budget, explicit effective depth and source entitlements, versioned water/atmosphere species mapping and producer coefficients, plus closed/open and nutrient-exhaustion comparisons. |
|
||||
| 12.6 | Retained archive, regional census/lineage queries, replayed history, scoped multi-region queries and consistent database head snapshots (12.6a-d) | Durable world/branch history context and required-region coverage, outside transfer/viability reconciliation, target-host query/index qualification and client permission/presentation; no account power or invented precise statistical ancestry. |
|
||||
| 12.7 | ADR-0133's viability/unknown semantics | Durable region-set/watermark reconciliation covering every owned representation, suspended/viable stock and migration escrow. Missing/stale regions must prevent terminal extinction. |
|
||||
| 12.6 | Retained archive, regional census/lineage queries, replayed history, scoped multi-region queries and consistent database head snapshots (12.6a-d); registered coverage follows in 12.7a | External branch history context, retained historical coverage/other viable stores, target-host query/index qualification and client permission/presentation; no account power or invented precise statistical ancestry. |
|
||||
| 12.7 | Viability/unknown policy; consistent head cuts and durable registered-region coverage (12.7a/schema 50) | External world/branch context, other viable-store/transfer reconciliation and durable accepted global evidence records. Missing/stale/unsupported owners must prevent terminal extinction. |
|
||||
| 12.8 | Deterministic seed/regime drivers and separate-process exact continuation | Reproducible integrated worlds running for actual days, real restarts/clone comparisons and retained accounting/diversity/cost audits. Accelerated tests cannot supply elapsed days. |
|
||||
| 12.9 | M11 feedback and current UI review retained | Returning-player samples on the integrated ecology and explicit product/3D readiness decision after soak evidence. No M12 product acceptance or World0 promotion is inferred. |
|
||||
|
||||
@@ -121,6 +121,15 @@ The short metadata transaction ends before object I/O and fenced recovery; faile
|
||||
evidence still returns no partial census. Durable world/branch identity, required
|
||||
owner coverage, outside transfers and viable-store reconciliation remain open.
|
||||
|
||||
M12.7a/schema 50's [registered coverage](../../schemas/ecology/registered-world-census-coverage-v1.md)
|
||||
now reads the existing durable authority registry before head selection in that
|
||||
same database snapshot. Every registered region is required, including inactive,
|
||||
expired, unsupported and unavailable owners. Unknown worlds and a 65th region
|
||||
fail rather than becoming zero or a truncated complete set. Concurrent later
|
||||
enrollment belongs to a later cut. No duplicate catalog/enrollment write is added.
|
||||
External branch identity, other viable stores and durable accepted reconciliation
|
||||
records remain open; this does not close M12.7 or authorize terminal extinction.
|
||||
|
||||
Current expansion readiness: **no-go for M13 integration or World0 promotion**
|
||||
while these M12 integration/evidence gates remain open. This records the present
|
||||
engineering evidence, not a substitute for the later maintainer product review.
|
||||
|
||||
@@ -1,5 +1,16 @@
|
||||
# Validation strategy
|
||||
|
||||
M12.7a's schema-50 tests require exact authority-registry coverage, including
|
||||
released/expired holders, unavailable heads and unsupported profiles. Check
|
||||
omitted/extra/duplicate resources, wrong frames, unknown worlds, integral numeric
|
||||
IDs across the full u64 range and the 64/65-region bound. Race committed enrollment
|
||||
after catalog selection, and exclude another connection's uncommitted enrollment.
|
||||
Registered raw/cohort migration snapshots retain the same catalog and preserve
|
||||
exact cut/stock evidence across real paired commits. Exercise both snapshot APIs
|
||||
under TLS BEGIN/COMMIT loss. Qualify checksum-preserving 1..50 upgrades/no-ops,
|
||||
runtime execute revocation/audit, backup/restore/DR, sanitizers and analysis. No
|
||||
test can promote registered coverage to terminal extinction or branch identity.
|
||||
|
||||
M12.6d's private PostgreSQL snapshot tests publish a real atomic migration from
|
||||
an independent connection between the first and second captured head reads.
|
||||
Both selected cuts must remain before the pair; the next snapshot sees both
|
||||
|
||||
@@ -1,5 +1,82 @@
|
||||
# Validation report
|
||||
|
||||
## 2026-10-18 - M12.7a registered-world census coverage
|
||||
|
||||
[ADR-0154](../adr/0154-registered-world-census-coverage.md)/schema 50 uses the
|
||||
existing durable authority registry as the required region set. It never filters
|
||||
on checkpoint availability, profile, activity or lease expiry. A registered query
|
||||
reads that catalog and all heads in one short read-only repeatable-read snapshot,
|
||||
requires exact region/frame resource coverage, then fully replays every immutable
|
||||
cut after metadata completion. Returned catalog evidence remains scoped to that
|
||||
connected database view, without a terminal-extinction or external branch claim.
|
||||
|
||||
Executed locally:
|
||||
|
||||
- MSVC **19.44.35217.0** default Release full CTest **79/79 passed**, **13.91 s**.
|
||||
- GCC 14 Release private population/history/registered-coverage qualification
|
||||
passed **exit 0, 17 s** (15 s population phase), with final normalized catalog
|
||||
parsing and full unsigned-64-bit/decimal-integral ID fixtures.
|
||||
- Clang 19 Linux **ASan/UBSan** final combined registered coverage, history,
|
||||
migration, TLS and storage fault matrix passed **exit 0, 71 s** (67 s population
|
||||
phase), without sanitizer reports.
|
||||
- Explicit clang-tidy 19 covered the shared snapshot/coverage adapter, population
|
||||
integration tests and migration-history tests; final changed adapter/assertions
|
||||
were rechecked with no owned warnings. Repository clang-format 19,
|
||||
`bash -n` for both changed shell scripts and `git diff --check` passed.
|
||||
|
||||
The first full frozen-binary attempt exited 1 after **437 s**, at the online
|
||||
migration test's stale expected schema-49 history. The database correctly reached
|
||||
50. That assertion now explicitly includes the accepted schema-50 addition;
|
||||
prior migration bytes and biological goldens are unchanged. The corrected full
|
||||
persistence/process/backup/DR and SQL rerun passed with **exit 0, 551 s**: C++
|
||||
integration/process/load/history **435 s**, registered/migration TLS/storage
|
||||
faults **41 s**, life reservations **4 s**, backup/restore **25 s**, host-loss DR
|
||||
**8 s**, upgrade **10 s**, migration history **5 s**, runtime permission audit
|
||||
**11 s**, checksum/future-version rejection **8 s**, plus setup/migration. The
|
||||
fresh frozen-binary runner recorded its successful completion sentinel. The
|
||||
permission checks include revoking the new registry read grant and requiring
|
||||
the effective-privilege audit to reject that configuration.
|
||||
|
||||
Real empty registered regions exercise exact coverage, omitted/extra/duplicate
|
||||
resources, wrong frames, unknown worlds, released/expired holders and the 64/65
|
||||
bound. The SQL listing retains the overflow sentinel. A numeric-domain fixture
|
||||
uses decimal-integral IDs/frames and region UINT64_MAX without narrowing; rows
|
||||
with missing heads fail rather than disappearing. A non-cellular registered
|
||||
profile fails unsupported before any payload inference. Genesis inclusion tests
|
||||
do not claim fresh authority from expired/released grants; genesis has no fenced
|
||||
transition evidence to read.
|
||||
|
||||
Independent committed enrollment after catalog selection leaves the earlier cut
|
||||
unchanged; the next query refuses the now-incomplete supplied list. Another
|
||||
connection's uncommitted enrollment is excluded. Registered raw/cohort snapshots
|
||||
also retain identical catalog/old-cut evidence across actual paired commits,
|
||||
while later snapshots see both new heads and preserve living count/body/reserve.
|
||||
Both declared and registered APIs fail closed under real loopback TLS request/
|
||||
reply loss at BEGIN and COMMIT. Caller transaction and exception/cleanup guards
|
||||
retain their prior behavior. No partial result or missing-as-zero fallback.
|
||||
|
||||
Schema 50 is additive: one bounded read function, no new catalog table/index or
|
||||
registration write. Prior migrations 1..49 remain unchanged. Migration tooling,
|
||||
rollback-history fixtures, runtime privilege audit, latest-prefix assertions and
|
||||
unknown-future-version rejection now include 50. No biological rule, payload
|
||||
format, golden, dependency, authority workflow or client endpoint change.
|
||||
|
||||
Limits: registry coverage is for the selected database view, not latest state at
|
||||
return or an external branch identity. Other viable stores, accepted permanent
|
||||
reconciliation records and complete global freshness remain open. No deployment-
|
||||
host query/capacity claim, new TSan, remote CI, multi-host drill or actual multi-day
|
||||
soak. No live database, deployment or World0 action. Private test clusters stop on
|
||||
exit and retain their forensic directories.
|
||||
|
||||
Logs: `build/m127a-native.log`, `build/m127a-final-postgres-build.log`,
|
||||
`build/m127a-final-postgres.log`, `build/m127a-final-asan-build.log`,
|
||||
`build/m127a-final-asan.log`, `build/m127a-tidy.log`,
|
||||
`build/m127a-final-tidy.log`, `build/m127a-final-format.log`,
|
||||
`build/m127a-shell-syntax.log`, `build/m127a-full-postgres.log`,
|
||||
`build/m127a-full-postgres.exit`, `build/m127a-recheck-build.log`,
|
||||
`build/m127a-recheck-tidy.log`, `build/m127a-recheck-format.log`,
|
||||
`build/m127a-full-recheck.log`, `build/m127a-full-recheck.exit` (completion only).
|
||||
|
||||
## 2026-10-18 - M12.6d consistent scoped census snapshots
|
||||
|
||||
[ADR-0153](../adr/0153-consistent-cellular-census-snapshots.md) captures the current
|
||||
|
||||
@@ -21,13 +21,40 @@ struct CellularCensusSnapshot {
|
||||
std::vector<CommittedCheckpoint> cuts; // Sorted regions, exact captured head/manifest pairs.
|
||||
server::CellularCensusSet census;
|
||||
};
|
||||
enum class CellularSnapshotError : std::uint8_t { invalid_request, transaction, history, census };
|
||||
enum class CellularSnapshotError : std::uint8_t {
|
||||
invalid_request,
|
||||
transaction,
|
||||
history,
|
||||
census,
|
||||
coverage
|
||||
};
|
||||
struct RegisteredCellularRegion {
|
||||
std::uint64_t region{}, frame{};
|
||||
bool operator==(const RegisteredCellularRegion&) const = default;
|
||||
};
|
||||
enum class CellularCoverageError : std::uint8_t {
|
||||
store,
|
||||
unknown_world,
|
||||
bounds,
|
||||
regions,
|
||||
frame,
|
||||
malformed
|
||||
};
|
||||
struct CellularCoverageFailure {
|
||||
CellularCoverageError kind{};
|
||||
PgFailure cause{}; // Meaningful for store.
|
||||
};
|
||||
struct RegisteredCellularCensusSnapshot {
|
||||
std::vector<RegisteredCellularRegion> catalog; // Exact registry view used for head selection.
|
||||
CellularCensusSnapshot snapshot;
|
||||
};
|
||||
struct CellularSnapshotFailure {
|
||||
CellularSnapshotError kind{};
|
||||
std::uint64_t region{}; // Failed region, or zero for a request/transaction/set error.
|
||||
PgFailure transaction{}; // Meaningful for transaction.
|
||||
CellularHistoryFailure history{}; // Meaningful for history.
|
||||
server::CellularCensusSetError census{}; // Meaningful for census.
|
||||
CellularCoverageFailure coverage{}; // Meaningful for coverage.
|
||||
};
|
||||
// Test/fault seam after each head has been captured (zero-based request index).
|
||||
// Production callers pass nothing. A throwing hook closes the capture connection.
|
||||
@@ -47,4 +74,16 @@ load_cellular_census_snapshot(Postgres&, std::uint64_t world,
|
||||
std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks = {});
|
||||
// Schema 50: require resources for EVERY registered region of this world,
|
||||
// including expired/released holders and regions with unavailable checkpoints.
|
||||
// Capture the registry and all heads in the SAME short database snapshot.
|
||||
// Empty/over-bound registries or any coverage/frame mismatch fail, never zero.
|
||||
// Completeness is only for registered regions of this connected database view;
|
||||
// no external branch identity, other viable-store reconciliation or extinction.
|
||||
[[nodiscard]] std::expected<RegisteredCellularCensusSnapshot, CellularSnapshotFailure>
|
||||
load_registered_cellular_census_snapshot(Postgres&, std::uint64_t world,
|
||||
std::span<const CellularCensusRegionRequest>,
|
||||
std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks = {});
|
||||
} // namespace evo::persistence
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
-- ADR-0154: the authority registry is also the source of census owner coverage.
|
||||
-- The 65th row detects overflow; callers must never treat a prefix as complete.
|
||||
CREATE FUNCTION evolution.load_world_region_catalog_v1(w evolution.u64)
|
||||
RETURNS TABLE(region_id evolution.u64,frame_id evolution.u64)
|
||||
LANGUAGE plpgsql STABLE SECURITY DEFINER SET search_path=pg_catalog,pg_temp AS $$
|
||||
BEGIN
|
||||
IF w IS NULL OR w=0 THEN
|
||||
RAISE EXCEPTION 'invalid world identity' USING ERRCODE='22023';
|
||||
END IF;
|
||||
RETURN QUERY SELECT r.region_id,r.frame_id FROM evolution.region_leases r
|
||||
WHERE r.world_id=w ORDER BY r.region_id LIMIT 65;
|
||||
END $$;
|
||||
REVOKE ALL ON FUNCTION evolution.load_world_region_catalog_v1(evolution.u64) FROM PUBLIC;
|
||||
GRANT EXECUTE ON FUNCTION evolution.load_world_region_catalog_v1(evolution.u64) TO :"runtime_role";
|
||||
INSERT INTO evolution.schema_migrations(version,sha256) VALUES(50,:'migration_sha_50');
|
||||
+13
-3
@@ -125,6 +125,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
|
||||
SELECT :'history_v46' || ',47:' || :'migration_sha_47' AS history_v47 \gset
|
||||
SELECT :'history_v47' || ',48:' || :'migration_sha_48' AS history_v48 \gset
|
||||
SELECT :'history_v48' || ',49:' || :'migration_sha_49' AS history_v49 \gset
|
||||
SELECT :'history_v49' || ',50:' || :'migration_sha_50' AS history_v50 \gset
|
||||
SELECT :'history' = :'history_v22' AS at_v22,
|
||||
:'history' = :'history_v23' AS at_v23,
|
||||
:'history' = :'history_v24' AS at_v24,
|
||||
@@ -152,8 +153,10 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
|
||||
:'history' = :'history_v46' AS at_v46,
|
||||
:'history' = :'history_v47' AS at_v47,
|
||||
:'history' = :'history_v48' AS at_v48,
|
||||
:'history' = :'history_v49' AS at_v49 \gset
|
||||
\if :at_v49
|
||||
:'history' = :'history_v49' AS at_v49,
|
||||
:'history' = :'history_v50' AS at_v50 \gset
|
||||
\if :at_v50
|
||||
\elif :at_v49
|
||||
\elif :at_v48
|
||||
\elif :at_v47
|
||||
\elif :at_v46
|
||||
@@ -545,7 +548,8 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24,
|
||||
coalesce(max(version), 0) = 46 AS at_v46,
|
||||
coalesce(max(version), 0) = 47 AS at_v47,
|
||||
coalesce(max(version), 0) = 48 AS at_v48,
|
||||
coalesce(max(version), 0) = 49 AS at_v49
|
||||
coalesce(max(version), 0) = 49 AS at_v49,
|
||||
coalesce(max(version), 0) = 50 AS at_v50
|
||||
FROM evolution.schema_migrations \gset
|
||||
\if :at_v24
|
||||
\ir 0025_cellular_v2_bundle.sql
|
||||
@@ -763,6 +767,7 @@ SELECT coalesce(max(version), 0) = 24 AS at_v24,
|
||||
\elif :at_v47
|
||||
\elif :at_v48
|
||||
\elif :at_v49
|
||||
\elif :at_v50
|
||||
\else
|
||||
\echo 'Unknown or changed migration history after v43'
|
||||
DO $$ BEGIN RAISE EXCEPTION 'unknown or changed migration history'; END $$;
|
||||
@@ -791,6 +796,10 @@ SELECT coalesce(max(version), 0) = 48 AS needs_v49 FROM evolution.schema_migrati
|
||||
\if :needs_v49
|
||||
\ir 0049_cellular_migration_publication.sql
|
||||
\endif
|
||||
SELECT coalesce(max(version), 0) = 49 AS needs_v50 FROM evolution.schema_migrations \gset
|
||||
\if :needs_v50
|
||||
\ir 0050_world_region_catalog.sql
|
||||
\endif
|
||||
-- Effective-privilege audit on every run, including no-ops (ADR-0060). Attributes
|
||||
-- alone are insufficient: predefined roles such as pg_write_all_data, PUBLIC
|
||||
-- grants or SET-able memberships would bypass the fenced functions. Checks the
|
||||
@@ -810,6 +819,7 @@ BEGIN
|
||||
UNION
|
||||
SELECT m.member FROM pg_catalog.pg_auth_members m JOIN closure c ON m.roleid = c.oid
|
||||
), fenced(oid) AS (VALUES
|
||||
('evolution.load_world_region_catalog_v1(evolution.u64)'::regprocedure::oid),
|
||||
('evolution.publish_cellular_migration(uuid,bytea,bytea,uuid,evolution.u64,uuid,evolution.u64)'::regprocedure::oid),
|
||||
('evolution.load_cellular_migration_commit(evolution.u64,evolution.u64,uuid,evolution.u64,uuid)'::regprocedure::oid),
|
||||
('evolution.stage_cellular_migration(evolution.u64,evolution.u64,uuid,evolution.u64,bytea,bytea,evolution.u64)'::regprocedure::oid),
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# Registered-world cellular census coverage v1
|
||||
|
||||
Semantic backend contract under
|
||||
[ADR-0154](../../docs/adr/0154-registered-world-census-coverage.md), not a wire or
|
||||
biological codec. API: `load_registered_cellular_census_snapshot`; SQL schema 50.
|
||||
|
||||
The durable source of required regions is the existing authority registry,
|
||||
`evolution.region_leases`. The new least-privilege read function
|
||||
`load_world_region_catalog_v1(world)` returns ascending region/frame pairs for
|
||||
one nonzero world. It reads at most 65 rows using the existing world/region key.
|
||||
There is no new table or parallel enrollment write. Missing table/function access
|
||||
is an error, not permission to fall back to a caller-declared subset.
|
||||
|
||||
All registered rows count, including released/expired holders, inactive or empty
|
||||
regions, unsupported profiles and regions lacking a usable committed head. No
|
||||
checkpoint, profile, activity or living-count filter may hide a required owner.
|
||||
Integral numeric IDs are normalized without narrowing the unsigned 64-bit domain.
|
||||
|
||||
Inside the snapshot adapter's owned short read-only repeatable-read transaction:
|
||||
|
||||
1. Read the registry before selecting heads. Empty means unknown world coverage.
|
||||
A 65th row means the world exceeds this query's 64-region work bound. Both
|
||||
cases fail; neither is an empty/partial census or a game-world size restriction.
|
||||
2. Require exactly one supplied store/grant resource for every registered region,
|
||||
with matching frame. Supplied resources remain strictly ordered/nonzero and
|
||||
must live through the call. Missing, extra, duplicate or foreign resources fail.
|
||||
3. Select every registered head and require its recorded frame to match the
|
||||
catalog. Read all metadata from the same database snapshot. Missing heads fail.
|
||||
4. Finish the metadata transaction, then fully replay each exact immutable cut
|
||||
and run the unchanged common-boundary/genome/resident census-set checks.
|
||||
5. Return the captured catalog, exact head/manifest pairs and scoped census only
|
||||
if all steps succeed. Failures return no partial evidence or zero filling.
|
||||
|
||||
Committed enrollment after catalog selection belongs to the next snapshot;
|
||||
uncommitted enrollment is invisible. The result covers the registered set of its
|
||||
selected database view, not the latest state at return. Atomic paired migrations
|
||||
cannot straddle that view. Existing fences during replay retain their meaning;
|
||||
genesis-only recovery has no fenced transition to check. Results grant no live
|
||||
authority, including when a registered holder is released or expired.
|
||||
|
||||
The caller-declared API remains available with its previous scoped semantics.
|
||||
Registered coverage adds no external branch/rewind identity, viable-reserve or
|
||||
outside-store reconciliation, durable accepted extinction record, client endpoint
|
||||
or permission. A consistent registered-region cut alone cannot accept terminal
|
||||
world extinction. Missing, stale or unsupported ecological evidence stays unknown.
|
||||
@@ -1,5 +1,9 @@
|
||||
#include "evo/persistence/cellular_census_snapshot.hpp"
|
||||
|
||||
#include <array>
|
||||
#include <charconv>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
|
||||
namespace evo::persistence {
|
||||
@@ -12,6 +16,43 @@ auto transaction_failure(PgFailure error) {
|
||||
auto history_failure(std::uint64_t region, CellularHistoryFailure error) {
|
||||
return std::unexpected(Failure{Error::history, region, {}, error, {}});
|
||||
}
|
||||
auto coverage_failure(CellularCoverageError kind, PgFailure cause = {}) {
|
||||
return std::unexpected(Failure{Error::coverage, 0, {}, {}, {}, {kind, cause}});
|
||||
}
|
||||
struct Captured {
|
||||
std::vector<RegisteredCellularRegion> catalog;
|
||||
std::vector<CommittedCheckpoint> cuts;
|
||||
};
|
||||
std::expected<std::vector<RegisteredCellularRegion>, Failure>
|
||||
catalog(Postgres& db, std::uint64_t world, std::chrono::milliseconds timeout) {
|
||||
const std::array<std::string, 1> parameters{std::to_string(world)};
|
||||
const auto rows = db.query("SELECT pg_catalog.trunc(region_id)::text,"
|
||||
"pg_catalog.trunc(frame_id)::text FROM "
|
||||
"evolution.load_world_region_catalog_v1($1::evolution.u64)",
|
||||
parameters, timeout);
|
||||
if (!rows)
|
||||
return coverage_failure(CellularCoverageError::store, rows.error());
|
||||
if (rows->empty())
|
||||
return coverage_failure(CellularCoverageError::unknown_world);
|
||||
if (rows->size() > server::max_cellular_census_regions)
|
||||
return coverage_failure(CellularCoverageError::bounds);
|
||||
const auto number = [](const std::optional<std::string>& text, std::uint64_t& value) {
|
||||
if (!text)
|
||||
return false;
|
||||
const auto [end, error] = std::from_chars(text->data(), text->data() + text->size(), value);
|
||||
return error == std::errc{} && end == text->data() + text->size() && value != 0;
|
||||
};
|
||||
std::vector<RegisteredCellularRegion> result;
|
||||
result.reserve(rows->size());
|
||||
for (const auto& row : *rows) {
|
||||
RegisteredCellularRegion entry;
|
||||
if (row.size() != 2 || !number(row[0], entry.region) || !number(row[1], entry.frame) ||
|
||||
(!result.empty() && entry.region <= result.back().region))
|
||||
return coverage_failure(CellularCoverageError::malformed);
|
||||
result.push_back(entry);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
// Exception-safe capture ownership: explicit completion disarms this guard.
|
||||
// Closing cannot allocate/throw or leave a caller with an open failed transaction.
|
||||
struct CaptureGuard {
|
||||
@@ -22,9 +63,11 @@ struct CaptureGuard {
|
||||
db.disconnect();
|
||||
}
|
||||
};
|
||||
std::expected<std::vector<CommittedCheckpoint>, Failure>
|
||||
capture(Postgres& db, std::uint64_t world, std::span<const CellularCensusRegionRequest> requests,
|
||||
std::chrono::milliseconds timeout, const CellularCensusSnapshotHooks& hooks) {
|
||||
std::expected<Captured, Failure> capture(Postgres& db, std::uint64_t world,
|
||||
std::span<const CellularCensusRegionRequest> requests,
|
||||
std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks,
|
||||
bool registered) {
|
||||
const auto begun = db.query("BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY", {}, timeout);
|
||||
if (!begun) {
|
||||
if (!db.transaction_idle())
|
||||
@@ -32,19 +75,35 @@ capture(Postgres& db, std::uint64_t world, std::span<const CellularCensusRegionR
|
||||
return transaction_failure(begun.error());
|
||||
}
|
||||
CaptureGuard guard{db};
|
||||
auto result = [&]() -> std::expected<std::vector<CommittedCheckpoint>, Failure> {
|
||||
std::vector<CommittedCheckpoint> cuts;
|
||||
cuts.reserve(requests.size());
|
||||
auto result = [&]() -> std::expected<Captured, Failure> {
|
||||
Captured captured;
|
||||
if (registered) {
|
||||
auto required = catalog(db, world, timeout);
|
||||
if (!required)
|
||||
return std::unexpected(required.error());
|
||||
if (required->size() != requests.size())
|
||||
return coverage_failure(CellularCoverageError::regions);
|
||||
for (std::size_t i = 0; i < requests.size(); ++i) {
|
||||
if ((*required)[i].region != requests[i].region)
|
||||
return coverage_failure(CellularCoverageError::regions);
|
||||
if ((*required)[i].frame != requests[i].lease.frame)
|
||||
return coverage_failure(CellularCoverageError::frame);
|
||||
}
|
||||
captured.catalog = std::move(*required);
|
||||
}
|
||||
captured.cuts.reserve(requests.size());
|
||||
for (std::size_t i = 0; i < requests.size(); ++i) {
|
||||
auto head = load_region_checkpoint(db, {world, requests[i].region}, timeout);
|
||||
if (!head)
|
||||
return history_failure(requests[i].region, {CellularHistoryError::recovery,
|
||||
{RecoveryError::store, head.error()}});
|
||||
cuts.push_back(std::move(*head));
|
||||
if (registered && head->head.frame != captured.catalog[i].frame)
|
||||
return coverage_failure(CellularCoverageError::frame);
|
||||
captured.cuts.push_back(std::move(*head));
|
||||
if (hooks.after_head)
|
||||
hooks.after_head(i);
|
||||
}
|
||||
return cuts;
|
||||
return captured;
|
||||
}();
|
||||
const auto done = db.query(result ? "COMMIT" : "ROLLBACK", {}, timeout);
|
||||
if (!done)
|
||||
@@ -54,12 +113,11 @@ capture(Postgres& db, std::uint64_t world, std::span<const CellularCensusRegionR
|
||||
guard.active = false;
|
||||
return result;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::expected<CellularCensusSnapshot, CellularSnapshotFailure> load_cellular_census_snapshot(
|
||||
Postgres& db, std::uint64_t world, std::span<const CellularCensusRegionRequest> requests,
|
||||
std::span<const Sha256Digest> builds, std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks) {
|
||||
std::expected<RegisteredCellularCensusSnapshot, Failure>
|
||||
load_snapshot(Postgres& db, std::uint64_t world,
|
||||
std::span<const CellularCensusRegionRequest> requests,
|
||||
std::span<const Sha256Digest> builds, std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks, bool registered) {
|
||||
if (!db.transaction_idle() || world == 0 || requests.empty() ||
|
||||
requests.size() > server::max_cellular_census_regions || timeout.count() <= 0 ||
|
||||
timeout.count() > 30000)
|
||||
@@ -72,14 +130,14 @@ std::expected<CellularCensusSnapshot, CellularSnapshotFailure> load_cellular_cen
|
||||
return std::unexpected(Failure{Error::invalid_request, 0, {}, {}, {}});
|
||||
required.push_back(request.region);
|
||||
}
|
||||
auto cuts = capture(db, world, requests, timeout, hooks);
|
||||
auto cuts = capture(db, world, requests, timeout, hooks, registered);
|
||||
if (!cuts)
|
||||
return std::unexpected(cuts.error());
|
||||
std::vector<server::CellularCensus> observations;
|
||||
observations.reserve(requests.size());
|
||||
for (std::size_t i = 0; i < requests.size(); ++i) {
|
||||
const auto& request = requests[i];
|
||||
const auto& cut = (*cuts)[i];
|
||||
const auto& cut = cuts->cuts[i];
|
||||
auto page = load_cellular_census_history(db, *request.objects, {world, request.region},
|
||||
request.lease, cut.head, cut.head.sequence, 1,
|
||||
builds, timeout);
|
||||
@@ -97,6 +155,27 @@ std::expected<CellularCensusSnapshot, CellularSnapshotFailure> load_cellular_cen
|
||||
required, observations);
|
||||
if (!census)
|
||||
return std::unexpected(Failure{Error::census, 0, {}, {}, census.error()});
|
||||
return CellularCensusSnapshot{std::move(*cuts), std::move(*census)};
|
||||
return RegisteredCellularCensusSnapshot{std::move(cuts->catalog),
|
||||
{std::move(cuts->cuts), std::move(*census)}};
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::expected<CellularCensusSnapshot, CellularSnapshotFailure> load_cellular_census_snapshot(
|
||||
Postgres& db, std::uint64_t world, std::span<const CellularCensusRegionRequest> requests,
|
||||
std::span<const Sha256Digest> builds, std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks) {
|
||||
auto result = load_snapshot(db, world, requests, builds, timeout, hooks, false);
|
||||
if (!result)
|
||||
return std::unexpected(result.error());
|
||||
return std::move(result->snapshot);
|
||||
}
|
||||
|
||||
std::expected<RegisteredCellularCensusSnapshot, CellularSnapshotFailure>
|
||||
load_registered_cellular_census_snapshot(Postgres& db, std::uint64_t world,
|
||||
std::span<const CellularCensusRegionRequest> requests,
|
||||
std::span<const Sha256Digest> builds,
|
||||
std::chrono::milliseconds timeout,
|
||||
const CellularCensusSnapshotHooks& hooks) {
|
||||
return load_snapshot(db, world, requests, builds, timeout, hooks, true);
|
||||
}
|
||||
} // namespace evo::persistence
|
||||
|
||||
@@ -791,15 +791,161 @@ void history_pages(Fixture& f) {
|
||||
"history transaction requirement changed for read-only callers");
|
||||
std::puts("committed census history pagination, replay, transaction and failure guards passed");
|
||||
}
|
||||
void registered_snapshot_coverage(Fixture& f) {
|
||||
constexpr std::uint64_t seed = 31001, world = seed + 200;
|
||||
const std::array<Sha256Digest, 1> builds{f.build};
|
||||
auto reader = connection();
|
||||
std::vector<std::unique_ptr<FilesystemObjectStore>> stores;
|
||||
std::vector<CellularCensusRegionRequest> resources;
|
||||
const auto enroll = [&](std::uint64_t region) {
|
||||
const RegionKey key{world, region};
|
||||
auto objects = take(
|
||||
FilesystemObjectStore::open(environment("PGHOST"), key, max_cellular_population_bytes),
|
||||
"registered empty region objects");
|
||||
const auto state = opening(seed, true, 1, false, region);
|
||||
take(provision_cellular_region(*f.owner, *objects, key, 1, *state, f.id(), f.build, 5s),
|
||||
"registered empty region genesis");
|
||||
const auto lease =
|
||||
take(acquire_region_lease(*f.db, key, f.build, 60000ms, 5s), "registered region grant");
|
||||
resources.push_back({region, lease, objects.get()});
|
||||
stores.push_back(std::move(objects));
|
||||
};
|
||||
const auto read = [&](std::span<const CellularCensusRegionRequest> requests) {
|
||||
return load_registered_cellular_census_snapshot(*reader, world, requests, builds, 5s);
|
||||
};
|
||||
const auto refused = [&](auto result, CellularCoverageError error) {
|
||||
require(!result && result.error().kind == CellularSnapshotError::coverage &&
|
||||
result.error().coverage.kind == error && reader->transaction_idle(),
|
||||
"registered coverage omitted an owner or retained its transaction");
|
||||
};
|
||||
enroll(1);
|
||||
const std::array first_request{resources.front()};
|
||||
const auto before = take(read(first_request), "registered single-region census");
|
||||
require(before.catalog == std::vector<RegisteredCellularRegion>{{1, 1}} &&
|
||||
before.snapshot.cuts.size() == 1,
|
||||
"registered snapshot lost catalog/cut evidence");
|
||||
const CellularCensusSnapshotHooks enrollment_hook{[&](std::size_t index) {
|
||||
if (index == 0)
|
||||
enroll(2); // Independent committed enrollment after the registry snapshot.
|
||||
}};
|
||||
const auto raced = take(load_registered_cellular_census_snapshot(*reader, world, first_request,
|
||||
builds, 5s, enrollment_hook),
|
||||
"registered snapshot concurrent enrollment");
|
||||
require(raced.catalog == before.catalog &&
|
||||
raced.snapshot.cuts[0].head == before.snapshot.cuts[0].head,
|
||||
"concurrent enrollment changed an already selected registry cut");
|
||||
refused(read(first_request), CellularCoverageError::regions);
|
||||
auto extra = resources;
|
||||
extra.push_back({999, resources.back().lease, stores.back().get()});
|
||||
refused(read(extra), CellularCoverageError::regions);
|
||||
auto wrong = resources;
|
||||
++wrong.back().lease.frame;
|
||||
refused(read(wrong), CellularCoverageError::frame);
|
||||
wrong = resources;
|
||||
wrong.back().region = 999;
|
||||
refused(read(wrong), CellularCoverageError::regions);
|
||||
wrong = resources;
|
||||
wrong.back().region = wrong.front().region;
|
||||
const auto duplicate = read(wrong);
|
||||
require(!duplicate && duplicate.error().kind == CellularSnapshotError::invalid_request,
|
||||
"registered snapshot accepted duplicate resources");
|
||||
refused(
|
||||
load_registered_cellular_census_snapshot(*reader, world + 1000, first_request, builds, 5s),
|
||||
CellularCoverageError::unknown_world);
|
||||
take(release_region_lease(*f.db, {world, 2}, resources[1].lease, 5s),
|
||||
"release registered empty owner");
|
||||
take(f.owner->query("UPDATE evolution.region_leases SET expires_at=clock_timestamp()-"
|
||||
"interval '1 second' WHERE world_id=31201 AND region_id=1",
|
||||
{}, 5s),
|
||||
"expire registered empty owner");
|
||||
const auto inactive = take(read(resources), "released and expired registry coverage");
|
||||
require(inactive.catalog == std::vector<RegisteredCellularRegion>{{1, 1}, {2, 1}},
|
||||
"registry hid released or expired holders");
|
||||
// Genesis recovery has no fenced transition to read; this checks inclusion,
|
||||
// not fresh ownership. No output grants authority to expired/released owners.
|
||||
auto pending_store = take(FilesystemObjectStore::open(environment("PGHOST"), {world, 3},
|
||||
max_cellular_population_bytes),
|
||||
"pending registration objects");
|
||||
const auto pending_state = opening(seed, true, 1, false, 3);
|
||||
take(f.owner->query("BEGIN", {}, 5s), "pending registration transaction");
|
||||
take(provision_cellular_region(*f.owner, *pending_store, {world, 3}, 1, *pending_state, f.id(),
|
||||
f.build, 5s),
|
||||
"pending registration genesis");
|
||||
const auto isolated = read(resources);
|
||||
take(f.owner->query("ROLLBACK", {}, 5s), "pending registration rollback");
|
||||
require(take(isolated, "registered snapshot with pending enrollment").catalog ==
|
||||
inactive.catalog,
|
||||
"uncommitted enrollment entered a registered evidence cut");
|
||||
take(reader->query("BEGIN READ ONLY", {}, 5s), "caller registered transaction");
|
||||
const auto caller = read(resources);
|
||||
take(reader->query("ROLLBACK", {}, 5s), "caller registered rollback");
|
||||
require(!caller && caller.error().kind == CellularSnapshotError::invalid_request,
|
||||
"registered snapshot took over a caller transaction");
|
||||
for (std::uint64_t region = 3; region <= max_cellular_census_regions; ++region)
|
||||
enroll(region);
|
||||
const auto bounded = take(read(resources), "64 registered regions");
|
||||
require(bounded.catalog.size() == max_cellular_census_regions &&
|
||||
bounded.snapshot.cuts.size() == max_cellular_census_regions &&
|
||||
bounded.catalog.back() == RegisteredCellularRegion{64, 1},
|
||||
"registered coverage lost its last in-bound owner");
|
||||
enroll(65);
|
||||
refused(read(std::span{resources}.first(max_cellular_census_regions)),
|
||||
CellularCoverageError::bounds);
|
||||
const auto rows = take(reader->query("SELECT * FROM evolution.load_world_region_catalog_v1("
|
||||
"31201::evolution.u64)",
|
||||
{}, 5s),
|
||||
"catalog overflow sentinel");
|
||||
require(rows.size() == 65, "SQL registry silently truncated the overflow owner");
|
||||
const RegionKey missing_key{world + 1, 1};
|
||||
take(f.owner->query("INSERT INTO evolution.region_leases(world_id,region_id,frame_id) "
|
||||
"VALUES(31202.00,1.0,1.00),(31202,18446744073709551615,1.000)",
|
||||
{}, 5s),
|
||||
"registered row with missing head fixture");
|
||||
auto missing_store = take(FilesystemObjectStore::open(environment("PGHOST"), missing_key,
|
||||
max_cellular_population_bytes),
|
||||
"missing registered checkpoint store");
|
||||
const std::array missing_resources{
|
||||
CellularCensusRegionRequest{1, resources[0].lease, missing_store.get()},
|
||||
CellularCensusRegionRequest{std::numeric_limits<std::uint64_t>::max(), resources[0].lease,
|
||||
missing_store.get()}};
|
||||
const auto missing = load_registered_cellular_census_snapshot(*reader, missing_key.world,
|
||||
missing_resources, builds, 5s);
|
||||
require(!missing && missing.error().kind == CellularSnapshotError::history &&
|
||||
missing.error().region == 1 && reader->transaction_idle(),
|
||||
"registered owner without a checkpoint disappeared from coverage");
|
||||
const RegionKey unsupported_key{world + 2, 1};
|
||||
const auto unsupported_manifest =
|
||||
take(genesis_manifest(unsupported_key, 1, f.id(), 50000, f.build, 1, f.build),
|
||||
"non-cellular registry manifest fixture");
|
||||
const std::array<std::string, 1> manifest_parameter{
|
||||
hexadecimal(take(encode_manifest(unsupported_manifest), "non-cellular fixture encoding"))};
|
||||
take(f.owner->query("SELECT evolution.provision_region_checkpoint($1::bytea)",
|
||||
manifest_parameter, 5s),
|
||||
"non-cellular registered owner");
|
||||
auto unsupported_store =
|
||||
take(FilesystemObjectStore::open(environment("PGHOST"), unsupported_key,
|
||||
max_cellular_population_bytes),
|
||||
"unsupported registered store");
|
||||
const std::array unsupported_resources{
|
||||
CellularCensusRegionRequest{1, resources[0].lease, unsupported_store.get()}};
|
||||
const auto unsupported = load_registered_cellular_census_snapshot(
|
||||
*reader, unsupported_key.world, unsupported_resources, builds, 5s);
|
||||
require(!unsupported && unsupported.error().kind == CellularSnapshotError::history &&
|
||||
unsupported.error().history.recovery.kind == RecoveryError::unsupported &&
|
||||
reader->transaction_idle(),
|
||||
"unsupported registered owner was filtered out or reported as zero");
|
||||
std::puts("registered world coverage, enrollment isolation and 64/65 bounds passed");
|
||||
}
|
||||
void snapshot_transport_faults(Fixture& f) {
|
||||
if (!std::getenv("EVO_MIGRATION_TCP_PORT"))
|
||||
return;
|
||||
const std::array<Sha256Digest, 1> builds{f.build};
|
||||
const std::array requests{CellularCensusRegionRequest{f.key.region, f.lease, f.objects.get()}};
|
||||
for (unsigned mode = 0; mode < 4; ++mode) {
|
||||
for (unsigned mode = 0; mode < 8; ++mode) {
|
||||
test::PostgresRelay relay(migration_backend_port());
|
||||
auto db = migration_tls_connection(relay);
|
||||
const bool at_completion = mode >= 2;
|
||||
const bool registered = mode >= 4;
|
||||
const bool at_completion = mode % 4 >= 2;
|
||||
const bool lost_request = mode % 2 == 0;
|
||||
const auto partition = [&] {
|
||||
if (lost_request)
|
||||
@@ -813,10 +959,17 @@ void snapshot_transport_faults(Fixture& f) {
|
||||
if (at_completion)
|
||||
partition();
|
||||
}};
|
||||
const auto lost =
|
||||
load_cellular_census_snapshot(*db, f.key.world, requests, builds, 300ms, hook);
|
||||
require(!lost && lost.error().kind == CellularSnapshotError::transaction &&
|
||||
lost.error().transaction.kind == PgError::unknown && !db->transaction_idle() &&
|
||||
const auto failure = [](auto result) {
|
||||
if (result)
|
||||
throw std::runtime_error("lost capture I/O returned a census");
|
||||
return result.error();
|
||||
};
|
||||
const auto lost = registered ? failure(load_registered_cellular_census_snapshot(
|
||||
*db, f.key.world, requests, builds, 300ms, hook))
|
||||
: failure(load_cellular_census_snapshot(
|
||||
*db, f.key.world, requests, builds, 300ms, hook));
|
||||
require(lost.kind == CellularSnapshotError::transaction &&
|
||||
lost.transaction.kind == PgError::unknown && !db->transaction_idle() &&
|
||||
relay.healthy() &&
|
||||
(lost_request ? relay.dropped_requests() : relay.dropped_responses()) > 0 &&
|
||||
take(load_region_checkpoint(*f.db, f.key, 5s), "head after lost capture I/O")
|
||||
@@ -1351,9 +1504,10 @@ void migration_snapshot_race(Fixture& f, RegionKey destination_key,
|
||||
const auto& dl = reverse ? f.lease : destination_lease;
|
||||
const auto sk = reverse ? destination_key : f.key;
|
||||
const auto dk = reverse ? f.key : destination_key;
|
||||
const auto before =
|
||||
take(load_cellular_census_snapshot(*reader, f.key.world, requests, builds, 5s),
|
||||
"before snapshot migration race");
|
||||
const auto before_registered = take(
|
||||
load_registered_cellular_census_snapshot(*reader, f.key.world, requests, builds, 5s),
|
||||
"before snapshot migration race");
|
||||
const auto& before = before_registered.snapshot;
|
||||
const CellularMigrationRequest request{f.id(),
|
||||
{sk.region, sh.head.commit, f.id(), sh.head.digest},
|
||||
{dk.region, dh.head.commit, f.id(), dh.head.digest},
|
||||
@@ -1381,9 +1535,10 @@ void migration_snapshot_race(Fixture& f, RegionKey destination_key,
|
||||
frozen.trace, f.build, 5s),
|
||||
"publish real pair between snapshot head reads");
|
||||
}};
|
||||
const auto raced =
|
||||
take(load_cellular_census_snapshot(*reader, f.key.world, requests, builds, 5s, hook),
|
||||
"coherent census across a concurrent pair");
|
||||
const auto raced_registered = take(load_registered_cellular_census_snapshot(
|
||||
*reader, f.key.world, requests, builds, 5s, hook),
|
||||
"coherent census across a concurrent pair");
|
||||
const auto& raced = raced_registered.snapshot;
|
||||
const auto committed =
|
||||
take(std::move(published), "pair was not published in snapshot hook");
|
||||
sh = committed.source;
|
||||
@@ -1396,9 +1551,17 @@ void migration_snapshot_race(Fixture& f, RegionKey destination_key,
|
||||
std::ranges::equal(raced.census.regions()[i].rows(),
|
||||
before.census.regions()[i].rows()),
|
||||
"atomic pair split a captured cut or changed historical replay");
|
||||
const auto after =
|
||||
take(load_cellular_census_snapshot(*reader, f.key.world, requests, builds, 5s),
|
||||
"snapshot after pair publication");
|
||||
const auto after_registered = take(
|
||||
load_registered_cellular_census_snapshot(*reader, f.key.world, requests, builds, 5s),
|
||||
"snapshot after pair publication");
|
||||
const auto& after = after_registered.snapshot;
|
||||
require(before_registered.catalog == raced_registered.catalog &&
|
||||
after_registered.catalog == before_registered.catalog &&
|
||||
before_registered.catalog ==
|
||||
std::vector<RegisteredCellularRegion>{
|
||||
{f.key.region, f.lease.frame},
|
||||
{destination_key.region, destination_lease.frame}},
|
||||
"atomic migration changed or omitted registered owner coverage");
|
||||
require(after.cuts[0].head == f.head.head && after.cuts[1].head == destination_head.head &&
|
||||
after.cuts[0].head.sequence == before.cuts[0].head.sequence + 1 &&
|
||||
after.cuts[1].head.sequence == before.cuts[1].head.sequence + 1,
|
||||
@@ -2853,6 +3016,7 @@ void run_population_publication_tests() {
|
||||
funded_source(fixture);
|
||||
history_pages(fixture);
|
||||
snapshot_guards(fixture);
|
||||
registered_snapshot_coverage(fixture);
|
||||
conversions_and_retries(fixture);
|
||||
interval_contract(fixture);
|
||||
ambiguous_publication(fixture, false);
|
||||
|
||||
@@ -215,6 +215,7 @@ void upgrade_across_migration_and_barrier() {
|
||||
" -v migration_sha_47=$(s 0047_cellular_migration_freeze.sql)"
|
||||
" -v migration_sha_48=$(s 0048_cellular_migration_staging.sql)"
|
||||
" -v migration_sha_49=$(s 0049_cellular_migration_publication.sql)"
|
||||
" -v migration_sha_50=$(s 0050_world_region_catalog.sql)"
|
||||
" -v runtime_role=evo_runtime_test -f migrations/apply.sql"
|
||||
" > /dev/null 2>&1") != 0);
|
||||
CHECK(history(*owner_db) == "1,2,3,4,5,6,7");
|
||||
@@ -232,7 +233,7 @@ void upgrade_across_migration_and_barrier() {
|
||||
"tools/migrate-postgres.sh\" > /dev/null") == 0);
|
||||
CHECK(history(*owner_db) ==
|
||||
"1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,"
|
||||
"25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49");
|
||||
"25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50");
|
||||
CHECK(scalar(*owner_db, "SELECT (c.condeferrable AND c.condeferred AND "
|
||||
"c.confrelid = 'evolution.region_commits'::regclass)::text "
|
||||
"FROM pg_catalog.pg_constraint c WHERE c.conrelid = "
|
||||
|
||||
@@ -152,7 +152,7 @@ PGDATABASE=upgrade_path psql -X -v ON_ERROR_STOP=1 -1 -v "migration_sha=$v1_sha"
|
||||
-f "$root/migrations/0001_region_bootstrap.sql" > /dev/null
|
||||
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',') FROM evolution.schema_migrations")" = 1
|
||||
PGDATABASE=upgrade_path bash "$root/tools/migrate-postgres.sh"
|
||||
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49
|
||||
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50
|
||||
# v2 heads have no payload binding: v2->v3 fails closed and leaves history at v2.
|
||||
v2_sha="$(sha256sum "$root/migrations/0002_region_fencing.sql")"
|
||||
v2_sha="${v2_sha%% *}"
|
||||
@@ -173,7 +173,7 @@ PGDATABASE=v3_commits psql -X -v ON_ERROR_STOP=1 \
|
||||
-c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \
|
||||
-c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '01000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32), 'hex'))" > /dev/null
|
||||
PGDATABASE=v3_commits bash "$root/tools/migrate-postgres.sh"
|
||||
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 49:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49
|
||||
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 50:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50
|
||||
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) FROM evolution.region_commits")" = 1
|
||||
# Review P1: a v4 database holding a published (possibly terminal) checkpoint has
|
||||
# no durable decisions for it. The v5 upgrade must refuse before changing anything.
|
||||
@@ -227,7 +227,7 @@ PGDATABASE=v6_populated psql -X -v ON_ERROR_STOP=1 \
|
||||
-c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \
|
||||
-c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '02000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32) || repeat('00', 80), 'hex'))" > /dev/null
|
||||
PGDATABASE=v6_populated bash "$root/tools/migrate-postgres.sh"
|
||||
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49
|
||||
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50
|
||||
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT count(*) FROM evolution.region_commits c, LATERAL evolution.parse_checkpoint_manifest(c.manifest) m WHERE m.commit_sequence = 0 AND m.predecessor_commit IS NULL")" = 1
|
||||
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(conname, ',') FROM pg_constraint WHERE conrelid = 'evolution.region_heads'::regclass AND contype = 'c' AND pg_get_constraintdef(oid) LIKE '%end_tick%'")" = region_heads_published_tick
|
||||
phase permission-audit
|
||||
@@ -278,11 +278,13 @@ expect_audit_failure 'GRANT SELECT ON evolution.cellular_mutation_action_publica
|
||||
expect_audit_failure 'GRANT SELECT ON evolution.account_controls TO evo_runtime_test' 'REVOKE SELECT ON evolution.account_controls FROM evo_runtime_test'
|
||||
expect_audit_failure 'REVOKE EXECUTE ON FUNCTION evolution.claim_account_control(evolution.u64, evolution.u64, uuid, evolution.u64, uuid, evolution.u64, evolution.u64, integer) FROM evo_runtime_test' 'GRANT EXECUTE ON FUNCTION evolution.claim_account_control(evolution.u64, evolution.u64, uuid, evolution.u64, uuid, evolution.u64, evolution.u64, integer) TO evo_runtime_test'
|
||||
bash "$root/tools/migrate-postgres.sh" # Restored configuration passes the audit again.
|
||||
# Schema 50: registry access is granted only through the bounded read function.
|
||||
expect_audit_failure 'REVOKE EXECUTE ON FUNCTION evolution.load_world_region_catalog_v1(evolution.u64) FROM evo_runtime_test' 'GRANT EXECUTE ON FUNCTION evolution.load_world_region_catalog_v1(evolution.u64) TO evo_runtime_test'
|
||||
phase checksum-drift
|
||||
# Unknown future versions and checksum drift must not be repaired silently.
|
||||
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=50 WHERE version=49'
|
||||
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=51 WHERE version=50'
|
||||
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
|
||||
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=49 WHERE version=50"
|
||||
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=50 WHERE version=51"
|
||||
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=14"
|
||||
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
|
||||
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=13"
|
||||
|
||||
@@ -110,6 +110,8 @@ digest_48="$(sha256sum "$root/migrations/0048_cellular_migration_staging.sql")"
|
||||
digest_48="${digest_48%% *}"
|
||||
digest_49="$(sha256sum "$root/migrations/0049_cellular_migration_publication.sql")"
|
||||
digest_49="${digest_49%% *}"
|
||||
digest_50="$(sha256sum "$root/migrations/0050_world_region_catalog.sql")"
|
||||
digest_50="${digest_50%% *}"
|
||||
psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \
|
||||
--set="migration_sha_2=$digest_2" --set="migration_sha_3=$digest_3" \
|
||||
--set="migration_sha_4=$digest_4" --set="migration_sha_5=$digest_5" \
|
||||
@@ -152,5 +154,6 @@ psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \
|
||||
--set="migration_sha_47=$digest_47" \
|
||||
--set="migration_sha_48=$digest_48" \
|
||||
--set="migration_sha_49=$digest_49" \
|
||||
--set="migration_sha_50=$digest_50" \
|
||||
--set="runtime_role=$runtime_role" \
|
||||
--file="$root/migrations/apply.sql"
|
||||
|
||||
Reference in New Issue
Block a user