M12.7: reconcile pending migration evidence in census snapshots
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (ecs) (push) Successful in 2m26s
Foundation / checks (network) (push) Successful in 21m1s
Foundation / checks (client) (push) Successful in 22m9s
Foundation / checks (tsan) (push) Successful in 24m36s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (postgres) (push) Canceled after 33m38s
Foundation / checks (checks) (push) Canceled after 33m49s

This commit is contained in:
dingo
2026-10-18 21:14:53 -04:00
parent 709dfb9557
commit 27ec5729bf
23 changed files with 592 additions and 23 deletions
+16
View File
@@ -1,5 +1,21 @@
# Project status and next steps
## M12.7c registered census pending handoff evidence - 2026-10-18
Schema 52 captures active biological migration pointers with history domain,
registry and heads. Both predecessor manifests are bound to retained builds;
source capsules and staged destination ghosts receive complete paired replay
outside the metadata transaction. Capsules/ghosts add no stock. Prepare/stage/
abort/commit interleavings preserve the selected view; unavailable or false evidence
fails the entire result. This accepts no terminal extinction or live authority.
Native Release **79/79**, final raw/population allowlist cases, ASan/UBSan TLS/storage
faults, Clang analysis and formatting pass. Final private persistence/process/
backup/DR/migration/permission qualification passed in **558 s**; results and limits
are recorded in [validation](engineering/VALIDATION.md). Other viable
stores, cross-domain clone provenance, accepted global reconciliation/freshness
and ecology/soak/product gates remain open.
## M12.7b database history domains and restore provenance - 2026-10-18
Schema 51 binds registered census evidence to a database history-domain UUID,
@@ -0,0 +1,68 @@
# ADR-0156: Pending migration evidence in registered census snapshots
## Status
Accepted for scoped M12.7c, 2026-10-18. Extends ADR-0133/0147–0149/0153–0155.
No biological format, rule, activation workflow or terminal-extinction decision.
## Decision
A registered census must expose its pending biological handoffs, rather than
letting a caller mistake a staged ghost for additional owned life or infer that
no transfer exists. Schema 52 reads the union of source/destination pending
pointers for one world, ordered by request UUID, at most 65 requests. The 65th
detects overflow of the 64-request work bound; never truncate a complete result.
This is the existing active-pointer inventory, not an audit of all historical
preparations or a new source of ownership. No duplicate state table/write.
Capture it in the same read-only repeatable-read view as domain, registry and
heads. Keep both retained predecessor manifests, canonical trace and, when staged,
ghost digest/length. Require internally coherent pointers, un-aborted/uncommitted
state and metadata identities; malformed/missing evidence fails the entire query.
Both predecessor builds must belong to the caller's retained build allowlist,
including an optimistic destination predecessor that is no longer its current head.
Every endpoint must be registered. The source predecessor must equal its selected
head; a staged destination must also equal its selected predecessor. A merely
prepared destination is an optimistic observation and may have advanced, as
ADR-0147 permits. Common census-boundary checks still apply independently.
After metadata capture ends, fully replay every captured handoff from its retained
source-side predecessor/capsule objects. For a stage, independently replay the
destination-side capsules and compare the complete derived ghost bytes with the
verified staged object. Reuse the existing migration codec, binding and paired
replay logic. Missing/corrupt/false capsules or ghosts fail with no partial census.
This is historical evidence verification, not a fenced live-owner lookup; a
concurrent abort/commit does not invalidate an already captured immutable view.
Return verified trace/member/material evidence and a staged flag alongside the
registered census. Source stock is already counted at its unchanged committed
head. Capsules and ghosts add **zero** census stock. After atomic commit, the next
view sees settled heads and no pending pointers; abort clears both pointers with
unchanged heads. Neither timeout nor lease expiry settles a workflow.
## Alternatives, costs and risks
Reading current fenced loaders after capture could combine newer pointer state
with older heads and also needs write locks. Reading inside the read-only view
cannot use those locking functions. Capture immutable evidence references without
locks, close the metadata transaction, then replay exactly those references.
Counting a ghost twice breaks conservation; ignoring unavailable evidence hides
uncertainty. The conservative error is an unavailable registered result.
This adds one bounded metadata read and complete pair/ghost replay per pending
request, plus transient bounded evidence buffers. No whole-query latency promise,
new cache, aggregate speedup, dependency or target-host capacity claim. The trusted
owner can bypass table invariants; this does not authenticate an untrusted owner
or invent stock outside the current committed owner model.
## Qualification and remaining scope
Test raw/population prepare/stage/abort/commit, unchanged census stock, corrupt or
unavailable source capsules/ghosts, malformed pointers, bounds, expired authority,
and capture interleavings with settlement/new preparation. Run full persistence,
backup/DR, migration/privilege checks, sanitizers, analysis and default CTest.
Prior migrations and biological goldens remain unchanged.
This reconciles the existing pending migration evidence with registered owner
cuts. Cross-domain clone ancestry, other viable-store coverage, accepted durable
global reconciliation/freshness, coupled ecology and actual days remain open.
+2
View File
@@ -1,5 +1,7 @@
# Architecture decision records
- [ADR-0156: Census pending migration evidence](0156-census-pending-migration-evidence.md) - schema 52 captures active handoff pointers with registered heads and fully verifies retained capsules/ghosts; source owns stock and ghosts add none, including across settlement races.
- [ADR-0155: Database history domains](0155-database-history-domain.md) - schema 51 binds registered census evidence to a database namespace; supported quarantined restores retain parent domains and exact fork cuts before readmission. Cross-domain clone provenance and accepted extinction remain open.
- [ADR-0154: Registered-world census coverage](0154-registered-world-census-coverage.md) - schema 50 uses the existing durable authority registry as the required regional set in a consistent snapshot; omitted/unavailable/over-bound owners fail without a new catalog table or extinction authority.
@@ -1,5 +1,13 @@
# Persistence and region authority
M12.7c/schema 52 adds [pending migration evidence](../../schemas/ecology/census-pending-migrations-v1.md)
to registered snapshots. Pending source/destination pointers, namespace, registry
and heads share one metadata view. Exact captured capsules/ghosts are fully
replayed after capture, even if settlement commits meanwhile. The source head
already owns prepared/staged life; ghosts add no census stock. Missing/false
evidence fails the entire result. This is existing handoff evidence reconciliation,
not coverage of every viable store or an accepted terminal-extinction record.
M12.7b/schema 51 adds [database history domains](../../schemas/ecology/database-history-domain-v1.md).
Registered census evidence identifies a world by `(history_domain, world_id)`;
domain, registry and heads share one metadata view. Supported restores fork the
+1 -1
View File
@@ -14,7 +14,7 @@ Commit and push each validated scoped segment separately.
| 12.4 | Owned intervals and durable paired migration/fault qualification implemented in scoped segments through 12.4l | ADR-0143–0149/schema 49; bounded admitted work, source freeze, staged ghost, atomic pair and exact recovery, local TLS and storage faults. Multi-host/orchestrator, linked ecology and integrated stock audits remain open. |
| 12.5 | Blocked on 12.2-12.4 | Finite M6 process fixtures do not qualify coupled persistent cellular/cohort atmosphere/water budgets. |
| 12.6 | Regional queries, replayed history, scoped multi-region queries and consistent database head snapshots implemented (12.6a-d) | ADR-0150–0153; short read-only snapshot capture prevents atomic pairs from straddling declared head sets, with exact immutable replay. Durable world/branch coverage, outside transfer/viability reconciliation, target-host query/index costs and client path remain required. |
| 12.7 | Evidence policy, registered coverage and database history domains implemented (12.7a-b) | ADR-0154/0155/schema 51 captures namespace/registry/heads together; supported quarantined restores fork with exact-cut provenance. Cross-domain clone provenance, other viable/in-flight stores and durable accepted global reconciliation remain required. |
| 12.7 | Evidence policy, registered coverage, history domains and verified pending handoff evidence implemented (12.7a-c) | ADR-0154–0156/schema 52 captures namespace/registry/heads/pending handoffs together, with exact capsule/ghost replay and supported restore forks. Cross-domain clone provenance, other viable-store coverage and durable accepted global reconciliation/freshness remain required. |
| 12.8 | Not qualified | Actual multi-day integrated ecology/restart/clone evidence remains required. |
| 12.9 | Not accepted | Returning-player product evidence and expansion review remain required; World0 stays separately blocked. |
+14 -1
View File
@@ -28,7 +28,7 @@ or authorize production enablement.
| 12.4 | Owned intervals; exact transfer/accounting; durable atomic pair/recovery; independent loopback TLS path loss and owner expiry/replacement/heal; local object sync/read-failure matrix | Multi-host/kernel-network, storage-host and orchestrator drills, linked-boundary contracts and integrated ownership/stock audits at scale. |
| 12.5 | Finite cellular contact feeding/decay and separate M6 producer/consumer resource owner | One jointly owned cellular/environment budget, explicit effective depth and source entitlements, versioned water/atmosphere species mapping and producer coefficients, plus closed/open and nutrient-exhaustion comparisons. |
| 12.6 | Retained archive, regional census/lineage queries, replayed history, scoped multi-region queries and consistent database head snapshots (12.6a-d); registered coverage/history domains follow in 12.7a-b | Cross-domain clone provenance, retained historical coverage/other viable stores, target-host query/index qualification and client permission/presentation; no account power or invented precise statistical ancestry. |
| 12.7 | Viability/unknown policy; consistent cuts, registered-region coverage and database history domains with supported restore forks (12.7a-b/schema 51) | Cross-domain clone provenance, other viable-store/transfer reconciliation and durable accepted global evidence records. Missing/stale/unsupported owners must prevent terminal extinction. |
| 12.7 | Viability/unknown policy; consistent cuts, registered coverage, history domains/restore forks and verified pending handoff evidence (12.7a-c/schema 52) | Cross-domain clone provenance, other viable-store coverage and durable accepted global evidence records/freshness. Missing/stale/unsupported owners must prevent terminal extinction. |
| 12.8 | Deterministic seed/regime drivers and separate-process exact continuation | Reproducible integrated worlds running for actual days, real restarts/clone comparisons and retained accounting/diversity/cost audits. Accelerated tests cannot supply elapsed days. |
| 12.9 | M11 feedback and current UI review retained | Returning-player samples on the integrated ecology and explicit product/3D readiness decision after soak evidence. No M12 product acceptance or World0 promotion is inferred. |
@@ -93,6 +93,11 @@ their subadult realization while reproduction requires a fully realized parent.
Finite extra food alone does not close repeated wildlife generations. A versioned
growth contract and real multi-generation evidence remain necessary; transfer
tests must not fabricate adult wildlife or silently change the rules to pass.
The existing metabolic construction budget already derives a channelwise
affordable growth transition with explicit intake/release, but cellular ticks do
not apply it. It is a concrete basis for a separately versioned funded maturation
contract that also resolves expanded-body collision, tick ordering and offline
freeze. Its existence alone does not admit a new growth rule.
M12.6a's [regional census projection](../../schemas/ecology/regional-cellular-census-v1.md)
independently implements local current-genome and descendant-closure queries.
@@ -138,6 +143,14 @@ unknown original context; direct privileged database copies can bypass the
workflow. Cross-domain clone provenance, other viable/in-flight stores and durable
accepted reconciliation remain open. This still does not close M12.7.
M12.7c/schema 52's [pending handoff evidence](../../schemas/ecology/census-pending-migrations-v1.md)
now captures active source/destination pointers with domain/registry/heads, binds
them to the selected cuts and fully verifies retained source capsules and staged
ghost biology after capture. Source-owned life is already in the census; ghosts
add none. Prepare/stage/abort/commit races preserve the selected view. Unavailable
or false evidence fails the whole query. Other viable stores, external clone
provenance and accepted durable global reconciliation/freshness remain open.
Current expansion readiness: **no-go for M13 integration or World0 promotion**
while these M12 integration/evidence gates remain open. This records the present
engineering evidence, not a substitute for the later maintainer product review.
+10
View File
@@ -1,5 +1,15 @@
# Validation strategy
M12.7c/schema 52 captures pending handoffs with registered heads, then fully
replays source capsules and destination ghosts outside the metadata transaction.
For raw/population owners, race prepare/stage/abort/commit with capture; verify
unchanged source-owned census totals and zero extra ghost stock. Deny capsule/
ghost reads and reject false transfer quantities or real objects with false
ghost biology. Owner fault seams test malformed pending pointers and an injected
65-row response (bounds before decoding). Retain full immutable replay, TLS and
cleanup checks, migration/checksum/permissions, backup/DR, sanitizers and default
CTest. These tests do not establish every viable store or terminal extinction.
M12.7b/schema 51 checks database-domain adoption/no-op stability, restore-only
quarantine, transaction rollback, exact fork cuts, idempotent/stale requests and
runtime read-only privileges. Registered snapshots retain their domain during an
+79
View File
@@ -1,5 +1,84 @@
# Validation report
## 2026-10-18 - M12.7c registered census pending handoff evidence
[ADR-0156](../adr/0156-census-pending-migration-evidence.md)/schema 52 captures
the union of active source/destination migration pointers in the same read-only
repeatable-read view as domain, complete registry and current heads. The source
predecessor must match its selected head; a staged destination must also match.
Both predecessor builds must be allowlisted, including an older optimistic
destination cut. After capture, full source capsule and destination ghost replay
verifies exact members/material/bytes, without adding any ghost stock to census.
Executed locally:
- MSVC default Release full CTest **79/79 passed, 14.14 s**.
- GCC 14 Release final semantic-failure population/history/migration/census matrix
passed **exit 0, 18 s** (16 s population). The final predecessor-build allowlist
guard and the complete affected cases passed **exit 0, 17 s** (16 s population).
- Clang 19 **ASan/UBSan** complete affected raw/population matrix, real loopback
TLS request/reply loss and local storage faults passed **exit 0, 77 s**
(73 s population), without sanitizer reports, before the final allowlist guard.
The final rerun passed **exit 0, 71 s** (69 s population), without reports.
- clang-tidy 19 passed on both changed persistence adapters and integration test
files; final semantic-failure assertions passed a separate recheck. Final
allowlist analysis/format also passed. Changed shell scripts pass `bash -n`.
- Initial full private persistence/process/backup/DR and SQL qualification passed
**exit 0, 539 s** before the final allowlist guard. The final frozen-binary rerun
passed **exit 0, 558 s**: C++ integration/process/load/history **435 s**, migration
TLS/storage faults **42 s**, life reservations **4 s**, domain/legacy/backup/
restore **31 s**, host-loss DR **8 s**, upgrade **10 s**, migration history
**5 s**, runtime permission audit **13 s**, checksum/future rejection **8 s**,
plus setup/migration. The final completion sentinel records zero and the log
contains the all-phase success summary. The new getter's missing-grant case
correctly fails the effective-privilege audit before restoring the grant.
- Final `git diff --check` passed; private test clusters stop on exit and preserve
forensic directories. No biological golden or prior numbered migration changed.
Both representations, in both directions, use real player-origin wildlife and
independent connections. Capture before preparation sees no pending request;
the next view sees source-only evidence. A concurrent stage leaves an earlier
view prepared and the next staged. A concurrent abort or paired commit preserves
the captured staged evidence; the next view sees no pointers and, for commit,
both new heads. All pending views retain the original source-owned census stock.
No receipt, ghost or transfer amount is added as another resident population.
Actual read denial of the source's destination capsule and the destination ghost
fails the registered result without fallback to live peer state. Metadata-only
false transferred material and a digest-valid object with the wrong complete
ghost biology are rejected by full replay. Trusted owner fault seams move a
source pointer to the wrong region and inject 65 backend rows; these fail without
partial evidence. The overflow fixture is a response-bound test, not 65 valid
simultaneous migrations or a qualified world size. The original SQL function and
pointer are restored before continuing the real workflow.
Schema 52 adds one least-privilege read function and no state table/index,
registration/publication writes, biological format/rule/golden or dependency.
Prior numbered migrations 1..51 are unchanged. Latest-prefix assertions include
52; unknown-future rejection uses 53. Runtime audit requires the new read grant.
The pre-context backup fixture removes both schema-51 metadata and schema-52
getter to reconstruct the exact old prefix; old external ancestry is not invented.
Limits: active-pointer evidence is not an audit of every historical preparation,
coverage of every viable store, accepted durable global freshness or terminal
extinction. No target-host capacity, new TSan, remote CI, multi-host/storage-host
failure, cellular clone/pending-migration backup restore, actual multi-day ecology
or product acceptance is claimed. Generic backup/DR regression remains separate
from cellular restore qualification. No live database, deployment or World0 action.
Logs: `build/m127c-build.log`, `build/m127c-rebuild.log`,
`build/m127c-final-build.log`, `build/m127c-native.log`,
`build/m127c-population.log`, `build/m127c-final-population.log`,
`build/m127c-asan-build.log`, `build/m127c-asan.log`,
`build/m127c-tidy.log`, `build/m127c-format.log`,
`build/m127c-final-tidy.log`, `build/m127c-final-format.log`,
`build/m127c-allowlist-build.log`, `build/m127c-allowlist.log`,
`build/m127c-final-asan-build.log`, `build/m127c-final-asan.log`,
`build/m127c-allowlist-tidy.log`, `build/m127c-allowlist-format.log`,
`build/m127c-shell.log`, `build/m127c-full.log`,
`build/m127c-full.exit`, `build/m127c-final-full.log`,
`build/m127c-final-full.exit` (completion only).
## 2026-10-18 - M12.7b database history domains and restore provenance
[ADR-0155](../adr/0155-database-history-domain.md)/schema 51 adds a database