M12.7: bind census evidence to database history domains
Foundation / checks (ecs) (push) Successful in 2m23s
Foundation / checks (deploy) (push) Successful in 4m10s
Foundation / checks (network) (push) Successful in 21m32s
Foundation / checks (client) (push) Successful in 22m14s
Foundation / checks (tsan) (push) Successful in 24m30s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (checks) (push) Canceled after 41m43s
Foundation / checks (postgres) (push) Canceled after 41m39s

This commit is contained in:
dingo
2026-10-18 20:33:16 -04:00
parent 145e6e1445
commit 709dfb9557
25 changed files with 547 additions and 27 deletions
+16
View File
@@ -1,5 +1,21 @@
# Project status and next steps
## M12.7b database history domains and restore provenance - 2026-10-18
Schema 51 binds registered census evidence to a database history-domain UUID,
captured with the registry and heads. Supported restores create a fresh domain
before verification/readmission, retaining the parent, restore claim and exact
restored cuts. Biological identities/bytes stay unchanged. Legacy backups have
unknown original context and acquire adoption only when migrated.
Default Release CTest **79/79**, focused domain/legacy-restore qualification,
ASan/UBSan TLS/storage faults, Clang analysis and formatting pass. Full private
persistence/process/backup/DR and migration/permissions qualification passed in
**558 s** after correcting an incomplete frozen-helper directory. Results and
limits are recorded in [validation](engineering/VALIDATION.md).
Cross-domain clone provenance, other viable stores, durable accepted extinction,
client access and ecology/soak/product gates remain open; this is not complete M12.
## M12.7a registered-world census coverage - 2026-10-18
Schema 50 now uses the durable authority registry as the required regional set
@@ -65,3 +65,9 @@ the world ID; external branch identity is still missing. Outside/unsupported sto
viable reserves, durable accepted reconciliation records and global freshness remain
M12.7 work. No terminal-extinction outcome, client endpoint, live database,
deployment or World0 action.
Schema-51 follow-up: [ADR-0155](0155-database-history-domain.md) adds a database
history domain to registered results in the same metadata view and forks supported
restores with parent/exact-cut provenance. Cross-domain clone provenance and the
other reconciliation limits above remain open; registered coverage alone still
does not authorize terminal extinction.
+92
View File
@@ -0,0 +1,92 @@
# ADR-0155: Database history domains for census and restore evidence
## Status
Accepted for scoped M12.7b, 2026-10-18. Extends ADR-0133/0153/0154 and the
supported ADR-0074/0075 restore workflow. This is an evidence namespace, not a
new biological identity, codec, simulation version or terminal-extinction rule.
## Context and decision
World IDs and exact immutable commits can be reused by a database restored from
a backup. Independently initialized databases can also use the same world IDs.
Neither a connection address nor an object path establishes evolutionary history.
Schema 51 adopts one random, nonzero UUID **history domain per database**, using
PostgreSQL's built-in UUID generator. External evidence identifies a world by
`(history_domain, world_id)`. Adoption means this database's retained history as
observed at migration; it does not reconstruct an unknown earlier namespace.
Unchanged migrations preserve the domain. Different worlds remain distinct within
the domain. No per-tick writes, payload changes, extension or new dependency.
Keep owner-controlled `history_domains` events, one `current_history_domain` row,
and `history_domain_cuts` for restore provenance. The trusted runtime receives only
the fixed-search-path security-definer `load_history_domain_v1()` read function;
neither table access nor the ability to fork. The migration privilege audit includes
both the new required read grant and denial of the owner-only function.
Registered cellular snapshots capture the domain **before** registry/head reads in
the same short read-only repeatable-read transaction. Return it with registry,
exact cuts and fully replayed census. Missing/zero/malformed context fails the
whole result; a legacy schema fails rather than inferring a domain. Existing
caller-declared snapshots and low-level diagnostic history queries retain their
local semantics; they do not acquire an external-history guarantee implicitly.
Concurrent context changes belong to a later metadata view. Context identifies
captured evidence, not current authority or a freshness guarantee at return.
## Supported restores and legacy backups
For either isolated or explicitly authorized dev/stage in-place restore, the
existing install transaction first restores the dump, retires all builds and
clears holders. Before committing that still-quarantined install, call the new
owner-only `fork_restored_history_v1(restore_claim_uuid)` when present. It locks
the singleton domain row, refuses any admitted build or held lease, creates a
fresh domain whose parent is the restored domain, retains the request, and copies
all restored heads' world/region/frame/sequence/commit/digest/tick as fork cuts
before switching the current domain. Rollback removes the whole fork. Same-request
retries return the same current domain without duplicating records; a request from
an older fork refuses instead of switching backwards. Lost install receipts keep
the existing restore-claim resolution. Biological heads, ancestry and retained
objects stay byte-identical. Readmission still waits for head/history verification
and, in-place, the full runtime privilege audit.
A backup predating schema 51 has no source domain. Its restore skips the absent
function; later migration adopts a fresh domain with unknown original context.
Do not fabricate a parent or exact external branch ancestry. New-schema backups
retain the event/cut tables because they dump the entire `evolution` schema.
Backup-of-restore forks preserve their known namespace ancestry and exact fork
cuts; they do not rewrite biological clone origins or old payloads.
The operator is trusted and the replacement database must remain isolated during
installation/verification. Owner-only SQL is not a new public online branch API.
Arbitrary direct `pg_restore`, copying a running database, privileged edits, or
readmitting builds outside the supported workflow can copy/forge a domain. No
database UUID can detect such a bypass. This contract does not secure an
untrusted owner or change fencing, backup durability, deployment or World0 gates.
## Alternatives, benefits, costs and risks
Per-world enrollment tables would duplicate the authority registry and complicate
legacy adoption. Host/database names are mutable routing, and manifest digests
can survive a rewind. Embedding new domains in all biological codecs would change
deterministic bytes without changing biology. A database namespace keeps those
contracts intact and makes restore provenance explicit at metadata cost
proportional to the restored regional head count, once per restore.
Random UUID uniqueness is probabilistic, using the existing supported PostgreSQL
runtime. Trusted workflows must rotate consistently; no claim is made for a copied
database outside those workflows. Initial adoption cannot recover missing external
history. Retention/GC must preserve these records; no deletion feature is added.
## Qualification and remaining scope
Test no-op adoption, quarantine refusal, rollback, exact cuts, idempotent and stale
requests, runtime privileges, missing context, concurrent snapshot capture, isolated
restore and in-place rewind. Run full persistence/backup/DR/migration checks,
sanitizers, analysis/format and the default build. No biological golden changes.
Cross-domain clone-origin provenance is still incomplete: ADR-0067 commits the
source biological cut but does not encode its external history domain. A domain
alone cannot supply that missing ancestry. Durable accepted reconciliation must
later bind domain, exact cuts, registered coverage, other viable/in-flight stores
and acceptance freshness; no terminal extinction or complete M12 gate is claimed.
+2
View File
@@ -1,5 +1,7 @@
# Architecture decision records
- [ADR-0155: Database history domains](0155-database-history-domain.md) - schema 51 binds registered census evidence to a database namespace; supported quarantined restores retain parent domains and exact fork cuts before readmission. Cross-domain clone provenance and accepted extinction remain open.
- [ADR-0154: Registered-world census coverage](0154-registered-world-census-coverage.md) - schema 50 uses the existing durable authority registry as the required regional set in a consistent snapshot; omitted/unavailable/over-bound owners fail without a new catalog table or extinction authority.
- [ADR-0153: Consistent scoped cellular census snapshots](0153-consistent-cellular-census-snapshots.md) - short read-only repeatable-read head capture followed by exact immutable replay; atomic pairs cannot straddle a declared-region cut, without claiming world/branch completeness.
+11 -2
View File
@@ -1,12 +1,21 @@
# Persistence and region authority
M12.7b/schema 51 adds [database history domains](../../schemas/ecology/database-history-domain-v1.md).
Registered census evidence identifies a world by `(history_domain, world_id)`;
domain, registry and heads share one metadata view. Supported restores fork the
domain while quarantined in the installation transaction, retaining parent/request
and exact restored head cuts before verification/readmission. Initial adoption
and legacy backups cannot reconstruct unknown original context. Arbitrary trusted
database copies outside that workflow can copy a domain. Cross-domain clone
provenance, viable-store reconciliation and durable accepted extinction remain open.
M12.7a/schema 50 adds [registered-world coverage](../../schemas/ecology/registered-world-census-coverage-v1.md).
The existing authority registry supplies every required region/frame, without
activity, expiry, profile or checkpoint filters. The bounded read function and
head selection share one database snapshot; resources must match the entire
registered set. Missing/unusable owners and a 65th region fail the query. There
is no new catalog table or registration write. External branch identity, other
viable stores and accepted global extinction reconciliation remain separate.
is no new catalog table or registration write. Schema 51 adds the namespace above;
other viable stores and accepted global extinction reconciliation remain separate.
M12.6d captures declared regional heads in one short read-only repeatable-read
transaction, closes it, then fully replays each exact immutable cut before
+1 -1
View File
@@ -14,7 +14,7 @@ Commit and push each validated scoped segment separately.
| 12.4 | Owned intervals and durable paired migration/fault qualification implemented in scoped segments through 12.4l | ADR-0143–0149/schema 49; bounded admitted work, source freeze, staged ghost, atomic pair and exact recovery, local TLS and storage faults. Multi-host/orchestrator, linked ecology and integrated stock audits remain open. |
| 12.5 | Blocked on 12.2-12.4 | Finite M6 process fixtures do not qualify coupled persistent cellular/cohort atmosphere/water budgets. |
| 12.6 | Regional queries, replayed history, scoped multi-region queries and consistent database head snapshots implemented (12.6a-d) | ADR-0150–0153; short read-only snapshot capture prevents atomic pairs from straddling declared head sets, with exact immutable replay. Durable world/branch coverage, outside transfer/viability reconciliation, target-host query/index costs and client path remain required. |
| 12.7 | Evidence policy and registered-region coverage implemented (12.7a) | ADR-0154/schema 50 requires the durable authority registry in one consistent database cut. External branch identity, other viable/in-flight stores and durable accepted global reconciliation remain required. |
| 12.7 | Evidence policy, registered coverage and database history domains implemented (12.7a-b) | ADR-0154/0155/schema 51 captures namespace/registry/heads together; supported quarantined restores fork with exact-cut provenance. Cross-domain clone provenance, other viable/in-flight stores and durable accepted global reconciliation remain required. |
| 12.8 | Not qualified | Actual multi-day integrated ecology/restart/clone evidence remains required. |
| 12.9 | Not accepted | Returning-player product evidence and expansion review remain required; World0 stays separately blocked. |
+12 -4
View File
@@ -27,8 +27,8 @@ or authorize production enablement.
| 12.3 | Dedicated population profile/trace/store/owning host; exact recovery and private PostgreSQL fault tests | Population-specific executable/network fault drills and target-host publication/capacity qualification. |
| 12.4 | Owned intervals; exact transfer/accounting; durable atomic pair/recovery; independent loopback TLS path loss and owner expiry/replacement/heal; local object sync/read-failure matrix | Multi-host/kernel-network, storage-host and orchestrator drills, linked-boundary contracts and integrated ownership/stock audits at scale. |
| 12.5 | Finite cellular contact feeding/decay and separate M6 producer/consumer resource owner | One jointly owned cellular/environment budget, explicit effective depth and source entitlements, versioned water/atmosphere species mapping and producer coefficients, plus closed/open and nutrient-exhaustion comparisons. |
| 12.6 | Retained archive, regional census/lineage queries, replayed history, scoped multi-region queries and consistent database head snapshots (12.6a-d); registered coverage follows in 12.7a | External branch history context, retained historical coverage/other viable stores, target-host query/index qualification and client permission/presentation; no account power or invented precise statistical ancestry. |
| 12.7 | Viability/unknown policy; consistent head cuts and durable registered-region coverage (12.7a/schema 50) | External world/branch context, other viable-store/transfer reconciliation and durable accepted global evidence records. Missing/stale/unsupported owners must prevent terminal extinction. |
| 12.6 | Retained archive, regional census/lineage queries, replayed history, scoped multi-region queries and consistent database head snapshots (12.6a-d); registered coverage/history domains follow in 12.7a-b | Cross-domain clone provenance, retained historical coverage/other viable stores, target-host query/index qualification and client permission/presentation; no account power or invented precise statistical ancestry. |
| 12.7 | Viability/unknown policy; consistent cuts, registered-region coverage and database history domains with supported restore forks (12.7a-b/schema 51) | Cross-domain clone provenance, other viable-store/transfer reconciliation and durable accepted global evidence records. Missing/stale/unsupported owners must prevent terminal extinction. |
| 12.8 | Deterministic seed/regime drivers and separate-process exact continuation | Reproducible integrated worlds running for actual days, real restarts/clone comparisons and retained accounting/diversity/cost audits. Accelerated tests cannot supply elapsed days. |
| 12.9 | M11 feedback and current UI review retained | Returning-player samples on the integrated ecology and explicit product/3D readiness decision after soak evidence. No M12 product acceptance or World0 promotion is inferred. |
@@ -127,8 +127,16 @@ same database snapshot. Every registered region is required, including inactive,
expired, unsupported and unavailable owners. Unknown worlds and a 65th region
fail rather than becoming zero or a truncated complete set. Concurrent later
enrollment belongs to a later cut. No duplicate catalog/enrollment write is added.
External branch identity, other viable stores and durable accepted reconciliation
records remain open; this does not close M12.7 or authorize terminal extinction.
That registered-coverage primitive alone does not identify external history or
authorize terminal extinction.
M12.7b/schema 51's [history domain](../../schemas/ecology/database-history-domain-v1.md)
now binds registered results to a database namespace in the same metadata view.
Supported restores fork before verification/readmission and retain parent/request
and exact restored cuts. Initial adoption and legacy backups cannot recover
unknown original context; direct privileged database copies can bypass the
workflow. Cross-domain clone provenance, other viable/in-flight stores and durable
accepted reconciliation remain open. This still does not close M12.7.
Current expansion readiness: **no-go for M13 integration or World0 promotion**
while these M12 integration/evidence gates remain open. This records the present
+9
View File
@@ -1,5 +1,14 @@
# Validation strategy
M12.7b/schema 51 checks database-domain adoption/no-op stability, restore-only
quarantine, transaction rollback, exact fork cuts, idempotent/stale requests and
runtime read-only privileges. Registered snapshots retain their domain during an
independent metadata change; the next query sees the new context even with the
same manifests. Missing context fails without a partial/zero result. Test actual
isolated and in-place restore workflows, lost-install receipt resolution and
legacy bundles without context. Run full private persistence/migration/permission
regression, default CTest, ASan/UBSan, Clang analysis/format and shell syntax.
M12.7a's schema-50 tests require exact authority-registry coverage, including
released/expired holders, unavailable heads and unsupported profiles. Check
omitted/extra/duplicate resources, wrong frames, unknown worlds, integral numeric
+76
View File
@@ -1,5 +1,81 @@
# Validation report
## 2026-10-18 - M12.7b database history domains and restore provenance
[ADR-0155](../adr/0155-database-history-domain.md)/schema 51 adds a database
history-domain UUID to registered census evidence. The domain, registry and heads
share one short metadata snapshot; every exact cut still receives full immutable
replay. Supported restores fork the namespace in the quarantined installation
transaction, retaining parent/request and exact restored head cuts. Biological
payloads, rules, versions, goldens and clone manifests do not change.
Executed locally:
- Default MSVC Release build succeeded. Initial sandboxed CTest passed 77/79;
filesystem/CLI hard-link fixtures were denied creation. The same 79 tests
rerun with normal filesystem rights passed **79/79, 13.60 s**.
- GCC 14 Release private population/history/registered-census qualification passed
**exit 0, 19 s**, including the original context capture/missing-context cases.
- Final Clang 19 **ASan/UBSan** population/history/census, migration, loopback TLS
request/reply loss and local storage-fault matrix passed **exit 0, 71 s**
(67 s population phase), without sanitizer reports.
- Focused private SQL qualification passed **exit 0, 8 s**, including domain
adoption/no-op stability, current-build/held-lease refusal, rollback, exact fork
cuts, idempotent and stale requests, runtime table/function denial, and actual
pre-context backups restored through isolated and in-place workflows. Those
legacy cases retain unknown original context; in-place migration adopts a
fresh domain without invented parent/fork records.
- Final clang-tidy 19 passed on the snapshot adapter and both changed integration
test files with no owned warnings. Repository clang-format 19, final `bash -n`
on all six changed/new shell scripts and `git diff --check` passed. Initial
analysis findings were corrected with explicit optional-value validation.
The first full run exited 1 after **74 s** because the frozen test directory
omitted the required `evo-load`/`evo-session-probe` sibling executables, causing
the process fixture's spawn failure. The failed log/sentinel are preserved. The
complete frozen-binary rerun passed **exit 0, 558 s**: C++ integration/process/
load/history **434 s**, migration TLS/storage faults **41 s**, life reservations
**4 s**, domain/legacy/backup/restore **31 s**, host-loss DR **7 s**, upgrade
**10 s**, migration history **6 s**, runtime permission audit **12 s** and
checksum/future-version rejection **8 s**, plus setup/migration. Its completion
sentinel records zero and the log contains the final all-phase success summary.
Actual schema-51 isolated and in-place restores have different source/current
domains with parent/exact-cut provenance. The restore claim binds the new domain
already at the post-install, pre-verification/readmission hook. Existing lost-
receipt, restart, quarantine and competing-restore cases also pass.
Snapshot tests change owner-controlled namespace metadata after head capture:
the selected result retains the old domain and the next result observes the new
domain even though manifests are unchanged. This is a trusted test fault seam,
not permission to fork an active environment. Removing the singleton context
fails the entire registered query and leaves the connection idle. Real restore
quarantine, parent/exact-cut provenance and claim binding are tested separately
by the supported backup/DR workflows.
Schema 51 is additive; prior numbered migrations 1..50 remain unchanged. Migration
history expectations now include 51, and unknown-future rejection uses 52. The
runtime rights audit requires the getter and rejects grants to the owner-only
fork function. No new dependency, worker thread, public client endpoint or live
authority is introduced. No new TSan run is claimed for this metadata change.
Limits: history adoption cannot recover unknown earlier context. Direct privileged
database copies/edits outside the supported restore workflow can copy/forge a
domain. Cross-domain clone-origin provenance, other viable/in-flight stores,
durable accepted reconciliation and complete M12 remain open. No target-host
query/capacity claim, remote CI, multi-host drill, actual multi-day soak, product
acceptance, live database, deployment or World0 action.
Logs: `build/m127b-build.log`, `build/m127b-final-build.log`,
`build/m127b-final2-build.log`, `build/m127b-native.log`,
`build/m127b-native-recheck.log`, `build/m127b-population.log`,
`build/m127b-domain-smoke.log`, `build/m127b-domain-smoke-recheck.log`,
`build/m127b-domain-final.log`, `build/m127b-domain-validated.log`,
`build/m127b-domain-legacy.log`, `build/m127b-final-asan-build.log`,
`build/m127b-final-asan.log`, `build/m127b-validated-tidy.log`,
`build/m127b-validated-format.log`, `build/m127b-shell.log`,
`build/m127b-full.log`, `build/m127b-full.exit`,
`build/m127b-full-recheck.log`, `build/m127b-full-recheck.exit` (completion only).
## 2026-10-18 - M12.7a registered-world census coverage
[ADR-0154](../adr/0154-registered-world-census-coverage.md)/schema 50 uses the
+6
View File
@@ -80,6 +80,12 @@ a worker recovery test.
2. Restore the newest verified bundle in place, as the migrator:
`restore-world.sh --bundle B --objects NEW_VOLUME --in-place --runtime-role ROLE --accept-rewind`.
Record every `REWIND` line: acknowledgements after those ticks are undone.
Schema-51 bundles also fork the database history domain before readmission;
retain the new/parent domains and exact `history_domain_cuts` with the restore
claim. External evidence keys worlds by `(history_domain, world_id)`.
Older bundles have no original domain: later migration records adoption only.
Direct database copies outside this workflow can copy a domain; see
[ADR-0155](../adr/0155-database-history-domain.md).
3. Point the environment's connection Secret and object volume at the replacement,
then sync the environment's Argo application; the host resumes under a newer epoch.
4. The moment the old database is reachable (or at once, if it never went away):