M12.6: derive replayed census history from immutable commits
Foundation / checks (deploy) (push) Successful in 6s
Foundation / checks (ecs) (push) Successful in 2m25s
Foundation / checks (network) (push) Successful in 20m50s
Foundation / checks (client) (push) Successful in 22m7s
Foundation / checks (tsan) (push) Successful in 24m42s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (postgres) (push) Canceled after 35m33s
Foundation / checks (checks) (push) Canceled after 35m43s

This commit is contained in:
dingo
2026-10-18 14:36:14 -04:00
parent da39fc8095
commit a180c41ae7
23 changed files with 732 additions and 18 deletions
+18
View File
@@ -1,5 +1,23 @@
# Project status and next steps
## M12.6b replayed census history from committed cuts - 2026-10-18
Regional census history now reads exact retained biological commits as canonical
durable events. Pages are bounded to eight commits and pinned to a fixed regional
cut. Shared full recovery verifies raw/cohort payloads, traces, transitions and
paired migration evidence before exposing census rows. Idle connections prevent
a writer's own pending changes from appearing durable; failed evidence never
becomes a partial page or zero population. No duplicate census write is added.
Native Release passes **79/79**; private GCC history qualification, ASan/UBSan
history/TLS/storage checks and Clang analysis/format checks pass.
Complete private persistence/process/backup/DR and SQL qualification passes in
**546 s**, including the new history matrix and existing fault tests.
[Evidence and limits](engineering/VALIDATION.md) retain world/branch reconciliation,
cross-region/public client access and deployment-host query costs. This regional
watermark cannot authorize terminal extinction. Linked ecology, maturation and
later integration/soak gates remain open; no deployment or World0 action.
## M12.6a bounded regional census and history queries - 2026-10-18
The portable census now distinguishes resident individuals/cohorts and
@@ -61,3 +61,8 @@ Durable append-only census/history with atomic commit binding, bounded backend
queries, branch/epoch/watermark reconciliation and client presentation remain
subsequent M12.6/12.7 work. Linked ecological resources and the rules-8 maturation
gap remain open independently of this observational primitive.
Follow-on ADR-0151 implements regional commit-bound history reads using the
existing immutable biological journal as canonical events, without a second
census-write window. Full world/branch reconciliation and client presentation
remain separate from both primitives.
@@ -0,0 +1,72 @@
# ADR-0151: Census history from retained biological commits
## Status
Accepted for scoped M12.6b implementation, 2026-10-18. This selects the existing
append-only biological commit journal as the durable source for regional census
history. No new SQL schema, biological format/rule or global extinction policy.
## Context and decision
ADR-0150's census is a detached observation. Publishing an independent mutable
summary after the biological transaction would create a second-write failure
window and a second demographic truth. Every accepted biological publication
already retains its exact manifest and complete payload in the immutable regional
journal, atomically with its head. Paired migration records both endpoints in one
transaction; uncommitted uploads and destination ghosts are not journal entries.
Use those committed cuts as canonical history events. Derive census projections
on read from their verified payloads and existing semantic replay evidence, rather
than storing duplicate census rows. A commit remains the event identity and its
digest binds the complete census source. There is no extra post-commit write,
backfill that invents observations, or new database write per simulation tick.
Add explicit historical raw/population recovery entry points that load an exact
journal sequence and run the same manifest/build, strong object digest, complete
restore, batch/no-tick/paired replay and outcome checks as current-head recovery.
Historical recovery is read-only; it never changes the selected head, resumes an
older owner or grants authority. Existing hosts continue to recover only the head.
Replay validates each selected cut's complete batch/no-tick operation from retained
predecessor checkpoints; it does not re-execute the entire world from genesis.
Evidence loaders retain their current regional-lease requirements; reading a
historical checkpoint is not proof of current ownership or world completeness.
Add bounded forward census pages, at most eight commits, pinned to a caller's
exact regional watermark (frame, sequence, commit ID, manifest digest and tick).
Validate that watermark against the immutable journal before and after reading.
Append-only head advancement cannot move the requested upper cut. Advance by
sequence, not tick: no-tick profile/format/migration commits remain distinct.
Every returned entry retains its exact committed manifest and detached census.
Require an idle connection at entry; reject all caller-owned transactions so a
publishing connection cannot present its own uncommitted writes as durable cuts.
The libpq adapter exposes read-only transaction status for this guard. Diagnostic
historical recovery retains the existing caller-owned transaction semantics.
Missing, unsupported, corrupt, unreplayable or foreign evidence fails the entire
page; never skip a cut or return a zero census for unavailable history.
All point lookups use the existing unique regional sequence index. Payload/state
work is one cut at a time, with at most eight bounded census projections retained.
Timeouts retain existing per-I/O semantics, not a page-wide service latency promise.
This is a backend read primitive, not a new endpoint or permission to expose it
directly to clients.
## Alternatives, costs and qualification
A separate mutable census table can disagree with the immutable biological source.
Adding summary bytes to every publisher would version the same redundant truth
through genesis, ordinary batches, profile/version transitions and paired commits.
Retained exact checkpoints already provide atomic durable history, at the cost of
object reads and semantic replay on query. No performance/capacity claim follows;
a later cache/index must be derivable and preserve these evidence checks.
Require stable pagination after new publication, sequence-zero and no-tick cuts,
raw/cohort restoration, historical migration ownership, exact retries without
extra events, current lease replacement, missing/corrupt/false replay evidence,
wrong cut/region/frame/build, bounded requests and unchanged heads/stock. Use
private PostgreSQL, sanitizer, analysis and complete persistence regression.
This primitive is scoped to the connected database's regional history/frame.
Its watermark is not a complete world watermark or an external recovery-branch
identity. Cross-database/clone/rewind reconciliation, world region-set coverage,
client presentation, actual multi-day ecology and terminal extinction remain
M12.6/12.7 and later gates. No live database, World0 or deployment action.
+2
View File
@@ -1,5 +1,7 @@
# Architecture decision records
- [ADR-0151: Committed cellular census history](0151-committed-cellular-census-history.md) - derives bounded, exact-cut census pages from the existing atomic immutable biological journal, with full historical replay; no duplicate census writes or world-extinction authority.
- [ADR-0150: Regional cellular census projection](0150-regional-cellular-census-projection.md) - bounded local resident/representation/presence census and exact/descendant queries; retained nonresident history never counts as owned stock or world extinction.
- [ADR-0149: Atomic paired biological migration publication](0149-atomic-cellular-migration-publication.md) - schema 49 binds both current fences, heads and exact candidates in one immutable receipt/transaction; recovery replays the pair and hosts verify the changed head before readiness.
@@ -1,5 +1,15 @@
# Persistence and region authority
M12.6b derives regional census history from the existing immutable biological
commit journal. A commit and its exact payload are the canonical event, already
atomic with the head; no duplicate census-write window or new SQL table is added.
Explicit historical recovery reuses the full current-head replay checks. Pages
are bounded to eight consecutive commits and pinned to an exact regional cut;
unavailable/false evidence fails the entire page. Idle connections prevent a
writer's uncommitted cuts from appearing durable. See
[ADR-0151](../adr/0151-committed-cellular-census-history.md). This is regional
history, not branch/world completeness, a client endpoint or extinction authority.
M12.4f defines cellular bundle 10 for owner-7 boundary accounting, with an explicit
same-profile, no-tick bundle-9 upgrade. M12.4g/schema 46 admits it through the
existing immutable version-upgrade journal and lease/head fence. Recovery
@@ -10,6 +10,10 @@ It reports local resident stock by current genome and exact/descendant scope,
including suspended players; retained nonresident records carry no owned stock.
This detached observation has no commit/branch binding or world coverage and
does not implement the global evidence envelope or terminal extinction below.
M12.6b's [committed regional history](../../schemas/ecology/committed-cellular-history-v1.md)
binds those projections to exact retained biological commits with full replay.
The commit journal supplies atomic durable events without a second census write.
Its fixed regional cut still supplies no complete world or external branch proof.
## Identity and classification
+1 -1
View File
@@ -13,7 +13,7 @@ Commit and push each validated scoped segment separately.
| 12.3 | Exact population profile/trace/store and opt-in host integration implemented (12.3b) | ADR-0142, schema 44; exact publication/recovery, mutation race guards, ambiguous replies and process kills. Executable/network population fault drills and target-host qualification remain open. |
| 12.4 | Owned intervals and durable paired migration/fault qualification implemented in scoped segments through 12.4l | ADR-0143–0149/schema 49; bounded admitted work, source freeze, staged ghost, atomic pair and exact recovery, local TLS and storage faults. Multi-host/orchestrator, linked ecology and integrated stock audits remain open. |
| 12.5 | Blocked on 12.2-12.4 | Finite M6 process fixtures do not qualify coupled persistent cellular/cohort atmosphere/water budgets. |
| 12.6 | Local archive plus regional resident/presence/cohort census and exact/descendant queries implemented (12.6a) | ADR-0150; local detached observations exclude departed history from stock. Commit/branch binding, atomic durable events and world-level query/client path remain required. |
| 12.6 | Regional census/lineage queries and replayed commit-bound history pages implemented (12.6a/b) | ADR-0150/0151; existing atomic biological commits are the durable events. Fixed regional watermarks and full replay bind observations; world/branch context, cross-region query/index costs and client path remain required. |
| 12.7 | Evidence policy defined; reconciler open | Complete durable world watermark and every viable/in-flight store remain required. |
| 12.8 | Not qualified | Actual multi-day integrated ecology/restart/clone evidence remains required. |
| 12.9 | Not accepted | Returning-player product evidence and expansion review remain required; World0 stays separately blocked. |
+9 -4
View File
@@ -27,7 +27,7 @@ or authorize production enablement.
| 12.3 | Dedicated population profile/trace/store/owning host; exact recovery and private PostgreSQL fault tests | Population-specific executable/network fault drills and target-host publication/capacity qualification. |
| 12.4 | Owned intervals; exact transfer/accounting; durable atomic pair/recovery; independent loopback TLS path loss and owner expiry/replacement/heal; local object sync/read-failure matrix | Multi-host/kernel-network, storage-host and orchestrator drills, linked-boundary contracts and integrated ownership/stock audits at scale. |
| 12.5 | Finite cellular contact feeding/decay and separate M6 producer/consumer resource owner | One jointly owned cellular/environment budget, explicit effective depth and source entitlements, versioned water/atmosphere species mapping and producer coefficients, plus closed/open and nutrient-exhaustion comparisons. |
| 12.6 | Retained archive plus bounded regional resident/presence/cohort census and exact/descendant queries (12.6a) | Append-only population census/history events in the same durable commit, bounded world history queries/indexing and client presentation; no account power or invented precise statistical ancestry. |
| 12.6 | Retained archive, bounded regional census/lineage queries and fully replayed census history pages from atomic biological commits (12.6a/b) | World/branch history context and cross-region query/index qualification, client permission/presentation path and deployment-host query costs; no account power or invented precise statistical ancestry. |
| 12.7 | ADR-0133's viability/unknown semantics | Durable region-set/watermark reconciliation covering every owned representation, suspended/viable stock and migration escrow. Missing/stale regions must prevent terminal extinction. |
| 12.8 | Deterministic seed/regime drivers and separate-process exact continuation | Reproducible integrated worlds running for actual days, real restarts/clone comparisons and retained accounting/diversity/cost audits. Accelerated tests cannot supply elapsed days. |
| 12.9 | M11 feedback and current UI review retained | Returning-player samples on the integrated ecology and explicit product/3D readiness decision after soak evidence. No M12 product acceptance or World0 promotion is inferred. |
@@ -98,9 +98,14 @@ M12.6a's [regional census projection](../../schemas/ecology/regional-cellular-ce
independently implements local current-genome and descendant-closure queries.
It keeps suspended stock and retained zero-carrier ancestry, separates exact cohort
tags from additional ownership, and excludes nonresident history from living stock.
It has no commit/branch/epoch binding or complete world coverage, so it cannot
publish authoritative world history or accept terminal extinction. Durable atomic
history publication and M12.7 reconciliation remain the next history boundaries.
The local projection itself has no commit/branch/epoch binding or complete world
coverage. M12.6b's [committed history reader](../../schemas/ecology/committed-cellular-history-v1.md)
now derives it from the existing immutable atomic biological journal, with full
historical replay, exact manifest binding and a fixed regional upper cut. No
redundant census write or separate publication window is added. That regional cut
is not an external branch identity or complete world watermark. World-level
history context/queries, client presentation and M12.7 reconciliation remain open;
neither primitive accepts terminal extinction.
Current expansion readiness: **no-go for M13 integration or World0 promotion**
while these M12 integration/evidence gates remain open. This records the present
+14
View File
@@ -1,5 +1,19 @@
# Validation strategy
M12.6b's private PostgreSQL history tests require exact fixed-watermark pagination
after new no-tick publications, preserved genesis/profile/migration cuts, unchanged
heads/stock and repeated event identity. Invalid limits, false cuts/regions/frames,
unretained builds, stale evidence readers and caller-owned transactions fail.
Missing/corrupt historical payloads and missing traces fail the entire page,
including after an earlier valid entry. Independent connections cannot read a
writer's pending cut. Historical raw/cohort recovery rejects canonical false
migration biology; missing counterpart capsules fail pages. Earlier migration
observations remain identical after return transfers under newer leases and match
the complete portable pair oracle. Existing head recovery shares those validators;
run sanitizer/analysis and complete private persistence regression for the refactor.
Global branch/region-set reconciliation, public client access and target-host
query costs remain separate gates.
M12.6a's regional census tests compare per-category counts and body/reserve
inventories with direct individual observations. Cover active/lingering/suspended
controlled lives, real funded wildlife, exact cohort conversion/restoration,
+59
View File
@@ -1,5 +1,64 @@
# Validation report
## 2026-10-18 - M12.6b committed regional census history
ADR-0151 uses the retained immutable biological commit journal as canonical
regional census events. That source already publishes atomically with the head;
no duplicate census-write window, new table or schema migration is added. New
historical raw/population recovery entry points share the unchanged complete
validators with head recovery. History pages retain exact manifests and census
projections for at most eight consecutive commits, pinned to an exact regional
upper cut. See [the contract](../../schemas/ecology/committed-cellular-history-v1.md).
Executed locally:
- Native MSVC **19.44.35217.0** Release full CTest **79/79 passed**, **14.51 s**.
- GCC 14 optional persistence/process targets rebuilt; private population/history
qualification passed **16 s** (15 s population phase), with NDEBUG.
- Linux Clang 19 **ASan/UBSan** combined history/TLS/storage fault matrix passed
**67 s** (64 s population phase), without reports.
- Explicit clang-tidy 19 on the PostgreSQL status adapter, history reader, shared
raw/population recovery and integration tests passed with no owned warnings.
Repository clang-format 19 and `git diff --check` passed.
- Complete frozen-binary private PostgreSQL qualification passed with **exit 0**,
**546 s**: C++ integration/process/load/history (434 s), combined history/TLS/
storage faults (39 s), life reservations, backup/restore, host-loss DR, upgrade,
migration history/rollback, runtime permission audit and checksum/future-version
rejection. The detached runner recorded its completion code.
Tests retain genesis and separate no-tick profile cuts, pin pages across later
publication and compare complete census rows with independently restored source
payloads. Invalid limits/regions/frames/cut digests/builds and stale evidence
readers fail. Missing/corrupt old payloads and missing traces fail an entire page,
even after a valid genesis entry. Independent connections cannot see pending
writer cuts. Public history requires a connected idle session; explicit writer
and read-only transactions are refused. Closed connections after real TLS reply/
request loss report non-idle, without a query or crash.
Historical raw/cohort recovery rejects a valid old object falsely declared as
migration biology. Missing counterpart capsules fail history pages. Both endpoint
observations match the exact portable pair, remain identical after return
transfers and are read successfully under newer epochs than their original
writers. Reads preserve current heads and biology; they never resume old state.
The first fixture draft replaced its lease before a separate test's account
control reservation, correctly invalidating that old control. The independent
history guard now uses a deliberately stale token; real replacement coverage
belongs to the existing migration/host replacement matrix. Static analysis found
an unchecked optional cursor in a test; the test now explicitly requires it.
Review also added the transaction-idle guard to exclude a writer's own pending
rows from the public durable-history API. No checks were suppressed.
Limits: this is regional backend history within the connected database's current
frame/context, not an external branch identity or complete world watermark.
World/branch reconciliation, cross-region/public client queries, derived indexes
and deployment-host query latency/capacity remain open. Per-I/O deadlines are not
a page-wide service SLO. Each selected cut uses the same complete tail/no-tick
replay from retained predecessors as head recovery; no genesis-to-cut execution
is claimed. No new TSan, Docker or remote-CI result is claimed. No thread/dependency,
biological format/rule, golden, deployment or World0 changed.
Unrelated `Build_Errors.txt` and `tash drop` remain untracked.
## 2026-10-18 - M12.6a bounded regional census and lineage queries
ADR-0150 adds a detached rules-8 regional projection, with explicit presence and