M12.3: publish and recover durable exact populations
Foundation / checks (ecs) (push) Successful in 2m41s
Foundation / checks (tsan) (push) Failing after 18m12s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (network) (push) Canceled after 20m41s
Foundation / checks (postgres) (push) Canceled after 20m43s
Foundation / checks (checks) (push) Canceled after 20m49s
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (client) (push) Failing after 17m4s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / checks (ecs) (push) Successful in 2m41s
Foundation / checks (tsan) (push) Failing after 18m12s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (network) (push) Canceled after 20m41s
Foundation / checks (postgres) (push) Canceled after 20m43s
Foundation / checks (checks) (push) Canceled after 20m49s
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (client) (push) Failing after 17m4s
Foundation / postgres-tools (push) Canceled after 0s
This commit is contained in:
@@ -1,5 +1,20 @@
|
||||
# Project status and next steps
|
||||
|
||||
## M12.3b durable exact populations - 2026-10-17
|
||||
|
||||
ADR-0142 and schema 44 add dedicated population profile 5, trace 1, immutable
|
||||
publication/transition evidence and exact source/target recovery. The owning
|
||||
cellular host recovers either representation, materializes before player
|
||||
operations and serializes mutation reservations against profile entry. The
|
||||
optional `--cellular-populations on` path batches unattended work; entry is off
|
||||
by default. No biological version, golden, deployment or live region changed.
|
||||
|
||||
[Scope and evidence](engineering/M12_DURABLE_POPULATIONS.md) distinguish portable
|
||||
contracts, real PostgreSQL ambiguity/process-loss tests and host recreation from
|
||||
remaining executable/network fault drills and target-host qualification. Durable
|
||||
interval admission/backlog is the next M12.4 segment. The dependent ecological,
|
||||
multi-day and returning-player gates remain open; M11.9 stays closed for now.
|
||||
|
||||
## M12.4a current-worker unattended scheduling - 2026-10-17
|
||||
|
||||
The cellular worker now keeps ticking for funded wildlife after players suspend,
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
# ADR-0142: Durable exact population profile and publication
|
||||
|
||||
## Status
|
||||
|
||||
Accepted for M12's opt-in engineering implementation, 2026-10-17. Extends
|
||||
ADR-0139/0140 with a dedicated durable profile. Production enabling and target-host
|
||||
qualification remain separate; no deployment or World0 operation is authorized.
|
||||
|
||||
## Context
|
||||
|
||||
The exact cohort envelope carries membership and policy beyond raw cellular
|
||||
bytes. Reusing profile 4 would make retained manifests ambiguous. Portable
|
||||
restoration alone cannot prove atomic publication, current authority or recovery
|
||||
after a lost database reply.
|
||||
|
||||
## Decision
|
||||
|
||||
Checkpoint profile 5 contains population envelope 1 and nested cellular bundle 9,
|
||||
with history profile 5 identifying exact retained membership. Profile 4 retains
|
||||
its raw cellular bytes and meaning. No biological version or golden is replaced.
|
||||
|
||||
Use explicit fenced, no-tick profile transitions: rules-8 profile 4 to population
|
||||
5 wraps the same biology with empty membership, sequence zero and a persisted
|
||||
policy; population 5 to profile 4 materializes the retained individuals by
|
||||
unwrapping the identical biology. Both require the exact current predecessor,
|
||||
current region fence, retained build and immutable transition evidence. Recovery
|
||||
checks the complete source/target pair, not just their aggregate inventory.
|
||||
Conversion sequence starts at zero for each entry generation; the region commit
|
||||
sequence remains monotonic and the transition records identify every generation.
|
||||
|
||||
Initially profile 5 admits unattended ecological batches only: controlled cells
|
||||
receive frozen intentions; player births, mutation selections and session action
|
||||
receipts remain profile-4 operations. A host must finish the reverse transition
|
||||
before processing those operations. This restriction avoids bypassing their
|
||||
existing account-control/reservation transactions. It does not freeze wildlife
|
||||
or erase controlled/suspended stock. An explicit profile selector is opt-in.
|
||||
The cellular worker accepts `--cellular-populations on|off` (default off). It may
|
||||
enter only for unattended rules-8 wildlife, with no waiting player birth or
|
||||
mutation. Session demand or a player operation completes the reverse transition
|
||||
first. SQL serializes profile entry and mutation reservation under the same
|
||||
region lease: entry refuses outstanding choices, and new choices require profile
|
||||
4 in the current host. Historical reservation contracts remain unchanged outside
|
||||
profile 5. An existing profile-5 head remains recoverable with entry disabled.
|
||||
The worker collects at least 20 due ticks (one second) and services at most 64
|
||||
per unattended publication, consuming local debt only on confirmation. This is
|
||||
bounded publication batching; durable interval admission/restart backlog is the
|
||||
next M12.4 segment, not supplied by a wall-clock inference on recovery.
|
||||
|
||||
Population batch format 1 records the unchanged canonical cellular intentions
|
||||
and an exact membership/sequence boundary for each tick. Biological replay reads
|
||||
the recorded intentions, never reruns the wildlife controller. It verifies each
|
||||
recorded boundary against the persisted deterministic policy. Manual conversion
|
||||
records operate on an entire wildlife genome group at the same biological tick.
|
||||
No random materialization sample exists in this exact model. Restoration installs
|
||||
the recorded checkpoint; validation cannot resample a different member set.
|
||||
|
||||
Store immutable payload and trace objects with SHA-256/read-back verification,
|
||||
then commit the manifest and publication evidence in one database transaction
|
||||
under the existing serialized lease/head guard. Unknown replies retain the exact
|
||||
prepared request; read-only resolution never grants a stale writer permission to
|
||||
continue. Dedicated evidence triggers prohibit generic publication bypasses.
|
||||
Retain all old formats, histories and source checkpoints. Do not prune objects.
|
||||
|
||||
## Validation and limits
|
||||
|
||||
Require malformed/profile-confusion rejection, whole-state source/target equality,
|
||||
conversion oscillation, trace/payload mismatch refusal, frozen controlled biology,
|
||||
duplicate/conflicting retries, expired writers and missing/corrupt objects. Exercise
|
||||
real PostgreSQL and process death before publication and after commit/receipt.
|
||||
Portable codec success alone does not close the durable gate.
|
||||
|
||||
This profile supplies a place for bounded unattended intervals and later escrow;
|
||||
it does not itself implement biological migration or environmental coupling.
|
||||
Existing player actions, NPC heredity and fixed 50 ms biology retain their rules.
|
||||
|
||||
## Alternatives, costs and consequences
|
||||
|
||||
Relabeling the raw profile breaks retained recovery; persisting only aggregate
|
||||
stock loses exact members. Reconstructing policy on restart can choose a different
|
||||
representation. Keep explicit profiles and recorded boundaries instead.
|
||||
|
||||
This retains every member and full trace, so it adds durable bytes and replay
|
||||
work and makes no compression claim. SQL parsers, grants and source/target
|
||||
verification must evolve together. Target-host cost and executable/network fault
|
||||
qualification remain risks before production enablement. All old objects and
|
||||
histories remain retained; no garbage collection or compatibility reset follows.
|
||||
@@ -31,6 +31,15 @@ the narrower environmental profile. M7.1 completes the audit, not those codecs.
|
||||
|
||||
## Identity, authority and watermarks
|
||||
|
||||
ADR-0142 adds opt-in exact population profile 5 (envelope 1, nested cellular
|
||||
bundle 9, history profile 5). Raw cellular profile 4 is unchanged. Fenced no-tick
|
||||
transitions retain complete biology and source/target digests; normal population
|
||||
publication binds recorded intentions and exact membership boundaries. Generic
|
||||
publishers cannot bypass dedicated immutable evidence. The host has one owning
|
||||
state, materializes before player operations and never treats read-only commit
|
||||
resolution as current authority. See [the contract](../../schemas/cellular-population-batch.md)
|
||||
and [qualification limits](../engineering/M12_DURABLE_POPULATIONS.md).
|
||||
|
||||
World/region/frame are nonzero durable IDs. No ID is a Pod name. The host records
|
||||
its unique holder ID and monotonically increasing nonzero lease epoch. Exhaustion
|
||||
of IDs, epochs, commit sequences or ticks fails closed; never wrap/reuse them.
|
||||
|
||||
@@ -10,7 +10,7 @@ Commit and push each validated scoped segment separately.
|
||||
|---|---|---|
|
||||
| 12.1 | Engineering evidence policy and explicit deferrals documented | ADR-0133, identity/time contract and worked histories; no new game rules or runtime versions. |
|
||||
| 12.2 | Exact local model selected and paired qualification implemented (12.2d); production cost gate open | ADR-0139, exact canonical equality across 196,608 paired ticks per platform. Regional approximation disabled; no compression claim. Deployment-host capacity and fuller interaction coverage remain open. |
|
||||
| 12.3 | Portable exact conversion closure, automatic policy and process recovery implemented | ADR-0140, population envelope 1. Dedicated PostgreSQL profile/trace/publication adapter and fault injection remain required before enabling the live selector. |
|
||||
| 12.3 | Exact population profile/trace/store and opt-in host integration implemented (12.3b) | ADR-0142, schema 44; exact publication/recovery, mutation race guards, ambiguous replies and process kills. Executable/network population fault drills and target-host qualification remain open. |
|
||||
| 12.4 | Current-worker unattended demand and confirmed-head pacing implemented (12.4a) | ADR-0141. Population durable intervals and biological migration escrow remain unimplemented. |
|
||||
| 12.5 | Blocked on 12.2-12.4 | Finite M6 process fixtures do not qualify coupled persistent cellular/cohort atmosphere/water budgets. |
|
||||
| 12.6 | Existing local archive retained; integrated population history open | World-level durable event/query/client path remains required. |
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
# M12.3b durable exact populations
|
||||
|
||||
ADR-0142 implements the persistence boundary identified in the remaining-gate
|
||||
audit. This is an opt-in engineering path; no live region was migrated or enabled.
|
||||
The optional cellular worker accepts `--cellular-populations on|off`, default off.
|
||||
When unattended rules-8 wildlife is present, entry can wrap its exact biological
|
||||
state as profile 5. A returning player, queued birth or mutation requires the
|
||||
durable reverse transition before processing player operations.
|
||||
|
||||
## State and publication
|
||||
|
||||
There is exactly one owning biological store in the host. Profile 4 owns raw
|
||||
CellularLives; profile 5 owns a CellularPopulation containing that store and its
|
||||
exact membership tags. Common callers receive only a const biological view.
|
||||
Entering, leaving and oscillating cannot heal, move, reproduce or fund a life.
|
||||
Every profile-5 generation starts membership sequence zero, while the immutable
|
||||
region publication sequence and source/target evidence preserve its identity.
|
||||
|
||||
Migration 44 retains all old parsers, histories and raw cellular formats. It adds
|
||||
dedicated immutable population publication and profile-transition records, fenced
|
||||
publish/resolve/load functions and generic-publication bypass guards. Entry and
|
||||
new mutation reservations serialize on the same region lease. Entry refuses
|
||||
outstanding player choices; profile 5 refuses new choices. Existing unrelated
|
||||
reservation contracts retain their original behavior.
|
||||
|
||||
Payloads and traces are immutable content-addressed objects. The publisher checks
|
||||
their SHA-256 and complete read-back before the atomic SQL head/evidence update.
|
||||
Recovery verifies the exact predecessor and full source/target bytes. Biological
|
||||
traces record intentions and exact membership boundaries; replay does not rerun
|
||||
the wildlife controller. Unknown replies stop publication until exact-byte
|
||||
resolution/retry succeeds under current authority.
|
||||
|
||||
## Evidence and qualification limits
|
||||
|
||||
The portable contract covers canonical codec/manifest/profile confusion, sampled
|
||||
truncations, appended bytes, membership/sequence tampering, controlled-input
|
||||
refusal, manual conversions and full biological replay equality. Native Release
|
||||
tests run under NDEBUG; sanitizer and static-analysis evidence is recorded in
|
||||
[VALIDATION](VALIDATION.md).
|
||||
|
||||
The private PostgreSQL fixture creates real reserved founders and waits for a
|
||||
funded player-derived descendant. It exercises durable membership oscillation,
|
||||
fresh-connection recovery, reverse materialization, generic bypass rejection,
|
||||
mutation/profile race guards, duplicates/conflicts and stale authority. Missing
|
||||
and corrupt payloads refuse recovery. Lease-row contention creates real adapter
|
||||
timeouts: one backend commits after its client disconnects; another is terminated
|
||||
before commit. Both resolve/retry the retained exact request.
|
||||
|
||||
SIGKILL tests use forked publisher processes at the pre-transaction boundary
|
||||
(after immutable uploads), after SQL commit and after a pipe acknowledgement.
|
||||
The parent restores through a new database connection and compares every retained
|
||||
byte. These are real process kills, but the pipe acknowledgement is a fixture
|
||||
signal, not an actual network client acknowledgement or an independently exec'd
|
||||
worker restart. The actual CellularRegionHost is separately drained/recreated
|
||||
and restores a profile-5 head with entry disabled, then materializes before a
|
||||
returning-player operation. Previous raw worker process drills remain regression
|
||||
coverage, not additional population-specific fault evidence.
|
||||
|
||||
The worker collects one second of due unattended time before publication and
|
||||
services at most 64 fixed steps per batch. The portable input builder matches an
|
||||
independent per-tick fixture. This removes per-tick SQL publication from the new
|
||||
population path; it does not yet make local pacing debt a durable admission queue.
|
||||
|
||||
Still open: profile-5 executable/network fault drills, deployment-host tail costs,
|
||||
durable ecological interval admission and biological migration escrow, jointly
|
||||
owned environmental budgets, world-level history/extinction reconciliation and
|
||||
actual multi-day/returning-player evidence. This work changes neither biological
|
||||
versions nor goldens, and does not establish useful compression or M13 readiness.
|
||||
@@ -5,31 +5,24 @@ all of M12. M11.9 remains closed for now. M12.2 decisions are delegated and reco
|
||||
in ADR-0139; no additional product approval is inferred as a coding prerequisite.
|
||||
Completed scoped commits do not mean their entire roadmap gates have passed.
|
||||
|
||||
## Persistence boundary before expansion
|
||||
## Persistence boundary resolved by 12.3b
|
||||
|
||||
The new exact population envelope cannot be labeled as the existing profile-4
|
||||
CellularLives payload. The live publisher/recovery path, immutable manifest,
|
||||
database validators and trace currently agree on that raw cellular format. A
|
||||
population checkpoint containing ownership tags and policy needs a dedicated
|
||||
profile and versioned transition/publication adapter. Reinterpreting profile 4
|
||||
would violate REGION_DURABILITY and break retained recovery evidence.
|
||||
|
||||
Resolution: add an opt-in population profile and an explicit no-tick migration
|
||||
from the accepted rules-8 cellular state, retaining identities, archive, funding
|
||||
and balances. Publish its complete immutable payload, bound trace and permanent
|
||||
events under the current region fence. Replay must retain recorded intentions
|
||||
and exact membership samples, rather than rerunning a decision after an uncertain
|
||||
commit. Verify lost replies and pre-upload/pre-transaction/post-commit/post-ACK
|
||||
process death against a private PostgreSQL fixture. Keep production disabled until
|
||||
these and target-host budgets pass. A portable codec is not a substitute for this
|
||||
adapter. This is remaining M12.3 implementation, not a user-permission hold.
|
||||
ADR-0142 adds dedicated profile 5, population trace 1 and schema 44; raw profile 4
|
||||
retains its meaning. Explicit no-tick migrations preserve the complete biology.
|
||||
The publisher, replay verifier, immutable SQL evidence, recovery and owning host
|
||||
now support exact population membership with entry disabled by default. Private
|
||||
PostgreSQL tests cover ambiguity, fencing, profile races and publisher process
|
||||
kills. [Scope and limits](M12_DURABLE_POPULATIONS.md) distinguish those tests from
|
||||
executable/network fault drills and deployment-host qualification still required.
|
||||
This resolves the architectural profile conflict and permits the next durable
|
||||
interval work. It does not close all of M12.3 or authorize production enablement.
|
||||
|
||||
## Dependent ecological integration
|
||||
|
||||
| Gate | Existing useful basis | Work/evidence still required |
|
||||
|---|---|---|
|
||||
| 12.2 | Exact cellular model and paired matrix; local samples through 256 lives | Deployment-host tail costs including publication; larger wildlife cohorts, predation/twins and contention coverage; useful compression remains unimplemented. |
|
||||
| 12.3 | Complete population codec, exact conversion candidates, automatic policy and fresh-process continuation | Dedicated durable profile/trace/store/host integration and fault-injected atomic publication described above. |
|
||||
| 12.3 | Dedicated population profile/trace/store/owning host; exact recovery and private PostgreSQL fault tests | Population-specific executable/network fault drills and target-host publication/capacity qualification. |
|
||||
| 12.4 | Current worker advances unattended wildlife and preserves running scheduling debt | Durable ecological interval admission/backlog recovery; exact biological source/destination escrow, partition/duplicate settlement and stock-in-transit audits. |
|
||||
| 12.5 | Finite cellular contact feeding/decay and separate M6 producer/consumer resource owner | One jointly owned cellular/environment budget, explicit effective depth and source entitlements, versioned water/atmosphere species mapping and producer coefficients, plus closed/open and nutrient-exhaustion comparisons. |
|
||||
| 12.6 | Retained genome/organism archive with ancestry and common-ancestor queries | Append-only population census/history events in the same durable commit, bounded world history queries/indexing and client presentation; no account power or invented precise statistical ancestry. |
|
||||
@@ -46,9 +39,9 @@ and photosynthesis steps B/C do not silently become accepted M12 mechanics.
|
||||
|
||||
The engineering constitution requires stopping expansion at this architectural
|
||||
boundary and documenting the resolution. That does not prevent the independent
|
||||
worker scheduling correction, which uses the existing raw cellular publication
|
||||
path. The outstanding population integration is the next substantive coding
|
||||
segment; none of the rows above is closed by adding a placeholder service.
|
||||
worker scheduling correction or the dedicated exact population adapter above.
|
||||
Durable ecological intervals are the next substantive coding segment; none of
|
||||
the rows above is closed by adding a placeholder service.
|
||||
|
||||
Current expansion readiness: **no-go for M13 integration or World0 promotion**
|
||||
while these M12 integration/evidence gates remain open. This records the present
|
||||
|
||||
@@ -1,5 +1,51 @@
|
||||
# Validation report
|
||||
|
||||
## 2026-10-17 - M12.3b durable exact population publication
|
||||
|
||||
ADR-0142 affects portable lifecycle batches/checkpoint manifests, optional
|
||||
PostgreSQL adapters/schema 44, the owning cellular host and opt-in worker batching.
|
||||
No biological version or golden changes; no deployment or World0 operation.
|
||||
|
||||
Executed:
|
||||
|
||||
- MSVC 19.44 Release configure/build and **76/76 CTests** (final run 9.53 s).
|
||||
- GCC 14.2 actual QUIC/PostgreSQL worker and integration-test build. Final private
|
||||
`postgres-check.sh --population-only` **passed, exit 0, 5 s** after checked-access
|
||||
refinements. Real reserved founders fund the wildlife fixture; tests exercise
|
||||
exact membership oscillation, duplicate/conflicting requests, mutation/profile
|
||||
race guards, missing/corrupt payloads, stale authority, host recreation and
|
||||
materialization before player operations.
|
||||
- Real PostgreSQL lease-row contention makes the adapter time out. One queued
|
||||
request later commits and another backend is terminated before commit; both
|
||||
resolve/retry the retained bytes. Forked publisher SIGKILLs occur before the
|
||||
transaction, after commit and after a pipe ACK, with fresh-connection recovery.
|
||||
This does not claim a population-specific network ACK or exec'd worker kill.
|
||||
- Clang 19 actual server build; explicit clang-tidy 19 on changed optional store,
|
||||
SQL adapter, cellular recovery, host, population integration tests and worker.
|
||||
Final refinements use separately checked stable optional values; those follow-up
|
||||
tidy runs pass. Analysis-preset portable checks and repository format check pass.
|
||||
- Portable analysis CTests pass **7/7** (29.70 s); final batch-builder/checkpoint
|
||||
checks pass **2/2** (3.73 s). ASan/UBSan population checkpoint/replay checks pass
|
||||
(final 12.50 s), with no reports. Tests use explicit checks under NDEBUG.
|
||||
|
||||
The broad regression first exposed an over-broad mutation guard and the schema-44
|
||||
upgrade expectation; both were corrected. The historical failed-migration drill
|
||||
now supplies all required checksums, so it reaches its intended bad-v8-checksum
|
||||
failure rather than an unrelated missing psql variable. The C++ suite, backup and
|
||||
DR passed in the next run, but a concurrent local relink interrupted the upgrade
|
||||
drill with `Text file busy`; this is not counted as a passing full run. Final
|
||||
qualification uses a frozen copy of the executables to avoid that interference.
|
||||
|
||||
Final frozen-binary `postgres-check.sh --full` **passed, exit 0, 496 s**: complete
|
||||
C++ suite (421 s), independent reservation connections, backup/restore, host-loss
|
||||
DR, upgrade, migration history, permissions and checksum drift. The final optional
|
||||
checked-access refinements additionally passed the targeted private population
|
||||
suite above; all populations remain opt-in and no live region was changed.
|
||||
|
||||
Remaining scope: [M12_DURABLE_POPULATIONS](M12_DURABLE_POPULATIONS.md) and
|
||||
[M12_REMAINING_GATES](M12_REMAINING_GATES.md). No compression, deployment-host
|
||||
capacity, durable interval admission or multi-day ecological acceptance is inferred.
|
||||
|
||||
## 2026-10-17 - M12.4a unattended demand and confirmed-head pacing
|
||||
|
||||
ADR-0141 changes the existing cellular worker's demand/pacing and metrics, with
|
||||
|
||||
Reference in New Issue
Block a user