ADR-0062 adds a canonical LineageArchive codec (parents before children, organisms by id, FNV) that decodes by replaying admit/record_birth/ record_death and requires byte-identical re-encoding; it never overwrites an existing archive. LineageRegion binds the environmental producer and consumer accounts to archived organisms with identity high-waters, enforces closure (lifecycle vs deaths, exact living set, tick bounds) and records deaths in the tick that causes them. Checkpoint profile 2 (environmental-lineage-v1, history profile 1) publishes that container; schema v4 admits it with a 4,268,144-byte cap and requires each commit to keep its predecessor's profile, RNG, history and durability declarations. Linux adds provision/publish/recover for the profile on the shared publication path. Qualification found WSL2 stepping CLOCK_REALTIME back ~0.5 s every ~32 s, which made two 1 s-lease tests flaky under ASan. Tests now wait on database-time expiry; ADR-0060 records the slewed-clock requirement. Validated: Windows CTest 57/57; Linux GCC Debug/Release, clang-tidy and ASan/UBSan CTest 57/57 each; PostgreSQL suites 2x Release, 2x analysis, 5x ASan; SQL and C++ mutations each caught. No Docker/Gitea CI, remote store, process kill or deployment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
13 KiB
M7 region durability contract
Design accepted by ADR-0058; no storage/host implementation yet. Locked invariants: one committed authority, no lost acknowledged permanent outcome under the declared failure model, no fallback empty world, and no database dependency in headless code.
State closure audit
| State | Existing evidence | M7 treatment / missing work |
|---|---|---|
| M6 environment, two accounts/capacities, terrain and debris | EnvironmentalCommunity envelope v2 and nested codecs | Required exact payload for environmental profile |
| Config, source/growth/efficiency residuals, lifecycle, tick, last funded interval | Same envelope | Restore all; never infer from tick or defaults |
| Process retries/audit anchor, resource revisions | Same envelope, 64 records | Preserve; not a durable command deduplication service |
| Interval scheduler | Stateless M6.7 adapter | No clock/debt to serialize; host queues are separate |
| Genome | Individual Genome v1 codec | No M6 account binding; unsupported in initial profile |
| LineageArchive and dead organisms | Separate append-only API/tests, no complete codec | Add bounded export/restore and binding before ancestry recovery is claimed; retain extinct genomes |
| Population/cohorts | Separate M4 state/codecs | No integration with M6; profile must explicitly add complete ownership before admission |
| Particle entities/positions/RNG | Independent proof snapshot | Not part of M6; never concatenate and imply unified ownership |
| Environmental randomness | None in rules 1/2 | Test-driver RNG is not simulation state; future runtime RNG needs versioned streams |
| Pending host commands | Not implemented | Bounded host queue; uncommitted receipt is provisional; durable decisions stored separately |
| Durable decisions/history/outbox | Not implemented | Required transactional records with commit references and immutable request bytes |
| Player accounts, logout, network sessions, cross-region transfers | Not implemented | Unsupported capabilities; initial profile cannot host players |
| Lease/routing readiness | Not simulation state | Live database authority, never resumed from a saved lease |
M7.4 must enumerate any newly attached state and extend its schema and tests before admitting it. For a history-capable profile, validate all organism-to-genome bindings, parent closure, lifecycle agreement, identity high-waters and archived dead members. Both checkpoint and durable history must agree. Empty/unrecognized sections cannot stand in for missing history. M7.9's retained-ancestry criterion is not waived by the narrower environmental profile. M7.1 completes the audit, not those codecs.
Identity, authority and watermarks
World/region/frame are nonzero durable IDs. No ID is a Pod name. The host records
its unique holder ID and monotonically increasing nonzero lease epoch. Exhaustion
of IDs, epochs, commit sequences or ticks fails closed; never wrap/reuse them.
Simulation tick is local to the fixed versioned region rules. simulated_tick
may lead committed_tick only within the current bounded pending batch.
commit_sequence counts publications, not ticks or field commands. Genesis has
sequence 0, tick 0, no predecessor and an explicit authorized opening payload.
Only provisioning can create genesis; recovery cannot do so.
The database arbitrates acquire/renew/commit/release under the same region lease row lock, using database expiry and matching holder/epoch. Commit additionally compares expected predecessor sequence, digest and tick. No direct worker table writes may bypass this guard. Renewal never revives expired authority. A stale process may retain memory but cannot publish a head or originate current output. The gateway must have bounded-validity registration, not just an epoch number. M7 has no gateway implementation; readiness alone does not prove remote fencing.
M7.3 selected lease duration, renewal cadence, local safety margin and store deadlines together in ADR-0060 (20 s / 5 s / 2 s / 2 s renew / 5 s commit). The latency behind them was measured only on a local socket. The database functions and runtime grants in migration v2 implement the guard described here. Any uncertain lease stops admission, simulation and final output. A commit that already linearized can remain durable after its writer expires; query can return its receipt, but does not grant that writer authority to resume.
Batch and I/O boundary
Profile limits: one pending batch, 1..64 contiguous ticks, payload <=1MiB for the environmental envelope, <=64 durable requests/outcomes and <=64KiB total canonical request/result bytes per batch. The batch additionally carries <=64KiB of ordered input/report validation trace; charge it separately from decision bytes. The full canonical commit request is bounded by 160KiB including trace, decisions and metadata. One host backlog holds <=64 future inputs and <=64KiB encoded input bytes. Overflow backpressures/rejects before admission; no dropping acknowledged inputs. Implementations must charge payload/output buffers, one upload buffer and the process budget separately. Compression is initially off. No additional queue may silently defeat these bounds.
The host drives one tick at a time when permanent outcomes are possible. M6.7 committed-prefix results describe in-memory ticks, not database commits. The host cannot blindly execute 64 ticks and discover a death afterward: after each tick, detect terminal transitions, include the immutable outcome, seal the current batch and pause the whole region before any dependent next tick. The terminal tick's ordered phases commit together, including decay. No subsequent tick runs until durable confirmation. Normal flush occurs by 64 ticks or an earlier host deadline; there is no SQL call from the tick itself. Exceptionally a death can force a one-tick batch; this is not a per-tick persistence loop.
sequenceDiagram
participant H as Single-owner region host
participant O as Immutable payload store
participant D as Fenced database authority
H->>H: Execute bounded prefix; seal exact end state and outcomes
H->>O: Put immutable candidate; verify length and SHA-256
H->>D: Commit expected head + epoch + candidate + decisions/outbox
D-->>H: Confirmed receipt / rejected / unknown
alt unknown
H->>D: Resolve same commit ID and canonical request
Note over H: Remain paused; never assume rollback
end
Note over H: Only confirmed durable receipt permits final outcome and next tick
Interface sketches (not compiled stubs): put_immutable(bytes)->VerifiedObject,
commit_region_batch(lease, expected_head, batch)->Confirmed|Rejected|Unknown,
resolve_commit(world,region,commit_id,canonical_request)->Committed|Absent|Conflict|Unavailable,
load_head(world,region)->CommittedManifest. Adapter errors carry categories and
bounded diagnostic context, never credentials. M7.2 chooses concrete C++ types.
Candidate state is untrusted to the durable validator: check closure, versions, identities, end tick, resource invariants and outcome consistency before publishing. Reject changed terminal histories and impossible outcomes. This does not establish general anti-cheat against a fully compromised simulation process; database roles limit its authority and M7 tests verify the selected transition validators.
Acknowledgments, retries and recovery
Received input and simulated input ACKs are explicitly provisional. Only a receipt for the database transaction that references durably stored bytes can finalize death/reproduction or another permanent decision. Durable request IDs are scoped by world/region and are never reused. Retain canonical request and result, compare exact bytes on retry; changed contents conflict. A process's 64-entry retry cache cannot authorize pruning the durable record. Permanent history/decision IDs remain retained for the world lifetime in this initial contract, with archival later.
On timeout/disconnect, freeze and query the same commit ID. If committed, reconcile exact digest/predecessor/outcomes and adopt the receipt. If absent, retry the same request only while still authorized; concurrent delayed originals converge under the unique ID and head guard. Absence does not prove a request can never arrive. If fenced, abandon mutation and let the current owner recover; do not undo a death or ACK speculation. Conflicting bytes or unavailable reconciliation fail closed.
Recovery acquires fresh authority, reads the committed head, verifies object length and strong digest, checks the explicit version/profile allowlist, restores the full payload and reconciles durable outcomes/history to the commit. Missing/corrupt objects quarantine the region; never select an older head and silently undo ACKs. Unpublished uploads and uncommitted memory are not recovery inputs. No replay tail exists in this profile. Required historical binaries/generators remain available. Readiness requires successful closure checks and a still-valid lease.
No orphan collection in initial M7. Retention/GC must prove reachability across heads, commit/decision records, clones and backups before deleting anything. A clone gets a new identity/authority domain; references cannot permit source writes. Database plus referenced-object backups need a verified common consistency cut. Worker/pod loss with surviving durable stores is M7's test model. Host/storage loss requires separate M8 replication/restore evidence; old backups cannot silently resurrect acknowledged deaths. World0 stays gated.
Crash-point acceptance matrix (required later, not executed by M7.1)
| Interruption | Required durable result / recovery | Gate |
|---|---|---|
| Before/within a tick | Previous committed head; no permanent ACK | 7.6/7.7 |
| After simulation, before upload | Previous head; pending outcomes not final | 7.5/7.7 |
| During upload or failed verification | No manifest; partial/orphan object unreachable | 7.4 |
| After verified upload, before transaction | Previous head; immutable orphan retained | 7.4/7.7 |
| Lease expires / new owner before transaction | Old writer rejected; new owner loads committed head | 7.3 |
| After manifest insert before transaction commit | Entire transaction rolls back, including outcomes/outbox | 7.5 |
| Commit succeeds, response lost | Resolve same ID returns exact receipt, no second death/birth | 7.5/7.7 |
| Timeout while original request still running | Retry converges or conflicts safely; never infer rollback | 7.5 |
| After final ACK, pod killed | Latest head reconstructs ACKed state and history | 7.7 |
| Outbox delivered twice / broker down | Idempotent projection or queued delivery; no second simulation mutation | 7.5 |
| Payload missing/corrupt, unknown versions | Quarantine/not ready, no fallback genesis or older world | 7.4/7.7 |
| Paused stale worker resumes | Renew/commit/output rejected; cannot skip recovery | 7.3/7.6 |
| Shutdown with pending/unknown commit | Drain boundedly or leave unavailable; next owner reconciles | 7.6/7.7 |
| Clone/migration fails mid-publication | Source untouched; target not ready, no partial version switch | 7.9 |
M7.1 verification is a source/state inventory and contract review. Real database, object-store, process-kill and network partition tests are still required.
M7.3 executed its rows against real PostgreSQL. After lease expiry, a new owner acquires a newer epoch. The old writer's renew, release and head advance are then rejected, and the head is unchanged. A paused stale worker resuming past its lease is fenced both locally and by the database. A writer queued on the lease lock past expiry is also rejected. These are in-process pauses, not killed pods or network partitions. The 7.6 parts of those rows remain open.
M7.4a (ADR-0061) executed its rows for environmental-v1 against real PostgreSQL and a local filesystem object store:
- A partial upload leaves only an unresolvable temporary name.
- A failed upload sends no manifest.
- A verified upload whose transaction is rejected stays an unreferenced orphan, and recovery returns the previous head.
- Missing, corrupt, symlinked or resized payloads quarantine the region.
- Unsupported builds are refused by both the database and recovery.
- Recovery never creates genesis or picks an older head.
M7.4b (ADR-0062) closes the checkpoint side of the history audit. The lineage
archive codec and the environmental-lineage-v1 profile validate:
- organism-to-genome bindings and parent closure (by archive replay);
- lifecycle agreement, with deaths recorded in the causing tick;
- the living set, identity high-waters and retained dead members. Recovery restores all of these. Durable history rows (M7.5) and process-kill recovery (M7.7/M7.9) remain before the retained-ancestry exit claim.