ADR-0063: manifest v2 (360 bytes) binds a per-tick validation trace object and a canonical decision block by SHA-256. Publication replays the trace from the committed predecessor's verified payload and requires every report and the candidate bytes, derives deaths deterministically, uploads payload and trace with verified read-back, then sends manifest and decisions to one fenced statement. RegionBatch seals at a terminal transition or 64 ticks and refuses ticks until durable confirmation. Schema v5 records the checkpoint, its decisions and outbox rows atomically; a unique index keeps one terminal outcome per actor (EVL08), triggers make commits/decisions immutable (EVL09), an exact retry of a committed ID returns the original receipt, changed bytes conflict, and read-only resolve_region_commit answers committed/absent/ conflict. Outbox pending/acknowledge give idempotent at-least-once delivery. Recovery reconciles durable deaths with restored lifecycle (and archive deaths for the lineage profile). Validated: Windows CTest 58/58; Linux GCC Debug/Release, clang-tidy and ASan/UBSan CTest 58/58 each; PostgreSQL suites Release, analysis, ASan (lost response, committed-after-timeout, terminated orphan retry, changed bytes, repeated/partial/reordered decisions, gap, outbox redelivery, history quarantine); five mutations each caught. No Docker/Gitea CI, process kill, broker or deployment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
204 lines
13 KiB
Markdown
204 lines
13 KiB
Markdown
# M7 region durability contract
|
|
|
|
Design accepted by ADR-0058; no storage/host implementation yet. Locked invariants:
|
|
one committed authority, no lost acknowledged permanent outcome under the declared
|
|
failure model, no fallback empty world, and no database dependency in headless code.
|
|
|
|
## State closure audit
|
|
|
|
| State | Existing evidence | M7 treatment / missing work |
|
|
|---|---|---|
|
|
| M6 environment, two accounts/capacities, terrain and debris | EnvironmentalCommunity envelope v2 and nested codecs | Required exact payload for environmental profile |
|
|
| Config, source/growth/efficiency residuals, lifecycle, tick, last funded interval | Same envelope | Restore all; never infer from tick or defaults |
|
|
| Process retries/audit anchor, resource revisions | Same envelope, 64 records | Preserve; not a durable command deduplication service |
|
|
| Interval scheduler | Stateless M6.7 adapter | No clock/debt to serialize; host queues are separate |
|
|
| Genome | Individual Genome v1 codec | No M6 account binding; unsupported in initial profile |
|
|
| LineageArchive and dead organisms | Separate append-only API/tests, no complete codec | Add bounded export/restore and binding before ancestry recovery is claimed; retain extinct genomes |
|
|
| Population/cohorts | Separate M4 state/codecs | No integration with M6; profile must explicitly add complete ownership before admission |
|
|
| Particle entities/positions/RNG | Independent proof snapshot | Not part of M6; never concatenate and imply unified ownership |
|
|
| Environmental randomness | None in rules 1/2 | Test-driver RNG is not simulation state; future runtime RNG needs versioned streams |
|
|
| Pending host commands | Not implemented | Bounded host queue; uncommitted receipt is provisional; durable decisions stored separately |
|
|
| Durable decisions/history/outbox | Not implemented | Required transactional records with commit references and immutable request bytes |
|
|
| Player accounts, logout, network sessions, cross-region transfers | Not implemented | Unsupported capabilities; initial profile cannot host players |
|
|
| Lease/routing readiness | Not simulation state | Live database authority, never resumed from a saved lease |
|
|
|
|
M7.4 must enumerate any newly attached state and extend its schema and tests before
|
|
admitting it. For a history-capable profile, validate all organism-to-genome bindings,
|
|
parent closure, lifecycle agreement, identity high-waters and archived dead members.
|
|
Both checkpoint and durable history must agree. Empty/unrecognized sections cannot
|
|
stand in for missing history. M7.9's retained-ancestry criterion is not waived by
|
|
the narrower environmental profile. M7.1 completes the audit, not those codecs.
|
|
|
|
## Identity, authority and watermarks
|
|
|
|
World/region/frame are nonzero durable IDs. No ID is a Pod name. The host records
|
|
its unique holder ID and monotonically increasing nonzero lease epoch. Exhaustion
|
|
of IDs, epochs, commit sequences or ticks fails closed; never wrap/reuse them.
|
|
Simulation tick is local to the fixed versioned region rules. `simulated_tick`
|
|
may lead `committed_tick` only within the current bounded pending batch.
|
|
`commit_sequence` counts publications, not ticks or field commands. Genesis has
|
|
sequence 0, tick 0, no predecessor and an explicit authorized opening payload.
|
|
Only provisioning can create genesis; recovery cannot do so.
|
|
|
|
The database arbitrates acquire/renew/commit/release under the same region lease
|
|
row lock, using database expiry and matching holder/epoch. Commit additionally
|
|
compares expected predecessor sequence, digest and tick. No direct worker table
|
|
writes may bypass this guard. Renewal never revives expired authority. A stale
|
|
process may retain memory but cannot publish a head or originate current output.
|
|
The gateway must have bounded-validity registration, not just an epoch number.
|
|
M7 has no gateway implementation; readiness alone does not prove remote fencing.
|
|
|
|
M7.3 selected lease duration, renewal cadence, local safety margin and store
|
|
deadlines together in ADR-0060 (20 s / 5 s / 2 s / 2 s renew / 5 s commit). The
|
|
latency behind them was measured only on a local socket. The database functions
|
|
and runtime grants in migration v2 implement the guard described here.
|
|
Any uncertain lease stops admission, simulation and final output. A commit that
|
|
already linearized can remain durable after its writer expires; query can return
|
|
its receipt, but does not grant that writer authority to resume.
|
|
|
|
## Batch and I/O boundary
|
|
|
|
Profile limits: one pending batch, 1..64 contiguous ticks, payload <=1MiB for the
|
|
environmental envelope, <=64 durable requests/outcomes and <=64KiB total canonical
|
|
request/result bytes per batch.
|
|
The batch additionally carries <=64KiB of ordered input/report validation trace;
|
|
charge it separately from decision bytes. The full canonical commit request is
|
|
bounded by 160KiB including trace, decisions and metadata. One host backlog holds <=64 future inputs and
|
|
<=64KiB encoded input bytes. Overflow backpressures/rejects before admission; no
|
|
dropping acknowledged inputs. Implementations must charge payload/output buffers,
|
|
one upload buffer and the process budget separately. Compression is initially off.
|
|
No additional queue may silently defeat these bounds.
|
|
|
|
The host drives **one tick at a time** when permanent outcomes are possible. M6.7
|
|
committed-prefix results describe in-memory ticks, not database commits. The host
|
|
cannot blindly execute 64 ticks and discover a death afterward: after each tick,
|
|
detect terminal transitions, include the immutable outcome, seal the current batch
|
|
and pause the whole region before any dependent next tick. The terminal tick's
|
|
ordered phases commit together, including decay. No subsequent tick runs until
|
|
durable confirmation. Normal flush occurs by 64 ticks or an earlier host deadline;
|
|
there is no SQL call from the tick itself. Exceptionally a death can force a
|
|
one-tick batch; this is not a per-tick persistence loop.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant H as Single-owner region host
|
|
participant O as Immutable payload store
|
|
participant D as Fenced database authority
|
|
H->>H: Execute bounded prefix; seal exact end state and outcomes
|
|
H->>O: Put immutable candidate; verify length and SHA-256
|
|
H->>D: Commit expected head + epoch + candidate + decisions/outbox
|
|
D-->>H: Confirmed receipt / rejected / unknown
|
|
alt unknown
|
|
H->>D: Resolve same commit ID and canonical request
|
|
Note over H: Remain paused; never assume rollback
|
|
end
|
|
Note over H: Only confirmed durable receipt permits final outcome and next tick
|
|
```
|
|
|
|
Interface sketches (not compiled stubs): `put_immutable(bytes)->VerifiedObject`,
|
|
`commit_region_batch(lease, expected_head, batch)->Confirmed|Rejected|Unknown`,
|
|
`resolve_commit(world,region,commit_id,canonical_request)->Committed|Absent|Conflict|Unavailable`,
|
|
`load_head(world,region)->CommittedManifest`. Adapter errors carry categories and
|
|
bounded diagnostic context, never credentials. M7.2 chooses concrete C++ types.
|
|
|
|
Candidate state is untrusted to the durable validator: check closure, versions,
|
|
identities, end tick, resource invariants and outcome consistency before publishing.
|
|
Reject changed terminal histories and impossible outcomes. This does not establish
|
|
general anti-cheat against a fully compromised simulation process; database roles
|
|
limit its authority and M7 tests verify the selected transition validators.
|
|
|
|
## Acknowledgments, retries and recovery
|
|
|
|
Received input and simulated input ACKs are explicitly provisional. Only a receipt
|
|
for the database transaction that references durably stored bytes can finalize
|
|
death/reproduction or another permanent decision. Durable request IDs are scoped
|
|
by world/region and are never reused. Retain canonical request and result, compare
|
|
exact bytes on retry; changed contents conflict. A process's 64-entry retry cache
|
|
cannot authorize pruning the durable record. Permanent history/decision IDs remain
|
|
retained for the world lifetime in this initial contract, with archival later.
|
|
|
|
On timeout/disconnect, freeze and query the same commit ID. If committed, reconcile
|
|
exact digest/predecessor/outcomes and adopt the receipt. If absent, retry the same
|
|
request only while still authorized; concurrent delayed originals converge under
|
|
the unique ID and head guard. Absence does not prove a request can never arrive.
|
|
If fenced, abandon mutation and let the current owner recover; do not undo a death
|
|
or ACK speculation. Conflicting bytes or unavailable reconciliation fail closed.
|
|
|
|
Recovery acquires fresh authority, reads the committed head, verifies object length
|
|
and strong digest, checks the explicit version/profile allowlist, restores the full
|
|
payload and reconciles durable outcomes/history to the commit. Missing/corrupt
|
|
objects quarantine the region; never select an older head and silently undo ACKs.
|
|
Unpublished uploads and uncommitted memory are not recovery inputs. No replay tail
|
|
exists in this profile. Required historical binaries/generators remain available.
|
|
Readiness requires successful closure checks and a still-valid lease.
|
|
|
|
No orphan collection in initial M7. Retention/GC must prove reachability across
|
|
heads, commit/decision records, clones and backups before deleting anything. A
|
|
clone gets a new identity/authority domain; references cannot permit source writes.
|
|
Database plus referenced-object backups need a verified common consistency cut.
|
|
Worker/pod loss with surviving durable stores is M7's test model. Host/storage loss
|
|
requires separate M8 replication/restore evidence; old backups cannot silently
|
|
resurrect acknowledged deaths. World0 stays gated.
|
|
|
|
## Crash-point acceptance matrix (required later, not executed by M7.1)
|
|
|
|
| Interruption | Required durable result / recovery | Gate |
|
|
|---|---|---|
|
|
| Before/within a tick | Previous committed head; no permanent ACK | 7.6/7.7 |
|
|
| After simulation, before upload | Previous head; pending outcomes not final | 7.5/7.7 |
|
|
| During upload or failed verification | No manifest; partial/orphan object unreachable | 7.4 |
|
|
| After verified upload, before transaction | Previous head; immutable orphan retained | 7.4/7.7 |
|
|
| Lease expires / new owner before transaction | Old writer rejected; new owner loads committed head | 7.3 |
|
|
| After manifest insert before transaction commit | Entire transaction rolls back, including outcomes/outbox | 7.5 |
|
|
| Commit succeeds, response lost | Resolve same ID returns exact receipt, no second death/birth | 7.5/7.7 |
|
|
| Timeout while original request still running | Retry converges or conflicts safely; never infer rollback | 7.5 |
|
|
| After final ACK, pod killed | Latest head reconstructs ACKed state and history | 7.7 |
|
|
| Outbox delivered twice / broker down | Idempotent projection or queued delivery; no second simulation mutation | 7.5 |
|
|
| Payload missing/corrupt, unknown versions | Quarantine/not ready, no fallback genesis or older world | 7.4/7.7 |
|
|
| Paused stale worker resumes | Renew/commit/output rejected; cannot skip recovery | 7.3/7.6 |
|
|
| Shutdown with pending/unknown commit | Drain boundedly or leave unavailable; next owner reconciles | 7.6/7.7 |
|
|
| Clone/migration fails mid-publication | Source untouched; target not ready, no partial version switch | 7.9 |
|
|
|
|
M7.1 verification is a source/state inventory and contract review. Real database,
|
|
object-store, process-kill and network partition tests are still required.
|
|
|
|
M7.3 executed its rows against real PostgreSQL. After lease expiry, a new owner
|
|
acquires a newer epoch. The old writer's renew, release and head advance are then
|
|
rejected, and the head is unchanged. A paused stale worker resuming past its lease
|
|
is fenced both locally and by the database. A writer queued on the lease lock past
|
|
expiry is also rejected. These are in-process pauses, not killed pods or network
|
|
partitions. The 7.6 parts of those rows remain open.
|
|
|
|
M7.4a (ADR-0061) executed its rows for environmental-v1 against real PostgreSQL and
|
|
a local filesystem object store:
|
|
- A partial upload leaves only an unresolvable temporary name.
|
|
- A failed upload sends no manifest.
|
|
- A verified upload whose transaction is rejected stays an unreferenced orphan,
|
|
and recovery returns the previous head.
|
|
- Missing, corrupt, symlinked or resized payloads quarantine the region.
|
|
- Unsupported builds are refused by both the database and recovery.
|
|
- Recovery never creates genesis or picks an older head.
|
|
|
|
M7.4b (ADR-0062) closes the checkpoint side of the history audit. The lineage
|
|
archive codec and the `environmental-lineage-v1` profile validate:
|
|
- organism-to-genome bindings and parent closure (by archive replay);
|
|
- lifecycle agreement, with deaths recorded in the causing tick;
|
|
- the living set, identity high-waters and retained dead members.
|
|
Recovery restores all of these. Durable history rows (M7.5) and process-kill
|
|
recovery (M7.7/M7.9) remain before the retained-ancestry exit claim.
|
|
|
|
M7.5 (ADR-0063) executed its rows against real PostgreSQL:
|
|
- A rejected publication (repeated death, partial or reordered decision block,
|
|
gap) leaves the head, commits and decisions unchanged. Publication is a single
|
|
statement; an injected failure between insert and commit is not yet tested.
|
|
- A lost response resolves to the original receipt, and an exact retry adds no
|
|
second death.
|
|
- A client timeout while queued resolved as committed after the lock cleared,
|
|
and was adopted.
|
|
- A terminated orphan resolved as absent and was retried with the same bytes.
|
|
- Changed bytes conflict.
|
|
- Outbox redelivery is idempotent and never touches the simulation.
|
|
Recovery now reconciles durable deaths with the restored lifecycle, so the
|
|
checkpoint and durable history must agree. Kill tests at each boundary remain
|
|
M7.7.
|