Implement M5.5 atomic terrain excavation and debris accounting
This commit is contained in:
+3
-1
@@ -264,7 +264,9 @@ M5.3 is complete under [ADR-0044](adr/0044-abiotic-generation.md):
|
||||
[versioned abiotic generation](world/BASE_GENERATION.md) with frozen reference fixtures.
|
||||
M5.4 is complete under [ADR-0045](adr/0045-bounded-chunk-ownership.md):
|
||||
[bounded ownership](world/CHUNK_OWNERSHIP.md), compact storage, validated revision tokens
|
||||
and dirty eviction refusal. Next is 5.5: accounted transactional terrain edits.
|
||||
and dirty eviction refusal. M5.5 now implements [atomic excavation](world/TERRAIN_EDITS.md)
|
||||
with per-material volume conservation under [ADR-0046](adr/0046-transactional-excavation.md).
|
||||
Next is 5.6: canonical snapshots and edit journals.
|
||||
|
||||
**Decisions still open:** final spherical layout, region dimensions, adaptive LOD,
|
||||
field quantization beyond the initial experiment, hydrology and surface extraction.
|
||||
|
||||
+10
-2
@@ -7,7 +7,7 @@ persistent simulation server. The founding vision remains unchanged.
|
||||
## Completed implementation
|
||||
|
||||
- Vision, architectural pillars, risks, open-question register, architecture
|
||||
diagrams, biological contracts, engineering constitution and 45 ADRs.
|
||||
diagrams, biological contracts, engineering constitution and 46 ADRs.
|
||||
Proposed/deferred choices remain distinct from accepted proof decisions.
|
||||
- M0–M4 task breakdowns with dependencies, acceptance evidence and risks,
|
||||
plus M5–M14 gates in [ROADMAP.md](ROADMAP.md).
|
||||
@@ -61,7 +61,7 @@ for the bounded proof; the general production ECS remains a later measured choic
|
||||
|
||||
| Priority | Work | Completion evidence |
|
||||
|---|---|---|
|
||||
| 1 | M5.5 accounted terrain edits | Transaction identity, revisions, matter destinations, atomic changes and retry validation |
|
||||
| 1 | M5.6 snapshots and edit journals | Bounded canonical format, geometry/debris/retry-state consistency and malformed-input rejection |
|
||||
| Research | Improve aggregate fidelity and measure Linux/target-host costs | M4.6 found no qualifying width; see the aggregate budget report |
|
||||
| 2 | Revisit the ADR-0031/0032 reproduction cost model | Reproduction is affordable only at the genome schema's floor; decide whether that is intended before M4 builds cohorts on it |
|
||||
| Delivery gate | Add/update the dev workload when a deployable increment needs it; Argo is ready | Observe Job/pod completion separately from image publication; no World0 changes |
|
||||
@@ -568,3 +568,11 @@ Added fixed-slot, budgeted ChunkStore and immutable uniform/palette/dense Compac
|
||||
Revision tokens reject stale and foreign reads. Clean chunks regenerate; dirty pins
|
||||
refuse eviction. [Ownership contract](world/CHUNK_OWNERSHIP.md) documents memory/lifetime
|
||||
limits and the absence of cell edits or durability. Next: M5.5 accounted terrain edits.
|
||||
|
||||
## M5.5 transactional terrain excavation
|
||||
|
||||
Added up to 64 cuts across four loaded chunks, atomically transferring occupied
|
||||
volume into finite per-material debris. Revision/digest/full-payload retry checks,
|
||||
64-command retention and peak allocation backpressure are explicit. The
|
||||
[edit contract](world/TERRAIN_EDITS.md) records excavation-only scope and persistence
|
||||
obligations. No mass conversion or durable acknowledgement is claimed. Next: M5.6.
|
||||
|
||||
@@ -61,3 +61,8 @@ apply backpressure rather than dropping state. Larger workloads require measurem
|
||||
Test far-apart chunks, uniform/palette/dense equality, exact budget boundaries,
|
||||
slot exhaustion, invalid input, stale/foreign views and dirty eviction refusal.
|
||||
Retain existing generator goldens and simulation formats. Next is M5.5.
|
||||
|
||||
## M5.5 follow-up
|
||||
|
||||
ADR-0046 adds atomic excavation and fixed replay metadata. Its 32 KiB scratch
|
||||
reservation supersedes the original 17 KiB allowance; dirty clearing remains absent.
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# ADR-0046: Atomic local excavation and material volume accounting
|
||||
|
||||
## Status
|
||||
Accepted for M5.5 bounded excavation only. Not mass, energy, chemistry or durability.
|
||||
|
||||
## Context
|
||||
Cell fill denotes volume, not kilograms. Excavation must not erase occupied material.
|
||||
M5.4 owns immutable chunks and rejects dirty eviction; transactions can prepare new
|
||||
payloads before publishing. Distributed edits and persistence are later boundaries.
|
||||
|
||||
## Decision
|
||||
Implement excavation only, with 1..64 unique ordered cuts touching at most four loaded
|
||||
chunks in one store. Each cut names chunk, local cell, expected local revision and
|
||||
positive removed fill numerator. Require lexicographic chunk order then X-fast cell
|
||||
ordinal, no duplicates. Coordinates, fill bounds, material membership, revision and
|
||||
source supply are checked. No implicit loading, deposits, conversions or unbounded edits.
|
||||
|
||||
One named destination (debris=1) is owned by each store. Creation specifies separate
|
||||
finite rock/regolith capacities, initially empty; defaults are zero. Credit exactly
|
||||
removed fill units by material. One unit is 1/65535 cubic metre at the v1 1m pitch.
|
||||
No density/mass conversion is assumed, and debris is not a second copy of terrain.
|
||||
Invariant per material: occupied volume + debris volume unchanged by every commit.
|
||||
M6 must explicitly reconcile these inventories with its material/density/composition
|
||||
model; no biological ledger femtogram values are fabricated from volume.
|
||||
|
||||
Request carries nonzero monotonically increasing command ID, destination, cuts and
|
||||
server-verified FNV-1a request digest over explicit fields (see contract). This is not
|
||||
cryptography/authentication. Retain 64 successful requests and receipts, including
|
||||
full payload equality so hash collisions cannot authorize changed-payload retries.
|
||||
Same retained ID/exact request returns the original receipt before checking now-stale
|
||||
revisions; changed payload is rejected. IDs <= high-water mark outside retention
|
||||
are stale, never executed again. Failed requests consume no ID; no out-of-order commits.
|
||||
|
||||
Prepare all compact replacement chunks and validate destination capacity and peak
|
||||
budget while originals remain owned. Publish geometry, debris, revisions, dirty flags,
|
||||
high-water mark and replay record in one nonthrowing single-owner phase. Allocation
|
||||
failure before publish leaves state intact. Every touched chunk gets a new revision
|
||||
and becomes dirty. No durable acknowledgement or dirty clearing is introduced.
|
||||
|
||||
Charge the fixed replay directory plus a conservative 32 KiB scratch allowance
|
||||
(replacing 17 KiB), and all old plus new payloads at preparation peak. No automatic
|
||||
cache eviction on memory failure. Caller buffers and allocator overhead remain excluded.
|
||||
Retain current tokens only by re-query; receipts contain historical revisions.
|
||||
|
||||
## Alternatives Considered
|
||||
A sink destroys matter. A client-owned destination cannot be atomically credited with
|
||||
private chunk state. Reusing the biological mass ledger would invent a density model.
|
||||
Generic deposit/replacement edits enlarge the first mutation surface without evidence.
|
||||
Digest-only replay is unsafe even with a strong hash; exact payloads are bounded here.
|
||||
|
||||
## Benefits
|
||||
Real multi-chunk excavation, conserved typed volume and atomic failure/retry behavior.
|
||||
No persistence fiction or cross-domain unit conversion.
|
||||
|
||||
## Costs
|
||||
Bounded fixed replay metadata, conservative peak memory, ordered commands and only one
|
||||
destination. Same-store single owner only. Debris cannot be withdrawn in this gate.
|
||||
|
||||
## Risks
|
||||
Crashes lose this entire in-memory state; M5.6/5.7 must persist debris, revisions,
|
||||
command high-water/retention and geometry consistently. FNV is not authentication.
|
||||
Allocation exceptions still require host handling; budgets are not RSS limits.
|
||||
|
||||
## Consequences
|
||||
Test seam-crossing cuts, partial fill, capacity/source/stale failure with no partial
|
||||
publication, same/changed retries, retention expiry, unknown destination/material,
|
||||
invalid/duplicate/order/size requests and peak-memory rejection. Keep existing goldens.
|
||||
@@ -49,3 +49,5 @@ design. No unadopted candidate is a link dependency of the proof.
|
||||
- [ADR-0044: Abiotic generation](0044-abiotic-generation.md) - accepted for M5.3 synthetic integer base, version retention and material table.
|
||||
|
||||
- [ADR-0045: Bounded chunk ownership](0045-bounded-chunk-ownership.md) - accepted for M5.4 local memory, revision tokens and dirty eviction guards.
|
||||
|
||||
- [ADR-0046: Transactional excavation](0046-transactional-excavation.md) - accepted for M5.5 atomic local volume transfers and bounded exact-request replay.
|
||||
|
||||
@@ -132,3 +132,12 @@ fixed-slot, byte-budgeted local store. [The contract](../world/CHUNK_OWNERSHIP.m
|
||||
specifies construction scratch accounting, current-revision read tokens, regeneration
|
||||
and dirty eviction refusal. No database, distributed lease or durable acknowledgement
|
||||
is implied. Cell edits remain M5.5 and persistence M5.6/5.7.
|
||||
|
||||
## Implemented M5.5 edit boundary
|
||||
|
||||
[ADR-0046](../adr/0046-transactional-excavation.md) implements bounded single-store
|
||||
atomic excavation, per-material occupied-volume transfer to debris, current revisions,
|
||||
request digest plus full-payload retry identity, and bounded history with a command
|
||||
high-water mark. [The contract](../world/TERRAIN_EDITS.md) specifies failure semantics.
|
||||
No mass/density model or persistence is implied. Debris/retry state must accompany
|
||||
geometry in M5.6/5.7 recovery, and M6 must reconcile volume with chemical inventories.
|
||||
|
||||
@@ -276,3 +276,11 @@ fingerprint. Conservation holding over a million ticks demonstrates arithmetic,
|
||||
not ecological plausibility. Strategy constants, lifespans and the targeting rule
|
||||
are chosen inputs, not derived biology, and must not become gameplay by having
|
||||
been measured. Real populations still require M4 aggregation.
|
||||
|
||||
## M5.5 terrain volume boundary
|
||||
|
||||
[Transactional excavation](../world/TERRAIN_EDITS.md) conserves occupied volume by
|
||||
material between terrain fill and a finite debris inventory. VolumeUnits uses
|
||||
1/65535 m^3 at the experimental 1m pitch, not femtograms. No conversion into the
|
||||
biological ledger is allowed until M6 specifies density/composition and reconciles
|
||||
both representations. Debris is the destination of removed terrain, not new matter.
|
||||
|
||||
@@ -1502,3 +1502,33 @@ accounting excludes allocator/runtime overhead and is not an RSS cap.
|
||||
|
||||
Linux CI/UBSan and target-host performance have not been run locally. No deployment
|
||||
or World0 change occurred. Next: M5.5 accounted transactional edits.
|
||||
|
||||
## M5.5 transactional excavation - 2026-09-26
|
||||
|
||||
Added bounded atomic excavation, finite per-material debris volume, full-payload
|
||||
retry identity and a 64-command replay ring under ADR-0046. Existing biology/world
|
||||
base goldens and persisted formats were not changed. Formatter coverage now includes
|
||||
private headers under src, including the shared chunk-state definitions.
|
||||
|
||||
Executed on Windows x64 / MSVC 19.44:
|
||||
|
||||
- `tools/windows-check.ps1 -Preset release -Action check`: 44/44 passed, including
|
||||
the final late-source-failure test and nonthrowing commit static assertions.
|
||||
- `tools/windows-check.ps1 -Preset analysis -Action check`: clang-tidy 19 build and
|
||||
44/44 passed. After the final targeted additions, rebuilt analysis and reran
|
||||
`ctest --test-dir build/analysis -R world-edit-contracts --output-on-failure`: passed.
|
||||
- `tools/windows-check.ps1 -Preset asan -Action check`: 44/44 passed. After those
|
||||
additions, rebuilt ASan and reran the same targeted test in build/asan: passed.
|
||||
- `tools/windows-check.ps1 -Preset release -Action format`: passed.
|
||||
- `git diff --check`: passed.
|
||||
|
||||
Coverage includes seam-crossing excavation, four chunks/64 cuts, partial fills,
|
||||
rock/regolith volume conservation, all-or-nothing capacity/memory/source failures,
|
||||
missing/stale revisions, duplicate/changed requests, malformed bounds and retention
|
||||
expiry. Late source failure verifies an earlier prepared chunk is not published.
|
||||
No allocator failure injection, revision exhaustion injection or crash recovery is
|
||||
claimed. The source/replacement commit operations have compile-time nonthrowing guards.
|
||||
|
||||
Linux CI and UBSan were not run locally. These are typed volume transfers, not a
|
||||
validated density/mass/chemistry model. No deployment or World0 modification occurred.
|
||||
Next: M5.6 canonical snapshots/edit journals preserving geometry, debris and retry state.
|
||||
|
||||
@@ -14,7 +14,7 @@ The owning unique_ptr may move, but the store itself cannot move or copy.
|
||||
|
||||
| Operation | Success | Refusal / invariant |
|
||||
|---|---|---|
|
||||
| minimum_bytes(slots) | Store object + fixed directory + 17 KiB scratch reservation | Invalid slot count rejected before multiplication |
|
||||
| minimum_bytes(slots) | Store object + fixed directory + replay records + 32 KiB scratch reservation | Invalid slot count rejected before multiplication |
|
||||
| create | Empty store with validated descriptor | Invalid config/version/budget before directory allocation |
|
||||
| load(address) | Return current token or regenerate and publish one base chunk | Invalid address, slot or byte exhaustion, revision exhaustion; no existing state changes |
|
||||
| view(address) | Token for loaded address | Missing is explicit; no implicit load |
|
||||
@@ -29,7 +29,7 @@ an object later allocated at the same address. They cannot authorize persistence
|
||||
cross-process operations. Epoch exhaustion is checked before increment; no wraparound.
|
||||
|
||||
Dirty marking currently changes only lifecycle metadata, not geometry. There is no
|
||||
clear-dirty or cell-edit API. M5.5 must introduce accounted edit transactions; M5.6/5.7
|
||||
clear-dirty API. M5.5 now adds [accounted excavation](TERRAIN_EDITS.md); M5.6/5.7
|
||||
must establish durable revision coverage before permitting cleanup. Destruction at
|
||||
host shutdown frees memory and is not a save operation. No production host should
|
||||
use this in-memory store as its sole durable copy of changes.
|
||||
@@ -62,8 +62,8 @@ not a wire format. M5.6 must specify canonical serialization independently.
|
||||
|
||||
Fixed directory bytes include optional payload objects and lifecycle metadata for
|
||||
all configured slots, occupied or not. Accounted bytes add exact allocated payload
|
||||
array element sizes plus a permanent 17 KiB allowance for load construction scratch:
|
||||
4096 logical words and 256 palette keys. Load generates through point queries to avoid
|
||||
array element sizes plus a permanent 32 KiB conservative scratch allowance and fixed replay records
|
||||
under ADR-0046. Loads use 4096 logical words and 256 palette keys. Load generates through point queries to avoid
|
||||
an extra dense vector, computes required payload before allocation, then publishes
|
||||
without throwing. On allocator failure existing state is unchanged; allocation
|
||||
exceptions propagate to the host. Clean eviction subtracts payload only.
|
||||
@@ -84,6 +84,6 @@ tokens, conservative dirty pinning and rejection without changing resident state
|
||||
Existing generator goldens remain active. Counter exhaustion is guarded in code but
|
||||
not forced via a public test-only state injection API.
|
||||
|
||||
M5.5 adds transaction IDs, payload identity, expected revisions, material destinations
|
||||
and atomic edits. No transactional edit, snapshot, crash recovery, threading, automatic
|
||||
LRU, renderer or Kubernetes integration is provided by M5.4.
|
||||
M5.5 now adds transaction IDs, full request identity, expected revisions, a finite debris
|
||||
destination and atomic excavation; see TERRAIN_EDITS.md. Snapshot/crash recovery,
|
||||
threading, automatic LRU, renderer and Kubernetes integration remain later work.
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# M5.5 transactional excavation
|
||||
|
||||
[ADR-0046](../adr/0046-transactional-excavation.md) scopes the first terrain edit to
|
||||
excavation. Public contracts are in `evo/world/chunks.hpp`; implementation is in
|
||||
`src/world/edits.cpp`. All calls require one sequential owner and stable request memory
|
||||
through return. Inputs are intentions, never trusted geometry or authoritative balances.
|
||||
|
||||
## Request and atomic scope
|
||||
|
||||
An ExcavationRequest names a nonzero command ID, destination 1 (this store's debris),
|
||||
a server-verified digest and 1..64 CellCuts. Cuts touch at most four loaded chunks in
|
||||
one store. Each names chunk/local address, expected revision and removed fill in
|
||||
[1,65535]. Sort lexicographically by chunk x/y/z, then X-fast local ordinal; duplicate
|
||||
cells and inconsistent revisions within one chunk are rejected. No implicit load or
|
||||
cross-store/world/region transaction occurs. No deposit, arbitrary replacement,
|
||||
material transmutation, biological cost or external inventory is implemented.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Host
|
||||
participant Store
|
||||
participant Prepared
|
||||
Host->>Store: command + digest + ordered cuts + expected revisions
|
||||
Store->>Store: Validate shape, digest, replay identity, ownership and revisions
|
||||
Store->>Prepared: Copy logical cells and stage compact replacements
|
||||
Store->>Store: Check source volume, debris capacity, old+new peak bytes
|
||||
alt all checks and allocations succeed
|
||||
Store->>Store: Publish all chunks + debris + revisions + replay receipt
|
||||
Store-->>Host: receipt
|
||||
else any refusal or allocation failure
|
||||
Store-->>Host: error; original state remains intact
|
||||
end
|
||||
```
|
||||
|
||||
Published chunks get distinct increasing local revisions and dirty flags. Previous
|
||||
views become stale; query `view` for current read tokens. The returned receipt lists
|
||||
historical committed revisions and removed volume by material, not current live views.
|
||||
Dirty chunks remain unevictable. Mutation of already-dirty chunks is allowed with
|
||||
current revisions. Geometry and debris commit together without throwing; all fallible
|
||||
allocations precede publication. Host must handle allocation exceptions. This is
|
||||
in-process atomicity, not crash consistency, client authentication or a durable ACK.
|
||||
|
||||
## Volume conservation and explicit destination
|
||||
|
||||
`VolumeUnits{n}` means n/65535 cubic metres at the v1 1m pitch. A CellSample's fill
|
||||
is already in these units. DebrisCapacity specifies independent uint64 capacities for
|
||||
rock and regolith at store creation; defaults are zero and balances start empty.
|
||||
Unknown destination IDs are rejected. No public capacity change, withdrawal or balance
|
||||
setter exists. Material keys alone never grant debris supply.
|
||||
|
||||
For each material m and committed command:
|
||||
|
||||
`sum(terrain fill[m])_after + debris[m]_after = sum(terrain fill[m])_before + debris[m]_before`.
|
||||
|
||||
Partial excavation reduces fill with the same material ID. Removing the full fill
|
||||
produces structurally valid void. Rock cannot credit regolith or vice versa. Compare
|
||||
credits to capacity minus balance before adding, avoiding unsigned overflow. Bounded
|
||||
commands remove at most 64*65535 units. Geometry of the removed sub-cell portion is
|
||||
unknown; no collision shape or smooth surface is claimed.
|
||||
|
||||
These inventories are volume, not kg/femtograms. M6 must define density/composition
|
||||
and reconcile them without double-counting before coupling to biology. No value in
|
||||
the existing biological ledger is reinterpreted or modified by this gate.
|
||||
|
||||
## Retry identity and compatibility
|
||||
|
||||
The [request digest schema](../../schemas/world-edits.md) defines the explicit byte
|
||||
order. FNV-1a is an accidental-corruption/identity aid, never security. Retained command
|
||||
IDs compare all original fields as well as digest, so collisions cannot authorize a
|
||||
changed request. For valid request shape and digest:
|
||||
|
||||
- Same retained ID and exact payload returns the original receipt without reapplying,
|
||||
even after newer edits made its expected revisions stale.
|
||||
- Same retained ID with different payload returns command_conflict.
|
||||
- An ID <= the high-water mark with no retained record returns stale_command.
|
||||
- A fresh ID must exceed the high-water mark; gaps are permitted. Failures consume no
|
||||
ID and mutate no cells, debris, revision, dirty flag or history record.
|
||||
|
||||
Only the last 64 successful commands are retained in a fixed ring. This is bounded
|
||||
local retry history, not event retention. M5.6/5.7 must persist geometry, debris,
|
||||
capacities, revisions, command high-water and replay records consistently. A restart
|
||||
that retains only geometry would duplicate or lose material and is not supported.
|
||||
|
||||
## Memory and tests
|
||||
|
||||
minimum_bytes now includes the fixed 64-record replay array and 32 KiB conservative
|
||||
scratch reservation, superseding M5.4's 17 KiB allowance. New replacement payloads
|
||||
must fit beside all old payloads; old storage is released only during commit. Unused
|
||||
replay fields remain bounded metadata. No cache eviction occurs to accommodate edits.
|
||||
Allocator/runtime overhead and caller buffers remain outside accounted bytes.
|
||||
|
||||
CTest world-edit-contracts covers a seam crossing, all four chunks/64 cuts, partial
|
||||
fill, rock/regolith conservation, destination and second-replacement memory failures,
|
||||
missing/stale/source failures, dirty eviction refusal, malformed request bounds,
|
||||
same/changed retries and expiry after 65 commands. Existing generator and simulation
|
||||
goldens remain active. Allocator fault injection and crash recovery remain future work.
|
||||
Reference in New Issue
Block a user