Complete M6.2 optional finite environmental fields

This commit is contained in:
dingo
2026-09-27 00:21:05 -04:00
parent 750afcd4d4
commit ab130b0fbb
15 changed files with 984 additions and 3 deletions
+5
View File
@@ -310,6 +310,11 @@ under ADR-0050. Pure checked habitat/ratio/energy-integration helpers and dimens
tests are implemented. No environmental stock, field codec, source scheduler or
terrain-to-biological-mass converter is implemented by this gate. Next: M6.2.
M6.2 implements [optional finite field storage](world/ENVIRONMENTAL_FIELDS.md),
material capacity defaults, bounded sparse pages, same-field local relocation and
canonical field snapshot v1 under ADR-0051. Rate-driven sources and combined
terrain/biology persistence are not provided by this independent codec. Next: M6.3.
## M7 - Persistent region server
**Goal:** a disposable authoritative headless process can recover a bounded region
+14 -2
View File
@@ -7,7 +7,7 @@ persistent simulation server. The founding vision remains unchanged.
## Completed implementation
- Vision, architectural pillars, risks, open-question register, architecture
diagrams, biological contracts, engineering constitution and 50 ADRs.
diagrams, biological contracts, engineering constitution and 51 ADRs.
Proposed/deferred choices remain distinct from accepted proof decisions.
- M0–M4 task breakdowns with dependencies, acceptance evidence and risks,
plus M5–M14 gates in [ROADMAP.md](ROADMAP.md).
@@ -61,7 +61,7 @@ for the bounded proof; the general production ECS remains a later measured choic
| Priority | Work | Completion evidence |
|---|---|---|
| 1 | M6.2 optional environmental fields | Finite versioned inventories, sparse field pages and round-trip persistence without regenerated supply |
| 1 | M6.3 read-only environment views | Footprint queries with explicit units, availability, tick and revision validity |
| Research | Improve aggregate fidelity and measure Linux/target-host costs | M4.6 found no qualifying width; see the aggregate budget report |
| 2 | Revisit the ADR-0031/0032 reproduction cost model | Reproduction is affordable only at the genome schema's floor; decide whether that is intended before M4 builds cohorts on it |
| Delivery gate | Add/update the dev workload when a deployable increment needs it; Argo is ready | Observe Job/pod completion separately from image publication; no World0 changes |
@@ -618,3 +618,15 @@ accounting and refusal rollback are tested. Windows Release, clang-tidy and ASan
suites pass; see VALIDATION.md for exact checks. Existing formats/goldens are unchanged.
No fields or biological coupling are implemented yet. Next: M6.2 optional finite
environmental fields with versioned persistence and explicit opening inventories.
## M6.2 optional finite environmental fields (2026-09-27)
Added a bounded FieldStore with fixed habitat mappings, material water-capacity
defaults, optional scalar channels and uniform baselines with sparse 64-site pages.
Opening inventories are explicit; missing differs from zero. Same-channel local
transfers preserve finite supply and reject stale commands/overspend without change.
Field snapshot v1 preserves mapping, quantities, defaults, presence and watermark;
restore does not regenerate supplies. [Contract and memory evidence](world/ENVIRONMENTAL_FIELDS.md)
and the byte schema document trust, budget and compatibility limits. M5 terrain
formats remain unchanged; combined terrain/biology durability and rate sources are
not implemented. Next: M6.3 read-only derived environment views.
@@ -0,0 +1,73 @@
# ADR-0051: Optional finite environmental field pages
## Status
Accepted for M6.2 bounded in-memory storage and binary-buffer persistence.
## Context
ADR-0050 defines units and one-voxel habitats. Terrain contains no extractable
nutrients. M6 needs finite stocks without allocating chemistry on every planet voxel.
The M5 codec cannot silently gain environmental state or promise atomic recovery of
independently saved geometry and biology.
## Decision
Add a separate `FieldStore` to evo::world. Creation fixes 1..4096 lexicographically
ordered habitat sites, one per voxel, each with an explicit local box and substrate
material ID. A nonzero opening ID records host provenance, not authentication.
World/frame, material defaults, mappings, field presence, quantities and command
watermark are serialized together. No site insertion, removal or regeneration API.
Three material water-capacity defaults (void/rock/regolith) are explicit nonnegative
fixture parameters. They supply no inventory or density. A site's substrate is a
field property, not a live terrain view; M6.4 must reconcile edits when binding stores.
Optional water, dissolved N/P, organic C/N/P/other/energy, incident energy and heat
channels own finite opening quantities. Missing differs from present zero. A field
opening supplies one uniform value or exactly one value per site; every repeated
uniform quantity is counted in global per-element/energy totals, checked against u64.
Compress each present channel with its first site's value as baseline, and allocate
64-value pages only where another site differs. Pages use stable site order, not
planet allocation. The bounded scalar layout is a scoped choice, not a claim of
optimal compression. Measure uniform and varying memory in tests.
Implement a same-channel, same-store transfer only: positive amount, distinct valid
sites, fresh nonzero action ID, sufficient balance, capacity and allocation budget.
Stage a replacement channel before atomic publication; success advances the persisted
watermark. Stale action IDs reject, including exact retries (ADR-0024 semantics).
There is no arbitrary setter, deletion, cross-compartment conversion or organism
credit. M6.4 still owns coordinated transactions and idempotent biological outcomes.
New field snapshot v1 uses canonical explicit little-endian logical values and an
FNV corruption checksum. Restore checks versions/identity/length/checksum and all
creation invariants into a detached store. It accepts no M5 snapshot as field state.
Host byte budgets exclude input/output buffers and allocator overhead; transient
channel replacement and bounded decode workspace are explicitly charged/documented.
Rate-driven sources are not supported: incident energy is only finite stock. Before
adding rates, persist rate identity, denominator, residual and interval watermark in
a new compatible format or explicit migration; never synthesize these on restore.
## Alternatives Considered
Dense chemistry per voxel violates sparse allocation goals. Derived material supply
would replenish after unloading. Piggybacking fields into M5 without a combined
transaction gives false durability. General adaptive grids and cross-store transfers
are unnecessary for the first storage contract. First-value baselines are simpler
than dictionaries, but an isolated change at site zero can allocate many pages.
## Benefits
Finite quantities, deterministic round trips, explicit absence and bounded sparse
allocation with no database or new dependency. Geometry formats remain unchanged.
## Costs
Logical wire arrays favor validation over compression. Fixed site membership caps
the proof; a changing baseline can increase replacement costs. Read access is scalar
by site/channel, not M6.3's tick/revision-tagged environment query.
## Risks
Host provenance and substrate declarations are trusted inputs. Restoring the same
snapshot twice does not fence process authority. Checksums do not authenticate data.
The snapshot is not automatically published by LocalWorldFiles or part of M5 recovery.
## Consequences
Document the byte schema and ownership boundary. Test absence/zero, uniform/varying
memory, exhaustion, rollback, global overflow, budgets, canonical continuation,
old/unknown versions and recomputed-checksum malformed inputs. Integrated persistent
region state remains M6.8/M7; no deployment or existing format/golden changes.
+1
View File
@@ -57,3 +57,4 @@ design. No unadopted candidate is a link dependency of the proof.
- [ADR-0048: Local publication](0048-local-publication.md) - accepted for M5.7 single-writer snapshot/journal publication and recovery.
- [ADR-0049: Bounded world proof](0049-bounded-world-proof.md) - accepted for M5.8 restart regression and cost evidence.
- [ADR-0050: Environmental units and coupling](0050-environmental-units-and-coupling.md) - accepted for M6's bounded habitat/accounting contract; no field storage or terrain conversion yet.
- [ADR-0051: Optional environmental fields](0051-optional-environmental-fields.md) - accepted for M6.2 finite sparse pages, local transfers and field snapshot v1.
+10
View File
@@ -163,3 +163,13 @@ derived concentrations/moisture, dimensional examples, closed/open accounting,
rate residual ownership and rejection of unsupported terrain mass conversions.
The implemented helpers are pure computations. Inventory pages, reservations,
ownership enforcement, transfer commits and persistence follow in M6.2-6.4.
## M6.2 optional field storage
[ADR-0051](../adr/0051-optional-environmental-fields.md) implements a separate
bounded [FieldStore](../world/ENVIRONMENTAL_FIELDS.md). It owns fixed habitat sites,
explicit finite inventories, property-only material defaults and optional uniform/
sparse scalar pages. Its versioned codec retains all field state and local transfer
watermark, without relying on procedural resource regeneration. M5 LocalWorldFiles
still publishes only terrain state: integrated terrain/fields/biology recovery must
use a later version bundle. No cross-store biological transfer is authorized here.
+6
View File
@@ -297,3 +297,9 @@ obligations of M6.2-6.7. Water is conserved `other` in a distinct compartment;
this does not authorize converting it into edible material or modeling reactions.
Bulk terrain-to-mass conversion remains unsupported pending density/composition
and an atomic representation transfer. No existing ledger or biological rules change.
M6.2 [optional fields](../world/ENVIRONMENTAL_FIELDS.md) now hold explicit water,
dissolved N/P, organic matter/energy, incident-energy and heat inventories. Material
defaults provide capacity, never supply. Local transfers can only relocate the same
channel; water cannot become organic other. Field snapshots preserve depletion.
No rate source, biochemical conversion or transfer to organisms exists yet.
+38
View File
@@ -1683,3 +1683,41 @@ Linux/UBSan and target-cluster checks were not executed locally; existing CI inc
the new CTest. There is no new concurrency requiring TSan work. Helpers do not enforce
owner identity, interval deduplication or residual persistence; those remain explicit
M6.2-6.7 obligations. No deployment or World0 operation occurred. Next: M6.2.
## M6.2 optional environmental fields - 2026-09-27
Added ADR-0051, FieldStore, field snapshot v1 and its byte schema. Optional water,
N/P, organic matter/energy, incident-energy and heat channels have explicit finite
openings. Material defaults specify water capacity only. Uniform baselines with
optional 64-site pages preserve absence versus zero. Same-channel local transfers
stage replacements before commit and persist a strictly increasing command watermark.
There is no resource setter, site regeneration, cross-compartment or organism credit.
Executed on Windows x64 / MSVC 19.44:
- `tools/windows-check.ps1 -Preset release -Action check`: 50/50 passed on final code.
- `tools/windows-check.ps1 -Preset analysis -Action check`: clang-tidy 19 and
50/50 passed on final code.
- `tools/windows-check.ps1 -Preset asan -Action check`: 50/50 passed on final code.
- `tools/windows-check.ps1 -Preset release -Action format`: passed.
- `git diff --check`: passed.
Initial build diagnostics corrected explicit aggregate initialization and bit widths.
The hostile fixed-underlying-type enum test has a narrowly scoped analyzer suppression
with rationale; actual channel validation remains exercised.
Tests cover uniform/sparse/varying memory, page seams and release, exhausted sources,
source/capacity/budget/stale-command rollback, absent/present-zero restore, cross-field
conserved-channel overflow, independent dense transfer continuation and all ten fields
at 4096 sites. Maximum logical snapshot is 630880 bytes. A separately constructed
170-byte absent-field fixture checks canonical schema bytes. Every truncation of a
260-byte fixture, invalid/old versions, identity, malformed/resealed mappings and
quantities, oversized snapshots and 256 repaired-checksum mutations are exercised.
Memory observations are in ENVIRONMENTAL_FIELDS.md; accounted bytes are not RSS.
Existing terrain/simulation formats and goldens are unchanged. Field snapshot v1
is a buffer codec, not LocalWorldFiles publication or combined terrain/biology
durability. Rate sources and their interval/residual persistence are not enabled.
Linux/UBSan, target-host performance, allocator-failure injection and cluster recovery
were not executed locally. Existing Linux CTest CI includes the new test. No deployment
or World0 operation occurred. Next: M6.3 read-only environment views.
+88
View File
@@ -0,0 +1,88 @@
# M6.2 optional environmental fields
[ADR-0051](../adr/0051-optional-environmental-fields.md) scopes `FieldStore`.
Its public API is `include/evo/world/fields.hpp`; it has no I/O, threads or biology.
## Ownership and creation
The store owns 1..4096 fixed sites in strict voxel-address order. Site index is a
stable local ID. Sites have valid one-voxel HabitatBoxes and a substrate material
from M5's table v1. Distinct voxels ensure boxes cannot overlap within this store.
The host must prevent ownership overlap with other stores and bind world/frame.
Creating another store is an independent fixture branch, not additional live supply.
Opening ID is nonzero, immutable host provenance; it is not verified externally.
Water capacity defaults for materials 0/1/2 are supplied explicitly and stored.
Capacity is not water, pore geometry, density or an extraction grant. Water must
not exceed its site's capacity, including transfers and restore. Zero capacity
allows only zero stored water; moisture remains undefined there per M6.1.
Substrate declarations are immutable field metadata. M6.4 must establish a coupled
geometry revision before terrain edits can change habitat capacity or substrate.
Optional channels: water; dissolved N, P; organic C, N, P, other, chemical energy;
incident energy; heat. Scalar matter uses fg and energy uses pJ. Channel identity
preserves compartments. Water contributes to `other`, dissolved/organic components
to their named elements, and all energy compartments to energy. The sum of each
conserved channel must fit u64 at creation/restore. Organic mass and total biomass
are derived; there is no second biomass inventory. Missing channels return `missing`,
not zero. A present all-zero field stays present through encoding and restore.
Opening spans contain either one value repeated at all sites or exactly site-count
values. Channels must be supplied in increasing enum order without duplicates.
No default stock is generated from a material, absent page, restore or read.
No sites/fields can be added after creation in v1.
## Storage, operations and bounds
Each present channel uses site zero's scalar as baseline. A 64-site page allocates
64 u64 values only if any valid value differs; the last page's padding is zero.
No pages are allocated for a uniform field. Memory accounting is
`sizeof(FieldStore) + site_count*sizeof(FieldSite) + allocated_pages*512`.
An operation reserves an additional 32768-byte scalar scratch allowance and charges
old plus new channel pages during replacement; this is accounted memory, not RSS.
Caller input/output buffers, allocator headers and small stack metadata are excluded.
Restore additionally uses at most 327680 bytes of dense field staging, a second
`site_count*sizeof(FieldSite)` mapping array and at most ten opening-span records,
outside the resident budget. All counts are bounded before those allocations.
`read(site, channel)` returns the owned scalar in that channel's units; it grants
no consumption permission. M6.3 will add derived footprint/tick/revision views.
`transfer(action, channel, from, to, amount)` moves a positive quantity between
distinct sites within one present field. It checks source, overflow, water capacity,
fresh nonzero command and replacement budget before publishing. Rejection preserves
all quantities and watermark. Same/older commands reject, never execute again.
This storage primitive cannot feed an organism, convert compartments or create heat.
It transfers no terrain volume. M6.4 supplies coordinated biological transactions.
Empty input, bad material, invalid box, duplicate voxel/channel, mismatched span,
sum overflow and insufficient budgets fail explicitly. Allocation exceptions
propagate to the host; candidates are detached until success. Stores are noncopyable
and nonmovable; scalar reads return values and retain no lifetime-sensitive views.
## Persistence and compatibility
[Snapshot schema](../../schemas/world-fields-v1.md) encodes logical quantities
independently of pages. It includes world/frame, nonzero opening identity, material
defaults, mapping, channel presence and transfer watermark. Canonical decode/re-encode
must match exactly. Runtime byte budgets and pointers are not persisted.
No old/unknown version is migrated implicitly; checksums detect accidental corruption
only. Restoring a field snapshot never calls the terrain generator.
Snapshot buffers are not durable publication. M5 LocalWorldFiles does not save these
fields and must not be advertised as recovering a coupled environment. M6.8/M7 must
publish a version bundle and ownership boundary across terrain, fields and biology.
There are no configured rate sources in v1, so no fractional rate state exists to
restore. Introducing rates requires explicit serialized residual/interval state.
Tests measure uniform, sparse and fully varying resident footprints, exhaust a
finite source into another site, reject overspending without mutation, restore the
depleted state and continue deterministically. Codec tests cover truncation, checksum,
version, identity, shape, channel order, capacity and total-overflow rejection, and
bounded repaired-checksum mutation smoke. Next: M6.3 environment read views.
Measured Windows x64/MSVC 19.44 fixture (130 sites): uniform or absent fields use
15768 accounted resident bytes; one varying 64-site page uses 16280; three varying
pages use 17304. A single present channel serializes to 10756 bytes regardless of
resident layout. These are owned-storage observations, not peak RSS or planetary
capacity claims. Test assertions use relative page costs so ABI metadata differences
do not falsely fail another supported platform.
+5
View File
@@ -131,3 +131,8 @@ M6.2 introduces optional fields, explicit initial inventories and versioned pers
M6.3 read views; M6.4 coordinated ownership/transfer validation; M6.5-6.8 integrated
processes, source budgets and replay. Density/composition conversion, chemical
reactions, global atmosphere/hydrology and cross-region transfer remain unsupported.
M6.2 now implements [optional fields](ENVIRONMENTAL_FIELDS.md) and independent field
snapshots under ADR-0051. Incident energy is finite stock only; no configured rates
or fractional source state exist in field format v1. Enabling rates will require
explicit versioned residual/interval persistence before any integration can commit.