Refuse the whole conversion rather than drop the inconvenient member

M4.2 moves individuals into the M4.1 record and does only that: no advance, no
materialization back, no transfer between habitats, no persistence.

Collapse is two phases over one sample. plan_collapse reads the members once and
returns an immutable plan that already carries a validated cohort; commit applies
exactly that plan and derives nothing, so replaying a committed sample cannot
resample, and a record that would not validate fails during planning rather than
after the ledger has moved.

A controlled, suspended, dead or unsettled member refuses the whole plan instead
of being skipped. Filtering would let a caller obtain a plan by omitting whoever
was inconvenient, and the result would be indistinguishable from an honest
conversion, which is the adversarial selection the model forbids. Ages are
carried into declared classes and a member fitting none refuses the plan, so
nobody is quietly moved or dropped for having nowhere to go. Reserves are summed
per class and never redistributed, so the distribution M4.1 exists to keep
survives the move.

Conservation runs through the ledger rather than beside it: one atomic
transaction, one leg per member, each moving that member's entire reserve, so the
conserved total is unchanged by construction. Dual representation is closed from
both ends. A member must be emptied exactly, and a reservoir holding anything
other than what was sampled makes the plan stale rather than partly applicable.
The destination is proved available before anything moves, so no rejection can
strand biomass in a reservoir nothing owns. Reversing that order is detected by
the tests, which is the point: the ordering prevents the failure rather than a
comment claiming it cannot happen.

Two gaps are stated rather than implied covered. This cannot tell whether the
members offered were all there were, so completeness is the caller's obligation.
And emptying a reservoir is not withdrawing the right to use it, because the
ledger has no removal.

ReservoirId and Transfer gain the defaulted comparisons every other value type
here already has, so a plan's legs can be compared. Additive, and ADR-0037 says
so rather than claiming no M1 change.

All ten guards were confirmed load-bearing by removing each and rerunning. One of
my own expectations was wrong rather than the code: a replayed commit reported
stale_plan where I expected destination_occupied. Both are pure checks made
before anything moves, so only the diagnosis differed; the destination check runs
first now, because "this already happened" says more about a replay.

Release 35/35, analysis, ASan 35/35 and clang-format all ran and passed here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-09-24 09:45:09 -04:00
co-authored by Claude Opus 5
parent 5a4c34a903
commit 1a59b63c7a
11 changed files with 932 additions and 3 deletions
+13
View File
@@ -106,6 +106,19 @@ genome and refuses replacement under a wrong worker or an older epoch, so
ECOLOGY_MODEL's "no inventory spendable in two tiers" is enforced rather than
described. No collapse, advance or materialization exists yet.
M4.2 moves individuals into that record (ADR-0037). Collapse is two phases over
one sample: `plan_collapse` reads the members once and returns an immutable plan
carrying an already-validated cohort, and `commit_collapse` applies exactly that
plan without deriving anything, so a replay cannot resample. A controlled,
suspended, dead or unsettled member refuses the whole plan instead of being
skipped, because filtering would make adversarial selection indistinguishable from
honest use. Ages are carried into declared classes and a member that fits none
refuses the plan rather than being moved. Conservation runs through the ledger as
one atomic transaction of one leg per member, each moving that member's whole
reserve, and the destination is proved available before anything moves, so every
rejection leaves both the ledger and the registry exactly as they were. What this
cannot check is that the caller offered every member it had.
M3.6 closes M3 by making the pieces meet (ADR-0035): `strategy_for` turns a genome
into the `StrategyConfig` the M2.6 driver already takes, founders are pre-diverged
variants of one ancestor admitted to the archive, and `evo-sim --ensemble` runs
+36 -2
View File
@@ -7,7 +7,7 @@ persistent simulation server. The founding vision remains unchanged.
## Completed implementation
- Vision, architectural pillars, risks, open-question register, architecture
diagrams, biological contracts, engineering constitution and 36 ADRs.
diagrams, biological contracts, engineering constitution and 37 ADRs.
Proposed/deferred choices remain distinct from accepted proof decisions.
- M0–M4 task breakdowns with dependencies, acceptance evidence and risks,
plus M5–M14 gates in [ROADMAP.md](ROADMAP.md).
@@ -61,7 +61,7 @@ for the bounded proof; the general production ECS remains a later measured choic
| Priority | Work | Completion evidence |
|---|---|---|
| 1 | M4.2 collapse active individuals | Transfer IDs and reserves into one cohort atomically, with exact ledger conservation |
| 1 | M4.3 advance local cohorts | Conservative birth, death, predation and migration accounting per interval |
| 2 | Revisit the ADR-0031/0032 reproduction cost model | Reproduction is affordable only at the genome schema's floor; decide whether that is intended before M4 builds cohorts on it |
| Delivery gate | Add/update the dev workload when a deployable increment needs it; Argo is ready | Observe Job/pod completion separately from image publication; no World0 changes |
@@ -332,3 +332,37 @@ classes while keeping its identities, and the registry then accepted through a
move exactly what `create` rejects by name; the type is now copy-only, as ADR-0030
already made the genome for the same reason. That precedent existed and should
have been applied when the type was written.
M4.2 moves individuals into that record (ADR-0037), and does only that: there is
no population advance, no materialization back into individuals, no transfer
between habitats and no persistence. Collapse is two phases over one sample.
`plan_collapse` reads the members once and returns an immutable plan that already
carries a validated cohort; `commit_collapse` applies exactly that plan and
derives nothing, so ECOLOGY_MODEL's rule that recovery replays a committed sample
rather than sampling again is a property of the types.
A controlled, suspended, dead or unsettled member refuses the whole plan rather
than being skipped. That is the answer to adversarial selection: filtering would
let a caller obtain a plan by omitting whoever was inconvenient, and the result
would be indistinguishable from an honest conversion. Ages are carried into
declared classes and a member fitting none refuses the plan, so a conversion
cannot quietly move anyone to a different class or discard one for having nowhere
to go. Reserves are summed per class, never redistributed.
Conservation runs through the ledger rather than beside it: one atomic transaction
of one leg per member, each moving that member's entire reserve, so the conserved
total is unchanged by construction. Dual representation is closed from both ends.
A member must be emptied exactly, and a reservoir holding anything other than what
was sampled makes the plan stale rather than partly applicable. The destination is
proved available before anything moves, so no rejection can strand inventory in a
reservoir nothing owns, and every refusal leaves the ledger and the registry as
they were. Committing one plan twice is refused without moving inventory again.
Two limits are stated rather than implied to be covered. The function cannot tell
whether the members it was given are all the members there were, so completeness
is a caller obligation. And emptying a reservoir is not withdrawing the right to
use it: the ledger has no removal, so nothing prevents a later transfer crediting
that reservoir again. Durable conversion sequences, escrow and replay across a
process boundary are M7 and M8. All ten construction and commit guards were
confirmed load-bearing by removing each and observing a failing run. Native
Release, analysis, ASan and formatting passed 35/35; Linux CI has not run yet.
+132
View File
@@ -0,0 +1,132 @@
# ADR-0037: Collapsing individuals into a cohort as one sampled transaction
## Status
Adopted for M4.2. It converts identified individuals into the ADR-0036 record and
nothing else: there is no population advance, no materialization back into
individuals, no transfer between habitats and no persistence. Those are M4.3,
M4.4 and M7. ADR-0036 is unchanged. The only change outside this milestone is
additive: `ReservoirId` and `Transfer` gain the defaulted comparisons every other
value type in the project already has, so a plan's legs can be compared. No
accounting behaviour changes.
## Context
ECOLOGY_MODEL's headline risk for this step is dual representation: the same
biomass existing as individuals and as a cohort at once, or existing as neither
because a conversion stopped halfway. It also forbids an adversarial conversion
that selects only healthy or high-energy members, or that resets age, injury,
fertility or incubation on the way in.
M4.1 gave the destination and proved it cannot be double-owned. What is missing is
the move itself, and the move is where conservation is either exact or quietly
wrong. The project already has the instrument for that: `accounting::Ledger`
applies ordered legs to a candidate copy and commits only if every leg holds, so
an atomic transfer is available rather than something this ADR has to invent.
## Decision
Collapse is two phases over one sample. `plan_collapse` reads the members once and
returns a `CollapsePlan`; `commit_collapse` applies exactly that plan and derives
nothing new. ECOLOGY_MODEL requires that recovery replays the committed sample and
never samples again, so the plan is the sample: it is immutable, it carries the
already-validated cohort, and committing it twice cannot resample.
A member that is controlled, suspended, dead or unsettled refuses the whole plan
rather than being skipped. This is the part that answers adversarial selection.
Filtering would mean a caller could always obtain a plan by quietly omitting
whoever was inconvenient, and the result would look identical to an honest one.
Refusing loudly means an excluded member is a plan that failed and named who.
What this cannot establish is that the caller offered every member it had;
completeness of the input set is outside what this function can see, and is stated
as a limit rather than implied to be covered.
Age is carried, never reset. The request declares class bounds, each member enters
the class containing its age, and a member whose age falls outside every declared
bound refuses the plan. A conversion therefore cannot quietly move anyone to a
different age class, and cannot discard one by having nowhere to put it.
Conservation runs through the ledger rather than beside it. The plan holds one leg
per member, moving that member's entire reserve to the cohort's reservoir, and
commit submits them as a single transaction. The ledger's conserved total is
unchanged by construction, so conservation is a property of the instrument rather
than an assertion this code makes about itself.
Dual representation is closed from both ends. A member must be emptied exactly: at
commit each member's reservoir has to hold precisely what the plan sampled, and
anything else means the world moved since the sample and the plan is stale.
Leaving a remainder behind would be the same biomass in two tiers, which
ECOLOGY_MODEL locks against.
Ordering inside commit is chosen so no failure can strand inventory. The
destination is checked before anything moves: the registry must not already hold
this habitat and genome and must have room. Only then does the ledger transaction
run, and only then is the cohort installed, which cannot fail because the plan
carries a cohort already validated and the destination was already proved
available. Every rejection therefore leaves the ledger and the registry exactly as
they were.
## Alternatives Considered
Skipping ineligible members instead of refusing would make adversarial selection
indistinguishable from ordinary use, which is the risk this step is named after.
Moving inventory first and installing afterwards would allow a transfer whose
destination then refuses, stranding biomass in a reservoir nothing owns. Building
the cohort during commit rather than during planning would let commit fail on
validity after the ledger had already moved. Summing member reserves into a single
total and redistributing it across classes would conserve mass while destroying
the distribution ADR-0036 exists to keep. Letting the collapse write directly into
a cohort's inventory without a ledger transaction would make conservation
something this code claims rather than something the instrument enforces. Removing
each member's reservoir from the ledger would be closer to "remove active spending
rights", but the ledger deliberately has no removal; emptying is the strongest
form available and is what the tests assert.
## Benefits
Conservation is exact because the ledger enforces it, not because a test checks a
sum. Every failure path leaves the world unchanged, so there is no partial
conversion to recover from. A conversion that drops a member fails and says which
one. Ages and the reserve distribution survive the move intact. The plan is a
value, so the sample can be inspected before anything is committed.
## Costs
A plan is bounded to 256 members, because that is the ledger's transaction limit
and one leg is needed per member; larger groups need several conversions, which
this ADR does not coordinate. The plan holds a copy of the cohort and of every
leg. Members are supplied by the caller as an explicit record rather than read
from any live representation, so something has to map a world onto them, and that
mapping is not written here.
## Risks
This is not persistence. The before-and-after property holds across a rejection
inside one process; a crash between the ledger transaction and the registry
install is not something an in-memory registry can be recovered from, and durable
conversion sequence numbers, escrow and replay are M7 and M8. The function cannot
tell whether the members it was given are all the members there were, so
completeness is a caller obligation and an honest gap. Emptying a reservoir is not
the same as withdrawing the right to use it: nothing prevents a later transfer
crediting that reservoir again, and only the cohort registry's single-ownership
rule limits what that could mean. Nothing here says a cohort may then advance, and
a collapsed cohort cannot yet be turned back into individuals, so this direction
is one-way until M4.4.
## Consequences
Test that a plan totals its members exactly, that classes carry the ages and
reserves the members had, and that a member's reserve is never redistributed. Test
that every ineligible state refuses the whole plan and names the member, that a
foreign habitat or genome, a duplicate identity, an unclassifiable age, an empty
group and a group beyond the transaction limit are each refused. Test that a
committed collapse leaves the ledger's conserved total unchanged, empties every
member reservoir, credits the destination with exactly the planned total and
installs the cohort. Test that a plan which is never committed changes nothing,
that a commit refused at the destination or on a stale balance changes nothing in
either the ledger or the registry, and that committing one plan twice is refused
without moving inventory a second time. Test that the plan is copy-only, since a
moved-from plan would keep its identities and lose its legs. No genome schema,
mutation catalog, archive, episode driver, ensemble, cohort record, renderer or
deployment changes. The ledger changes only by gaining defaulted comparisons on
two of its value types, which no behaviour depends on.
+1
View File
@@ -29,6 +29,7 @@ Each record has the eight required sections.
0034 adopts the permanent lineage archive and absence that is never extinction.
0035 derives strategies from genomes and measures them over a split-half gradient.
0036 defines the tier 2 cohort, its declared resolution and exclusive ownership.
0037 collapses individuals into a cohort as one sampled, ledger-exact transaction.
Every dependency adoption records exact version/commit and integrity hash,
transitive license obligations, maintenance/security review, owner and removal
+34
View File
@@ -157,3 +157,37 @@ Carrying a bound is not meeting one; M4.5 is where resolution equivalence is
measured. Conservation between a cohort and the rest of the world is the ledger's
property and is not asserted by this type, which holds an inventory without
participating in any transfer.
## Implemented M4.2 collapse
[ADR-0037](../adr/0037-collapse-to-cohort.md) implements `PlanCollapse` and
`CommitCollapse` for a single habitat and genome in one process. It settles
nothing else: `AdvancePopulation`, `PlanMaterialize`, `CommitMaterialize` and
`TransferPopulation` remain contracts nothing implements, and the direction is
one-way until M4.4.
Collapse is two phases over one sample, so recovery replays a committed sample and
never samples again: the plan is immutable, carries an already-validated cohort,
and commit derives nothing. Members that are controlled, suspended, dead or
unsettled are rejected as this document requires, and rejection refuses the whole
plan rather than dropping the member. That is what makes adversarial selection
visible: a conversion cannot quietly exclude the unhealthy and still look honest.
Age, and the per-class reserve distribution, are carried through unchanged; a
member whose age fits no declared class refuses the plan rather than being moved
into one.
Exact inventory is totalled through the ledger, not alongside it. One atomic
transaction carries one leg per member, each moving that member's entire reserve,
so the conserved total is unchanged by construction and no remainder is left in a
second tier. The destination is proved available before anything moves, so a
refusal cannot strand biomass in a reservoir nothing owns, and every rejection
leaves both the ledger and the registry exactly as they were.
Two gaps are explicit. This cannot establish that the members offered were all the
members there were; completeness is the caller's obligation. And "remove active
spending rights" is implemented as emptying, because the ledger has no reservoir
removal: nothing prevents a later transfer crediting an emptied reservoir, and
only the registry's single-ownership rule bounds what that would mean. Durable
conversion sequence numbers, escrow and crash recovery across a process boundary
are M7 and M8; the before-and-after property proved here holds across a rejection
inside one process.
+66
View File
@@ -961,3 +961,69 @@ No existing contract changed. The proof golden `d61ad5bf0c928c76`, the ecosystem
fingerprint `2cd4a15f9fbe9949`, the ensemble fingerprint `5ec59d390909b526` and
the frozen genome fixture are unchanged. Linux validation is pending; no Gitea
result is claimed since `b01a824`.
## M4.2 collapse to cohort
Executed on this Windows host (2026-09-24): Release configure, build and `ctest`
35/35; the analysis preset (clang-tidy 19, warnings as errors) built clean; the
ASan preset configured, built and ran 35/35; `clang-format` 19.1.5 reported no
violations. All four checks ran and passed on the committed tree. No Clang or GCC
compile happens on this host, so Linux CI remains the first such result.
`evo-collapse-tests` covers seven properties: that a plan totals its members
exactly and moves each member's whole reserve, that ages and the per-class reserve
distribution survive the move without redistribution, that every ineligible state
and every malformed request refuses the whole plan by name, that a committed
collapse leaves the ledger's conserved total unchanged while emptying every member
reservoir and crediting the destination, that every rejection leaves both the
ledger and the registry exactly as they were, that one plan cannot be committed
twice, and that a moved-from plan is still whole.
All ten guards were confirmed load-bearing by removing each in turn and rebuilding:
the ineligible, foreign, duplicate, self-destination, age classification, stale
sample, occupied destination, empty class and commit ordering guards, and the
copy-only declaration. Two needed a second pass. The first age mutation did not
compile, which is not evidence about the test, so it was rewritten to make the
class predicate always match, which places every member in the first class and is
detected at runtime. Removing the copy declarations fails to compile in Release
for the reason recorded for the cohort; with the static assertion also removed the
same mutation was rerun under ASan, where it exits 1, so the runtime assertions
are load-bearing independently of the compile-time proof.
The commit ordering mutation is worth naming: moving the ledger transaction ahead
of the destination check is detected, because a rejected conversion then leaves
biomass in a reservoir nothing owns while the members are already empty. That is
the dual-representation failure this milestone is named after, and the ordering is
what prevents it rather than a comment saying it does not happen.
One expectation of my own was wrong rather than the code. A replay of a committed
plan was expected to report `destination_occupied`, but the staleness check ran
first and reported `stale_plan`, which is equally true. Both are pure checks made
before anything moves, so safety was unaffected and only the diagnosis differed;
the destination check now runs first, because "this conversion already happened"
says more about a replay than "the world moved". The ordering rationale is
recorded beside it.
`accounting::ReservoirId` and `accounting::Transfer` gained the defaulted
comparisons that every other value type in the project already carries, so a
plan's legs can be compared. This is additive, no behaviour depends on it, and
ADR-0037's status was corrected to say so rather than claim no M1 change.
Coverage limits. Completeness of the offered member set is not checkable here and
is not checked: nothing establishes that the caller offered every member it had.
Emptying a reservoir is not withdrawing the right to use it, since the ledger has
no removal, so a later transfer could credit an emptied reservoir and no test
asserts otherwise. The before-and-after property is demonstrated across rejections
inside one process; there is no persistence, so a crash between the ledger
transaction and the registry install is outside what an in-memory registry can
show, and durable conversion sequences, escrow and replay stay M7 and M8. Plans
are bounded to 256 members by the ledger's transaction limit and nothing
coordinates several conversions. The ledger-failure and destination-unavailable
branches are reachable only past bounds the tests do not build, so they are
refused-by-construction rather than exercised.
No existing behaviour changed. The proof golden `d61ad5bf0c928c76`, the ecosystem
fingerprint `2cd4a15f9fbe9949`, the ensemble fingerprint `5ec59d390909b526` and
the frozen genome fixture are unchanged, and the other 34 tests pass untouched.
No deployment manifest or image pin changed. Linux validation is pending; no Gitea
result is claimed since `b01a824`.