M12.4: isolate continued population replay after transfer
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (ecs) (push) Successful in 2m23s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (client) (push) Canceled after 14m44s
Foundation / checks (checks) (push) Canceled after 14m44s
Foundation / checks (network) (push) Canceled after 14m43s
Foundation / checks (postgres) (push) Canceled after 14m42s
Foundation / checks (tsan) (push) Canceled after 14m43s

This commit is contained in:
dingo
2026-10-17 22:07:47 -04:00
parent 1e207b5ee4
commit 89f29fef3a
13 changed files with 209 additions and 10 deletions
+12
View File
@@ -1,5 +1,17 @@
# Project status and next steps
## M12.4e continued population replay after transfer - 2026-10-17
Portable population batch 2 binds envelope 2/owner 7 and preserves boundary flows
and nonresident history through biological intervals and manual conversions.
Batch 1 retains its old meaning/bytes; cross-version replay is refused. A fresh
process also reproduces a 64-tick destination interval after cohort transfer.
Native Release passes 78/78; Clang analysis/format, ASan/UBSan and GCC checks pass.
The private PostgreSQL suite proves both prepared C++ and SQL reject batch 2 under
retained bundle 9, without moving the head. [Scope](engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md)
keeps durable version admission, paired escrow/activation and later M12 ecology
gates open. No SQL, live flag, biological rule or deployment changed.
## M12.4d paired biological transfer replay - 2026-10-17
Canonical bounded migration trace 1 binds both predecessor cuts, both resulting
@@ -72,6 +72,12 @@ the trace's embedded request cannot supply that trust. Independent-process repla
qualifies the raw and population representations without implying durable escrow.
See [the trace schema](../../schemas/cellular-migration-trace-v1.md).
Continuing a migrated population uses separately identified population batch 2
with the same bounded intention/member layout, bound to envelope 2/owner 7.
Batch 1 continues to mean envelope 1/owner 6. Cross-version replay is refused;
the portable addition does not admit batch 2 under durable bundle 9. Boundary
accounting persists through normal biology and no-tick representation conversion.
This primitive changes ownership at an explicitly compatible cut and preserves
coordinates. It does not choose a physical route or let current clamped movement
cross an arena edge. Automatic linked-habitat movement, boundary contacts and
@@ -70,3 +70,16 @@ has no freeze, fence, durable retry outcome or activation rights. Independently
establishing manifest/payload SHA-256 bindings and journal authority remains an
I/O obligation; neither a checksum nor a UUID grants ownership. The durable
escrow and linked-geometry limits above remain unchanged.
## M12.4e portable population continuation
Population batch 2 records ordinary biological intervals and whole-group no-tick
conversions for envelope 2/owner 7. Batch 1 retains its old meaning and bytes;
cross-version replay and mismatched magic/version are refused. Continued biology
retains boundary flows/nonresident history while conserving actual current stock.
The restart fixture now also records and independently replays a 64-tick interval
after cohort transfer, comparing the complete continued destination snapshot.
This closes the portable continuation format boundary needed by a future durable
adapter. It does not add a durable bundle or SQL admission: both the current C++
prepared validator and SQL checker continue to admit only population batch 1.
+4
View File
@@ -50,6 +50,10 @@ history. They do not yet supply durable paired head publication or activation.
Paired transfer trace 1 now supplies bounded, independently replayable input and
member/material evidence, including raw/cohort separate-process reproduction.
It does not grant ownership or close database escrow/retry/partition qualification.
Population batch 2 continues owner-7 biology and conversions with retained flows;
cross-version replay is refused and a new process reproduces a post-transfer
64-tick interval. Its durable admission/upgrade remains intentionally unsupported
by bundle 9 and schema 45.
The transfer tests also expose the current rules-8 maturation gap: buds retain
their subadult realization while reproduction requires a fully realized parent.
Finite extra food alone does not close repeated wildlife generations. A versioned
+41
View File
@@ -1,5 +1,46 @@
# Validation report
## 2026-10-17 - M12.4e versioned population continuation
Portable population batch 2 is bound to envelope 2/owner 7. Batch 1 keeps its
original bytes and meaning. The existing durable prepared validator still admits
only batch 1; no SQL, biological rule, dependency, live admission or deploy changed.
Executed:
- MSVC 19.44 Release configure/build: **78/78 CTests passed**, 13.64 s.
- Clang 19 analysis build with clang-tidy: migration/checkpoint subset **3/3
passed**, 18.71 s; final reverse-version refusal test **1/1 passed**, 5.31 s.
Final repository clang-format 19 gate passes.
- Clang 19 ASan/UBSan: the three-contract subset **3/3 passed**, 64.26 s,
no reports, including a fresh process continuing a transferred cohort.
- Actual GCC 14 server/integration-test build: portable subset **3/3 passed**,
2.43 s. Final private `postgres-check.sh --population-only` **passed, exit 0,
6 s**, including explicit C++ prepared retry/resolve and direct SQL rejection
of batch 2 mislabeled as bundle 9. The SQL test requires the invalid-format
category, so an unrelated duplicate-commit conflict cannot count as proof.
The durable head remains identical after refusal.
- Explicit clang-tidy 19 on the optional population store and its integration
tests passes, with no owned warnings.
Tests replay ordinary 64-tick intervals for both sides after a real transfer,
matching complete outputs while retaining each ledger and current stock. Manual
whole-group conversion preserves exact biology. Wrong version/owner pairing and
magic/version confusion are refused. Separate-process replay reproduces the full
continued destination snapshot, including history, flows and cohort state.
Existing batch-1 tests still pass under NDEBUG; no golden was replaced.
Limits: this qualifies portable continued replay and retained admission refusal.
It does not add a durable bundle/upgrade, two-head transaction, transfer freeze,
destination activation or migration-specific database fault proof. Those remain
required before live admission. See [scope](M12_EXACT_BIOLOGICAL_TRANSFER.md).
The maintainer's `Build_Errors.txt` also records CI failures on the preceding
78-test build: the instrumented cohort qualification exceeded 180 s, and the two
new restart tests tried to create fixtures under a read-only `/source` mount.
Those CI execution constraints are not covered by the local passes above and
are being repaired as a separate validation segment; no CI pass is claimed.
## 2026-10-17 - M12.4d paired biological transfer trace and replay
ADR-0144's portable trace segment affects lifecycle replay/serialization, its
@@ -20,6 +20,7 @@ struct CellularPopulationBatch {
CellularBatch biology;
std::optional<CellularPopulationConversion> conversion;
std::vector<CellularPopulationBoundary> boundaries;
std::uint32_t format_version{1}; // 1: envelope 1/owner 6; 2: envelope 2/owner 7.
bool operator==(const CellularPopulationBatch&) const = default;
};
inline constexpr std::size_t population_batch_header_bytes = 136;
+20
View File
@@ -43,3 +43,23 @@ Their immutable SQL evidence binds both source digests and profiles; recovery
loads and verifies both payloads. A biological interval or manual conversion
instead requires full recorded trace evidence, replay and exact payload equality.
Runtime cannot publish either case through the generic region publisher.
## Portable batch 2 after biological transfer
ADR-0144 adds magic `AEVOPBT2`, format 2, with this identical bounded layout.
It requires population envelope 2 containing owner 7/rules 8, including its
boundary flows and nonresident history. Batch 1 requires envelope 1/owner 6.
Magic and full u32 version must agree; cross-version replay is refused before
executing biology. Existing batch-1 bytes and replay semantics are unchanged.
Preparation selects the trace version from the owned envelope. Batch 2 records
the same complete frozen/wildlife intentions and cohort boundaries. Biological
advancement conserves each owner's current stock and preserves cumulative flows;
death or representation conversion cannot spend that metadata. Manual conversion
retains the same no-tick whole-group contract. No new biological rule or missing
random/residual state is introduced.
Batch 2 is portable replay support only. Durable bundle 9's C++ prepared-commit
validator and schema 45's SQL trace checker admit only batch 1. A new durable
bundle/upgrade and exact history/recovery contract remain required before any
publication, scheduling or transfer activation can use the new representation.
@@ -28,7 +28,8 @@ bool canonical(const PopulationPreparedCommit& prepared, const CheckpointManifes
m.begin_tick == m.end_tick && m.batch == BatchDigests{};
const auto trace = server::decode_population_batch(prepared.trace);
const auto digest = sha256(prepared.trace);
if (!trace || !digest || m.profile != CheckpointProfile::cellular_population_v1 ||
if (!trace || !digest || trace->format_version != 1 ||
m.profile != CheckpointProfile::cellular_population_v1 ||
m.batch != BatchDigests{prepared.trace.size(), *digest, 0, {}})
return false;
const auto& h = trace->header;
@@ -6,14 +6,16 @@
namespace evo::server {
namespace {
constexpr std::uint64_t magic = 0x315442504F564541ULL; // "AEVOPBT1"
constexpr std::uint64_t magic = 0x315442504F564541ULL; // "AEVOPBT1"
constexpr std::uint64_t migration_magic = 0x325442504F564541ULL; // "AEVOPBT2"
bool shape(const CellularPopulationBatch& batch, bool recording) {
const auto& h = batch.header;
const auto zero = [](const auto& bytes) {
return std::ranges::all_of(bytes, [](auto byte) { return byte == 0; });
};
if (h.world == 0 || h.region == 0 || h.frame == 0 || zero(h.predecessor) ||
zero(h.predecessor_digest) || h.end_tick < h.begin_tick)
if ((batch.format_version != 1 && batch.format_version != 2) || h.world == 0 || h.region == 0 ||
h.frame == 0 || zero(h.predecessor) || zero(h.predecessor_digest) ||
h.end_tick < h.begin_tick)
return false;
const auto count = batch.conversion ? std::size_t{1} : batch.biology.ticks.size();
if (count == 0 || count > max_cellular_batch_ticks ||
@@ -117,9 +119,9 @@ unattended_population_inputs(const CellularLives& base, const CellularBatchHeade
std::expected<std::unique_ptr<CellularPopulation>, CellularPopulationError>
execute_population_batch(const CellularPopulation& base, CellularPopulationBatch& batch,
bool record) {
// Trace 1 is retained for envelope 1. A future durable migration adapter must
// bind the new owner/envelope tuple explicitly before recording its history.
if (base.format_version() != 1)
// Each trace version retains its exact owner/envelope meaning. Portable trace
// 2 is not admission to an existing durable bundle.
if (base.format_version() != batch.format_version)
return std::unexpected(CellularPopulationError::unsupported);
if (!shape(batch, record) || batch.header.world != base.state().world() ||
batch.header.region != base.state().region() ||
@@ -192,6 +194,7 @@ prepare_population_batch(const CellularPopulation& base, CellularBatch biology)
if (!population_interval_safe(base.state()))
return std::unexpected(CellularPopulationError::ineligible);
CellularPopulationBatch batch{biology.header, std::move(biology), {}, {}};
batch.format_version = base.format_version();
auto state = execute_population_batch(base, batch, true);
if (!state)
return std::unexpected(state.error());
@@ -201,6 +204,7 @@ std::expected<CellularPopulationExecution, CellularPopulationError>
prepare_population_conversion(const CellularPopulation& base, CellularBatchHeader header,
CellularPopulationConversion conversion) {
CellularPopulationBatch batch{header, {}, conversion, {}};
batch.format_version = base.format_version();
auto state = execute_population_batch(base, batch, true);
if (!state)
return std::unexpected(state.error());
@@ -224,8 +228,8 @@ encode_population_batch(const CellularPopulationBatch& batch) {
biological = std::move(*encoded);
}
std::vector<std::uint8_t> out;
put(out, magic);
put(out, 1, 4);
put(out, batch.format_version == 2 ? migration_magic : magic);
put(out, batch.format_version, 4);
put(out, batch.conversion ? 1 : 0, 4);
const auto& h = batch.header;
for (const auto value : {h.world, h.region, h.frame})
@@ -256,10 +260,14 @@ decode_population_batch(std::span<const std::uint8_t> bytes) {
if (bytes.size() < population_batch_header_bytes || bytes.size() > max_population_batch_bytes)
return std::unexpected(CellularPopulationError::format);
Reader in(bytes);
if (in.get() != magic || in.get(4) != 1)
const auto saved_magic = in.get();
const auto version = in.get(4);
if (!((saved_magic == magic && version == 1) ||
(saved_magic == migration_magic && version == 2)))
return std::unexpected(CellularPopulationError::unsupported);
const auto kind = in.get(4);
CellularPopulationBatch batch;
batch.format_version = static_cast<std::uint32_t>(version);
auto& h = batch.header;
h.world = in.get();
h.region = in.get();
+6
View File
@@ -18,3 +18,9 @@ foreach(representation raw pop)
endif()
endforeach()
endforeach()
execute_process(COMMAND "${CMAKE_COMMAND}" -E compare_files
"${prefix}.pop.continuation.expected" "${prefix}.pop.continuation.resumed"
RESULT_VARIABLE continued)
if(NOT continued EQUAL 0)
message(FATAL_ERROR "Transferred population interval diverged across process replay")
endif()
+57
View File
@@ -153,6 +153,13 @@ CellularMigrationRequest request_for(const CellularLives& source, const Cellular
{target.region(), filled<16>(2), filled<16>(4), filled<32>(6)},
wildlife_genome(source)};
}
CellularPopulationExecution continuation(const CellularPopulation& population) {
const auto& s = population.state();
const CellularBatchHeader header{s.world(), s.region(), s.frame(), filled<16>(9),
filled<32>(10), s.tick(), s.tick() + 64};
auto inputs = checked(unattended_population_inputs(s, header), "transferred interval inputs");
return checked(prepare_population_batch(population, std::move(inputs)), "transferred interval");
}
void trace_contract() {
auto source = checked(funded()->migration_upgrade_candidate(), "trace source");
auto target = checked(destination(*source, 1)->migration_upgrade_candidate(), "trace target");
@@ -250,6 +257,46 @@ void trace_contract() {
require(!replay_migration_trace(*population, *peer, request, trace) &&
!replay_migration_trace(*source, *target, request, pop_trace),
"trace representation confused");
for (const auto* owner : {pop_replayed.source.get(), pop_replayed.destination.get()}) {
const auto bytes_before = owner->snapshot();
const auto ledger_before = owner->state().boundary_ledger();
auto interval = continuation(*owner);
require(interval.batch.format_version == 2, "new owner recorded old trace version");
const auto encoded =
checked(encode_population_batch(interval.batch), "encode new interval");
const auto decoded = checked(decode_population_batch(encoded), "decode new interval");
auto replay = checked(replay_population_batch(*owner, decoded), "replay new interval");
require(replay->snapshot() == interval.state->snapshot() &&
replay->state().boundary_ledger() == ledger_before &&
replay->state().current_total() == owner->state().current_total() &&
owner->snapshot() == bytes_before,
"new interval changed flows/history or failed exact replay");
auto legacy = decoded;
legacy.format_version = 1;
require(!replay_population_batch(*owner, legacy), "old trace reinterpreted for owner 7");
auto malformed = encoded;
malformed[7] = '1';
require(!decode_population_batch(malformed),
"new population trace magic/version confusion");
if (!owner->members().empty()) {
const auto& s = owner->state();
const CellularBatchHeader h{s.world(), s.region(), s.frame(), filled<16>(9),
filled<32>(10), s.tick(), s.tick()};
auto conversion = checked(
prepare_population_conversion(*owner, h, {owner->members().front().genome, false}),
"new manual trace");
const auto manual =
checked(decode_population_batch(checked(encode_population_batch(conversion.batch),
"new manual encode")),
"new manual decode");
auto materialized =
checked(replay_population_batch(*owner, manual), "new manual replay");
require(manual.format_version == 2 &&
materialized->state().snapshot() == s.snapshot() &&
materialized->members().empty(),
"new conversion changed exact biology");
}
}
require(source->snapshot() == before && target->snapshot() == peer_before,
"trace validation changed its input");
}
@@ -293,6 +340,10 @@ void record_restart(const std::string& prefix) {
checked(encode_migration_trace(pop.trace), "restart pop trace"));
write_bytes(prefix + ".pop.source.expected", pop.execution.source->snapshot());
write_bytes(prefix + ".pop.destination.expected", pop.execution.destination->snapshot());
auto interval = continuation(*pop.execution.destination);
write_bytes(prefix + ".pop.continuation.trace",
checked(encode_population_batch(interval.batch), "restart interval trace"));
write_bytes(prefix + ".pop.continuation.expected", interval.state->snapshot());
}
void replay_restart(const std::string& prefix) {
const auto raw =
@@ -321,6 +372,12 @@ void replay_restart(const std::string& prefix) {
"fresh population replay");
write_bytes(prefix + ".pop.source.resumed", pop_replayed.source->snapshot());
write_bytes(prefix + ".pop.destination.resumed", pop_replayed.destination->snapshot());
const auto interval =
checked(decode_population_batch(read_bytes(prefix + ".pop.continuation.trace")),
"fresh interval trace");
auto continued = checked(replay_population_batch(*pop_replayed.destination, interval),
"fresh transferred continuation");
write_bytes(prefix + ".pop.continuation.resumed", continued->snapshot());
}
void upgrade_and_move() {
auto original = funded();
+11
View File
@@ -159,6 +159,17 @@ void manifests() {
require(!restore_cellular_payload(population->snapshot(), next),
"population mislabeled as cellular");
auto future = checked(population->migration_upgrade_candidate(), "future portable envelope");
const auto member = std::ranges::find_if(
future->state().lives(), [](const auto& life) { return life.account == LifeAccount{}; });
require(member != future->state().lives().end(), "future trace fixture lacks wildlife");
auto future_conversion = checked(
prepare_population_conversion(*future, header(*future, 0), {member->identity.genome, true}),
"future portable conversion");
require(future_conversion.batch.format_version == 2,
"future owner recorded historical population trace");
auto legacy_replay = replay_population_batch(*population, future_conversion.batch);
require(!legacy_replay && legacy_replay.error() == CellularPopulationError::unsupported,
"new trace reinterpreted for old owner");
auto mislabeled = next;
mislabeled.payload_bytes = future->snapshot().size();
auto refused = restore_population_payload(future->snapshot(), mislabeled);
+19
View File
@@ -259,6 +259,25 @@ void conversions_and_retries(Fixture& f) {
require(take(resolve_population_prepared(*f.db, prepared, 5s), "read-only resolve").status ==
Resolution::committed,
"commit unresolved");
auto future_batch = conversion.batch;
future_batch.format_version = 2;
auto future = prepared;
future.trace = take(encode_population_batch(future_batch), "future trace fixture");
auto future_manifest = take(decode_manifest(future.manifest), "future manifest fixture");
future_manifest.batch.trace_bytes = future.trace.size();
future_manifest.batch.trace_sha256 = take(sha256(future.trace), "future trace digest");
future.manifest = take(encode_manifest(future_manifest), "future manifest encoding");
auto rejected_future = retry_population_prepared(*f.db, future, 5s);
require(!rejected_future && rejected_future.error().kind == PublicationError::candidate,
"bundle 9 prepared retry admitted population trace 2");
require(!resolve_population_prepared(*f.db, future, 5s),
"bundle 9 prepared resolve admitted population trace 2");
auto rejected_sql = publish_population_checkpoint(*f.db, future.manifest, future.trace, 5s);
require(!rejected_sql && rejected_sql.error().kind == LeaseStoreError::invalid,
"schema 45 did not reject the unsupported trace format before commit lookup");
require(take(load_region_checkpoint(*f.db, f.key, 5s), "head after future rejection").head ==
f.head.head,
"future trace changed the retained durable head");
auto changed = prepared;
auto changed_manifest = take(decode_manifest(changed.manifest), "conflict manifest");
changed_manifest.payload_sha256[0] ^= 1;