M12.4: isolate continued population replay after transfer
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (ecs) (push) Successful in 2m23s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (client) (push) Canceled after 14m44s
Foundation / checks (checks) (push) Canceled after 14m44s
Foundation / checks (network) (push) Canceled after 14m43s
Foundation / checks (postgres) (push) Canceled after 14m42s
Foundation / checks (tsan) (push) Canceled after 14m43s
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (ecs) (push) Successful in 2m23s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (client) (push) Canceled after 14m44s
Foundation / checks (checks) (push) Canceled after 14m44s
Foundation / checks (network) (push) Canceled after 14m43s
Foundation / checks (postgres) (push) Canceled after 14m42s
Foundation / checks (tsan) (push) Canceled after 14m43s
This commit is contained in:
@@ -1,5 +1,17 @@
|
||||
# Project status and next steps
|
||||
|
||||
## M12.4e continued population replay after transfer - 2026-10-17
|
||||
|
||||
Portable population batch 2 binds envelope 2/owner 7 and preserves boundary flows
|
||||
and nonresident history through biological intervals and manual conversions.
|
||||
Batch 1 retains its old meaning/bytes; cross-version replay is refused. A fresh
|
||||
process also reproduces a 64-tick destination interval after cohort transfer.
|
||||
Native Release passes 78/78; Clang analysis/format, ASan/UBSan and GCC checks pass.
|
||||
The private PostgreSQL suite proves both prepared C++ and SQL reject batch 2 under
|
||||
retained bundle 9, without moving the head. [Scope](engineering/M12_EXACT_BIOLOGICAL_TRANSFER.md)
|
||||
keeps durable version admission, paired escrow/activation and later M12 ecology
|
||||
gates open. No SQL, live flag, biological rule or deployment changed.
|
||||
|
||||
## M12.4d paired biological transfer replay - 2026-10-17
|
||||
|
||||
Canonical bounded migration trace 1 binds both predecessor cuts, both resulting
|
||||
|
||||
@@ -72,6 +72,12 @@ the trace's embedded request cannot supply that trust. Independent-process repla
|
||||
qualifies the raw and population representations without implying durable escrow.
|
||||
See [the trace schema](../../schemas/cellular-migration-trace-v1.md).
|
||||
|
||||
Continuing a migrated population uses separately identified population batch 2
|
||||
with the same bounded intention/member layout, bound to envelope 2/owner 7.
|
||||
Batch 1 continues to mean envelope 1/owner 6. Cross-version replay is refused;
|
||||
the portable addition does not admit batch 2 under durable bundle 9. Boundary
|
||||
accounting persists through normal biology and no-tick representation conversion.
|
||||
|
||||
This primitive changes ownership at an explicitly compatible cut and preserves
|
||||
coordinates. It does not choose a physical route or let current clamped movement
|
||||
cross an arena edge. Automatic linked-habitat movement, boundary contacts and
|
||||
|
||||
@@ -70,3 +70,16 @@ has no freeze, fence, durable retry outcome or activation rights. Independently
|
||||
establishing manifest/payload SHA-256 bindings and journal authority remains an
|
||||
I/O obligation; neither a checksum nor a UUID grants ownership. The durable
|
||||
escrow and linked-geometry limits above remain unchanged.
|
||||
|
||||
## M12.4e portable population continuation
|
||||
|
||||
Population batch 2 records ordinary biological intervals and whole-group no-tick
|
||||
conversions for envelope 2/owner 7. Batch 1 retains its old meaning and bytes;
|
||||
cross-version replay and mismatched magic/version are refused. Continued biology
|
||||
retains boundary flows/nonresident history while conserving actual current stock.
|
||||
The restart fixture now also records and independently replays a 64-tick interval
|
||||
after cohort transfer, comparing the complete continued destination snapshot.
|
||||
|
||||
This closes the portable continuation format boundary needed by a future durable
|
||||
adapter. It does not add a durable bundle or SQL admission: both the current C++
|
||||
prepared validator and SQL checker continue to admit only population batch 1.
|
||||
|
||||
@@ -50,6 +50,10 @@ history. They do not yet supply durable paired head publication or activation.
|
||||
Paired transfer trace 1 now supplies bounded, independently replayable input and
|
||||
member/material evidence, including raw/cohort separate-process reproduction.
|
||||
It does not grant ownership or close database escrow/retry/partition qualification.
|
||||
Population batch 2 continues owner-7 biology and conversions with retained flows;
|
||||
cross-version replay is refused and a new process reproduces a post-transfer
|
||||
64-tick interval. Its durable admission/upgrade remains intentionally unsupported
|
||||
by bundle 9 and schema 45.
|
||||
The transfer tests also expose the current rules-8 maturation gap: buds retain
|
||||
their subadult realization while reproduction requires a fully realized parent.
|
||||
Finite extra food alone does not close repeated wildlife generations. A versioned
|
||||
|
||||
@@ -1,5 +1,46 @@
|
||||
# Validation report
|
||||
|
||||
## 2026-10-17 - M12.4e versioned population continuation
|
||||
|
||||
Portable population batch 2 is bound to envelope 2/owner 7. Batch 1 keeps its
|
||||
original bytes and meaning. The existing durable prepared validator still admits
|
||||
only batch 1; no SQL, biological rule, dependency, live admission or deploy changed.
|
||||
|
||||
Executed:
|
||||
|
||||
- MSVC 19.44 Release configure/build: **78/78 CTests passed**, 13.64 s.
|
||||
- Clang 19 analysis build with clang-tidy: migration/checkpoint subset **3/3
|
||||
passed**, 18.71 s; final reverse-version refusal test **1/1 passed**, 5.31 s.
|
||||
Final repository clang-format 19 gate passes.
|
||||
- Clang 19 ASan/UBSan: the three-contract subset **3/3 passed**, 64.26 s,
|
||||
no reports, including a fresh process continuing a transferred cohort.
|
||||
- Actual GCC 14 server/integration-test build: portable subset **3/3 passed**,
|
||||
2.43 s. Final private `postgres-check.sh --population-only` **passed, exit 0,
|
||||
6 s**, including explicit C++ prepared retry/resolve and direct SQL rejection
|
||||
of batch 2 mislabeled as bundle 9. The SQL test requires the invalid-format
|
||||
category, so an unrelated duplicate-commit conflict cannot count as proof.
|
||||
The durable head remains identical after refusal.
|
||||
- Explicit clang-tidy 19 on the optional population store and its integration
|
||||
tests passes, with no owned warnings.
|
||||
|
||||
Tests replay ordinary 64-tick intervals for both sides after a real transfer,
|
||||
matching complete outputs while retaining each ledger and current stock. Manual
|
||||
whole-group conversion preserves exact biology. Wrong version/owner pairing and
|
||||
magic/version confusion are refused. Separate-process replay reproduces the full
|
||||
continued destination snapshot, including history, flows and cohort state.
|
||||
Existing batch-1 tests still pass under NDEBUG; no golden was replaced.
|
||||
|
||||
Limits: this qualifies portable continued replay and retained admission refusal.
|
||||
It does not add a durable bundle/upgrade, two-head transaction, transfer freeze,
|
||||
destination activation or migration-specific database fault proof. Those remain
|
||||
required before live admission. See [scope](M12_EXACT_BIOLOGICAL_TRANSFER.md).
|
||||
|
||||
The maintainer's `Build_Errors.txt` also records CI failures on the preceding
|
||||
78-test build: the instrumented cohort qualification exceeded 180 s, and the two
|
||||
new restart tests tried to create fixtures under a read-only `/source` mount.
|
||||
Those CI execution constraints are not covered by the local passes above and
|
||||
are being repaired as a separate validation segment; no CI pass is claimed.
|
||||
|
||||
## 2026-10-17 - M12.4d paired biological transfer trace and replay
|
||||
|
||||
ADR-0144's portable trace segment affects lifecycle replay/serialization, its
|
||||
|
||||
@@ -20,6 +20,7 @@ struct CellularPopulationBatch {
|
||||
CellularBatch biology;
|
||||
std::optional<CellularPopulationConversion> conversion;
|
||||
std::vector<CellularPopulationBoundary> boundaries;
|
||||
std::uint32_t format_version{1}; // 1: envelope 1/owner 6; 2: envelope 2/owner 7.
|
||||
bool operator==(const CellularPopulationBatch&) const = default;
|
||||
};
|
||||
inline constexpr std::size_t population_batch_header_bytes = 136;
|
||||
|
||||
@@ -43,3 +43,23 @@ Their immutable SQL evidence binds both source digests and profiles; recovery
|
||||
loads and verifies both payloads. A biological interval or manual conversion
|
||||
instead requires full recorded trace evidence, replay and exact payload equality.
|
||||
Runtime cannot publish either case through the generic region publisher.
|
||||
|
||||
## Portable batch 2 after biological transfer
|
||||
|
||||
ADR-0144 adds magic `AEVOPBT2`, format 2, with this identical bounded layout.
|
||||
It requires population envelope 2 containing owner 7/rules 8, including its
|
||||
boundary flows and nonresident history. Batch 1 requires envelope 1/owner 6.
|
||||
Magic and full u32 version must agree; cross-version replay is refused before
|
||||
executing biology. Existing batch-1 bytes and replay semantics are unchanged.
|
||||
|
||||
Preparation selects the trace version from the owned envelope. Batch 2 records
|
||||
the same complete frozen/wildlife intentions and cohort boundaries. Biological
|
||||
advancement conserves each owner's current stock and preserves cumulative flows;
|
||||
death or representation conversion cannot spend that metadata. Manual conversion
|
||||
retains the same no-tick whole-group contract. No new biological rule or missing
|
||||
random/residual state is introduced.
|
||||
|
||||
Batch 2 is portable replay support only. Durable bundle 9's C++ prepared-commit
|
||||
validator and schema 45's SQL trace checker admit only batch 1. A new durable
|
||||
bundle/upgrade and exact history/recovery contract remain required before any
|
||||
publication, scheduling or transfer activation can use the new representation.
|
||||
|
||||
@@ -28,7 +28,8 @@ bool canonical(const PopulationPreparedCommit& prepared, const CheckpointManifes
|
||||
m.begin_tick == m.end_tick && m.batch == BatchDigests{};
|
||||
const auto trace = server::decode_population_batch(prepared.trace);
|
||||
const auto digest = sha256(prepared.trace);
|
||||
if (!trace || !digest || m.profile != CheckpointProfile::cellular_population_v1 ||
|
||||
if (!trace || !digest || trace->format_version != 1 ||
|
||||
m.profile != CheckpointProfile::cellular_population_v1 ||
|
||||
m.batch != BatchDigests{prepared.trace.size(), *digest, 0, {}})
|
||||
return false;
|
||||
const auto& h = trace->header;
|
||||
|
||||
@@ -6,14 +6,16 @@
|
||||
|
||||
namespace evo::server {
|
||||
namespace {
|
||||
constexpr std::uint64_t magic = 0x315442504F564541ULL; // "AEVOPBT1"
|
||||
constexpr std::uint64_t magic = 0x315442504F564541ULL; // "AEVOPBT1"
|
||||
constexpr std::uint64_t migration_magic = 0x325442504F564541ULL; // "AEVOPBT2"
|
||||
bool shape(const CellularPopulationBatch& batch, bool recording) {
|
||||
const auto& h = batch.header;
|
||||
const auto zero = [](const auto& bytes) {
|
||||
return std::ranges::all_of(bytes, [](auto byte) { return byte == 0; });
|
||||
};
|
||||
if (h.world == 0 || h.region == 0 || h.frame == 0 || zero(h.predecessor) ||
|
||||
zero(h.predecessor_digest) || h.end_tick < h.begin_tick)
|
||||
if ((batch.format_version != 1 && batch.format_version != 2) || h.world == 0 || h.region == 0 ||
|
||||
h.frame == 0 || zero(h.predecessor) || zero(h.predecessor_digest) ||
|
||||
h.end_tick < h.begin_tick)
|
||||
return false;
|
||||
const auto count = batch.conversion ? std::size_t{1} : batch.biology.ticks.size();
|
||||
if (count == 0 || count > max_cellular_batch_ticks ||
|
||||
@@ -117,9 +119,9 @@ unattended_population_inputs(const CellularLives& base, const CellularBatchHeade
|
||||
std::expected<std::unique_ptr<CellularPopulation>, CellularPopulationError>
|
||||
execute_population_batch(const CellularPopulation& base, CellularPopulationBatch& batch,
|
||||
bool record) {
|
||||
// Trace 1 is retained for envelope 1. A future durable migration adapter must
|
||||
// bind the new owner/envelope tuple explicitly before recording its history.
|
||||
if (base.format_version() != 1)
|
||||
// Each trace version retains its exact owner/envelope meaning. Portable trace
|
||||
// 2 is not admission to an existing durable bundle.
|
||||
if (base.format_version() != batch.format_version)
|
||||
return std::unexpected(CellularPopulationError::unsupported);
|
||||
if (!shape(batch, record) || batch.header.world != base.state().world() ||
|
||||
batch.header.region != base.state().region() ||
|
||||
@@ -192,6 +194,7 @@ prepare_population_batch(const CellularPopulation& base, CellularBatch biology)
|
||||
if (!population_interval_safe(base.state()))
|
||||
return std::unexpected(CellularPopulationError::ineligible);
|
||||
CellularPopulationBatch batch{biology.header, std::move(biology), {}, {}};
|
||||
batch.format_version = base.format_version();
|
||||
auto state = execute_population_batch(base, batch, true);
|
||||
if (!state)
|
||||
return std::unexpected(state.error());
|
||||
@@ -201,6 +204,7 @@ std::expected<CellularPopulationExecution, CellularPopulationError>
|
||||
prepare_population_conversion(const CellularPopulation& base, CellularBatchHeader header,
|
||||
CellularPopulationConversion conversion) {
|
||||
CellularPopulationBatch batch{header, {}, conversion, {}};
|
||||
batch.format_version = base.format_version();
|
||||
auto state = execute_population_batch(base, batch, true);
|
||||
if (!state)
|
||||
return std::unexpected(state.error());
|
||||
@@ -224,8 +228,8 @@ encode_population_batch(const CellularPopulationBatch& batch) {
|
||||
biological = std::move(*encoded);
|
||||
}
|
||||
std::vector<std::uint8_t> out;
|
||||
put(out, magic);
|
||||
put(out, 1, 4);
|
||||
put(out, batch.format_version == 2 ? migration_magic : magic);
|
||||
put(out, batch.format_version, 4);
|
||||
put(out, batch.conversion ? 1 : 0, 4);
|
||||
const auto& h = batch.header;
|
||||
for (const auto value : {h.world, h.region, h.frame})
|
||||
@@ -256,10 +260,14 @@ decode_population_batch(std::span<const std::uint8_t> bytes) {
|
||||
if (bytes.size() < population_batch_header_bytes || bytes.size() > max_population_batch_bytes)
|
||||
return std::unexpected(CellularPopulationError::format);
|
||||
Reader in(bytes);
|
||||
if (in.get() != magic || in.get(4) != 1)
|
||||
const auto saved_magic = in.get();
|
||||
const auto version = in.get(4);
|
||||
if (!((saved_magic == magic && version == 1) ||
|
||||
(saved_magic == migration_magic && version == 2)))
|
||||
return std::unexpected(CellularPopulationError::unsupported);
|
||||
const auto kind = in.get(4);
|
||||
CellularPopulationBatch batch;
|
||||
batch.format_version = static_cast<std::uint32_t>(version);
|
||||
auto& h = batch.header;
|
||||
h.world = in.get();
|
||||
h.region = in.get();
|
||||
|
||||
@@ -18,3 +18,9 @@ foreach(representation raw pop)
|
||||
endif()
|
||||
endforeach()
|
||||
endforeach()
|
||||
execute_process(COMMAND "${CMAKE_COMMAND}" -E compare_files
|
||||
"${prefix}.pop.continuation.expected" "${prefix}.pop.continuation.resumed"
|
||||
RESULT_VARIABLE continued)
|
||||
if(NOT continued EQUAL 0)
|
||||
message(FATAL_ERROR "Transferred population interval diverged across process replay")
|
||||
endif()
|
||||
|
||||
@@ -153,6 +153,13 @@ CellularMigrationRequest request_for(const CellularLives& source, const Cellular
|
||||
{target.region(), filled<16>(2), filled<16>(4), filled<32>(6)},
|
||||
wildlife_genome(source)};
|
||||
}
|
||||
CellularPopulationExecution continuation(const CellularPopulation& population) {
|
||||
const auto& s = population.state();
|
||||
const CellularBatchHeader header{s.world(), s.region(), s.frame(), filled<16>(9),
|
||||
filled<32>(10), s.tick(), s.tick() + 64};
|
||||
auto inputs = checked(unattended_population_inputs(s, header), "transferred interval inputs");
|
||||
return checked(prepare_population_batch(population, std::move(inputs)), "transferred interval");
|
||||
}
|
||||
void trace_contract() {
|
||||
auto source = checked(funded()->migration_upgrade_candidate(), "trace source");
|
||||
auto target = checked(destination(*source, 1)->migration_upgrade_candidate(), "trace target");
|
||||
@@ -250,6 +257,46 @@ void trace_contract() {
|
||||
require(!replay_migration_trace(*population, *peer, request, trace) &&
|
||||
!replay_migration_trace(*source, *target, request, pop_trace),
|
||||
"trace representation confused");
|
||||
for (const auto* owner : {pop_replayed.source.get(), pop_replayed.destination.get()}) {
|
||||
const auto bytes_before = owner->snapshot();
|
||||
const auto ledger_before = owner->state().boundary_ledger();
|
||||
auto interval = continuation(*owner);
|
||||
require(interval.batch.format_version == 2, "new owner recorded old trace version");
|
||||
const auto encoded =
|
||||
checked(encode_population_batch(interval.batch), "encode new interval");
|
||||
const auto decoded = checked(decode_population_batch(encoded), "decode new interval");
|
||||
auto replay = checked(replay_population_batch(*owner, decoded), "replay new interval");
|
||||
require(replay->snapshot() == interval.state->snapshot() &&
|
||||
replay->state().boundary_ledger() == ledger_before &&
|
||||
replay->state().current_total() == owner->state().current_total() &&
|
||||
owner->snapshot() == bytes_before,
|
||||
"new interval changed flows/history or failed exact replay");
|
||||
auto legacy = decoded;
|
||||
legacy.format_version = 1;
|
||||
require(!replay_population_batch(*owner, legacy), "old trace reinterpreted for owner 7");
|
||||
auto malformed = encoded;
|
||||
malformed[7] = '1';
|
||||
require(!decode_population_batch(malformed),
|
||||
"new population trace magic/version confusion");
|
||||
if (!owner->members().empty()) {
|
||||
const auto& s = owner->state();
|
||||
const CellularBatchHeader h{s.world(), s.region(), s.frame(), filled<16>(9),
|
||||
filled<32>(10), s.tick(), s.tick()};
|
||||
auto conversion = checked(
|
||||
prepare_population_conversion(*owner, h, {owner->members().front().genome, false}),
|
||||
"new manual trace");
|
||||
const auto manual =
|
||||
checked(decode_population_batch(checked(encode_population_batch(conversion.batch),
|
||||
"new manual encode")),
|
||||
"new manual decode");
|
||||
auto materialized =
|
||||
checked(replay_population_batch(*owner, manual), "new manual replay");
|
||||
require(manual.format_version == 2 &&
|
||||
materialized->state().snapshot() == s.snapshot() &&
|
||||
materialized->members().empty(),
|
||||
"new conversion changed exact biology");
|
||||
}
|
||||
}
|
||||
require(source->snapshot() == before && target->snapshot() == peer_before,
|
||||
"trace validation changed its input");
|
||||
}
|
||||
@@ -293,6 +340,10 @@ void record_restart(const std::string& prefix) {
|
||||
checked(encode_migration_trace(pop.trace), "restart pop trace"));
|
||||
write_bytes(prefix + ".pop.source.expected", pop.execution.source->snapshot());
|
||||
write_bytes(prefix + ".pop.destination.expected", pop.execution.destination->snapshot());
|
||||
auto interval = continuation(*pop.execution.destination);
|
||||
write_bytes(prefix + ".pop.continuation.trace",
|
||||
checked(encode_population_batch(interval.batch), "restart interval trace"));
|
||||
write_bytes(prefix + ".pop.continuation.expected", interval.state->snapshot());
|
||||
}
|
||||
void replay_restart(const std::string& prefix) {
|
||||
const auto raw =
|
||||
@@ -321,6 +372,12 @@ void replay_restart(const std::string& prefix) {
|
||||
"fresh population replay");
|
||||
write_bytes(prefix + ".pop.source.resumed", pop_replayed.source->snapshot());
|
||||
write_bytes(prefix + ".pop.destination.resumed", pop_replayed.destination->snapshot());
|
||||
const auto interval =
|
||||
checked(decode_population_batch(read_bytes(prefix + ".pop.continuation.trace")),
|
||||
"fresh interval trace");
|
||||
auto continued = checked(replay_population_batch(*pop_replayed.destination, interval),
|
||||
"fresh transferred continuation");
|
||||
write_bytes(prefix + ".pop.continuation.resumed", continued->snapshot());
|
||||
}
|
||||
void upgrade_and_move() {
|
||||
auto original = funded();
|
||||
|
||||
@@ -159,6 +159,17 @@ void manifests() {
|
||||
require(!restore_cellular_payload(population->snapshot(), next),
|
||||
"population mislabeled as cellular");
|
||||
auto future = checked(population->migration_upgrade_candidate(), "future portable envelope");
|
||||
const auto member = std::ranges::find_if(
|
||||
future->state().lives(), [](const auto& life) { return life.account == LifeAccount{}; });
|
||||
require(member != future->state().lives().end(), "future trace fixture lacks wildlife");
|
||||
auto future_conversion = checked(
|
||||
prepare_population_conversion(*future, header(*future, 0), {member->identity.genome, true}),
|
||||
"future portable conversion");
|
||||
require(future_conversion.batch.format_version == 2,
|
||||
"future owner recorded historical population trace");
|
||||
auto legacy_replay = replay_population_batch(*population, future_conversion.batch);
|
||||
require(!legacy_replay && legacy_replay.error() == CellularPopulationError::unsupported,
|
||||
"new trace reinterpreted for old owner");
|
||||
auto mislabeled = next;
|
||||
mislabeled.payload_bytes = future->snapshot().size();
|
||||
auto refused = restore_population_payload(future->snapshot(), mislabeled);
|
||||
|
||||
@@ -259,6 +259,25 @@ void conversions_and_retries(Fixture& f) {
|
||||
require(take(resolve_population_prepared(*f.db, prepared, 5s), "read-only resolve").status ==
|
||||
Resolution::committed,
|
||||
"commit unresolved");
|
||||
auto future_batch = conversion.batch;
|
||||
future_batch.format_version = 2;
|
||||
auto future = prepared;
|
||||
future.trace = take(encode_population_batch(future_batch), "future trace fixture");
|
||||
auto future_manifest = take(decode_manifest(future.manifest), "future manifest fixture");
|
||||
future_manifest.batch.trace_bytes = future.trace.size();
|
||||
future_manifest.batch.trace_sha256 = take(sha256(future.trace), "future trace digest");
|
||||
future.manifest = take(encode_manifest(future_manifest), "future manifest encoding");
|
||||
auto rejected_future = retry_population_prepared(*f.db, future, 5s);
|
||||
require(!rejected_future && rejected_future.error().kind == PublicationError::candidate,
|
||||
"bundle 9 prepared retry admitted population trace 2");
|
||||
require(!resolve_population_prepared(*f.db, future, 5s),
|
||||
"bundle 9 prepared resolve admitted population trace 2");
|
||||
auto rejected_sql = publish_population_checkpoint(*f.db, future.manifest, future.trace, 5s);
|
||||
require(!rejected_sql && rejected_sql.error().kind == LeaseStoreError::invalid,
|
||||
"schema 45 did not reject the unsupported trace format before commit lookup");
|
||||
require(take(load_region_checkpoint(*f.db, f.key, 5s), "head after future rejection").head ==
|
||||
f.head.head,
|
||||
"future trace changed the retained durable head");
|
||||
auto changed = prepared;
|
||||
auto changed_manifest = take(decode_manifest(changed.manifest), "conflict manifest");
|
||||
changed_manifest.payload_sha256[0] ^= 1;
|
||||
|
||||
Reference in New Issue
Block a user