Add cellular checkpoint publication adapter
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (checks) (push) Failing after 39s
Foundation / checks (postgres) (push) Canceled after 0s
Foundation / checks (tsan) (push) Canceled after 0s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (client) (push) Canceled after 1m36s
Foundation / checks (network) (push) Canceled after 59s
Foundation / checks (ecs) (push) Canceled after 1m38s

This commit is contained in:
dingo
2026-10-10 23:21:40 -04:00
parent d057e5ba04
commit ad6e53c2e9
8 changed files with 228 additions and 1 deletions
+1
View File
@@ -231,6 +231,7 @@ if(EVO_BUILD_POSTGRES)
find_package(OpenSSL 3 REQUIRED COMPONENTS Crypto)
add_library(evo_region_store STATIC src/persistence/sha256.cpp src/persistence/object_store.cpp
src/persistence/checkpoint_store.cpp src/persistence/primitive_checkpoint_store.cpp
src/persistence/cellular_checkpoint_store.cpp
src/persistence/region_clone.cpp)
add_library(evo::region_store ALIAS evo_region_store)
target_link_libraries(evo_region_store PUBLIC evo::postgres evo::region_checkpoint
+8
View File
@@ -235,6 +235,14 @@ scoped change before starting the next segment.
per-tick replay intention. Tests cover successful matching and changed-input
refusal. Durable action result reconciliation and full checkpoint recovery
remain open; CI execution is pending.
- 11.3d-6e: a dedicated persistence adapter now replays a cellular batch from
the verified predecessor, binds durable actions to those replay inputs, checks
byte-identical candidate state, uploads and reads back payload/trace objects,
and publishes through the isolated profile-4 SQL wrapper. Prepared retry and
ambiguity resolution retain the exact trace and action bytes. The portable
manifest validator now applies ADR-0113's variable cellular trace bounds,
matching migration 21. Result-row reconciliation and full restart recovery,
host integration, and CI execution remain open.
## Exit boundary
+15
View File
@@ -6255,3 +6255,18 @@ tick. Lifecycle tests cover exact match and changed-input refusal.
`git diff --check` passes. C++ compilation and Linux sanitizer/CI tests are still
pending; this Windows environment has no C++ compiler in `PATH`.
## 2026-10-10 — M11.3d-6e cellular checkpoint publication adapter (CI pending)
Added profile-4 batch publication with verified predecessor replay, exact state
comparison, action-to-trace binding, verified immutable object uploads, and the
isolated SQL publisher. Prepared retry and ambiguous resolution retain and
validate the exact manifest, trace, and action block. The manifest codec now
enforces the variable trace-size range from ADR-0113 and migration 21; focused
tests cover its lower and upper rejection boundaries.
`git diff --check` and a GCC 13.2 C++23 `-fsyntax-only` pass for the new adapter,
manifest validator, and boundary tests pass. Full CMake build/CTest and PostgreSQL
integration are pending CI. The documented debug preset currently selects the
cached MSVC 19.44 compiler and fails CMake's C++23 feature check; this environment
also has no PostgreSQL runtime on `PATH`.
@@ -4,6 +4,7 @@
#include "evo/persistence/region_batch.hpp"
#include "evo/persistence/region_checkpoint.hpp"
#include "evo/persistence/region_lease_store.hpp"
#include "evo/server/lifecycle/cellular_batch.hpp"
#include <chrono>
#include <expected>
#include <functional>
@@ -100,6 +101,20 @@ struct RecoveredPrimitiveCheckpoint {
std::unique_ptr<server::PrimitiveLives> state;
std::vector<PrimitiveLifeBinding> bindings;
};
struct CellularPreparedCommit {
RegionKey key{};
Uuid commit{};
std::vector<std::uint8_t> manifest, trace, actions;
};
struct CellularPublicationFailure {
PublicationError kind{};
LeaseStoreFailure store{};
ObjectError object{};
CellularPreparedCommit prepared;
};
struct PublishedCellularBatch {
CommittedCheckpoint committed;
};
// Owner/operator connection only: uploads a tick-0 state and records genesis.
[[nodiscard]] std::expected<CommittedCheckpoint, PublicationFailure> provision_environmental_region(
@@ -174,6 +189,22 @@ recover_primitive_checkpoint(Postgres& db, const FilesystemObjectStore& objects,
std::span<const Sha256Digest> retained_builds,
std::chrono::milliseconds timeout);
// Profile 4 publication replays the bounded cellular trace from the verified
// predecessor and requires exact candidate bytes and action-to-life agreement.
[[nodiscard]] std::expected<PublishedCellularBatch, CellularPublicationFailure>
publish_cellular_batch(Postgres& db, const FilesystemObjectStore& objects,
const LeaseGrant& lease, const CommittedCheckpoint& predecessor,
const server::CellularBatch& batch, const server::CellularLives& candidate,
Uuid commit, const Sha256Digest& build,
std::chrono::milliseconds timeout, const PublishHooks& hooks = {},
std::span<const server::CellularActionRecord> actions = {});
[[nodiscard]] std::expected<CommittedCheckpoint, CellularPublicationFailure>
retry_cellular_prepared(Postgres& db, const CellularPreparedCommit& prepared,
std::chrono::milliseconds timeout);
[[nodiscard]] std::expected<ResolvedCommit, LeaseStoreFailure>
resolve_cellular_prepared(Postgres& db, const CellularPreparedCommit& prepared,
std::chrono::milliseconds timeout);
// After an `unknown` publication: resolve exactly the prepared bytes. A committed
// result yields the final batch; absent may be retried with the same prepared bytes
// only while the lease is still valid; conflict fails closed.
@@ -4,6 +4,7 @@
#include "evo/processes/environmental_community.hpp"
#include "evo/processes/lineage_region.hpp"
#include "evo/server/lifecycle/cellular_lives.hpp"
#include "evo/server/lifecycle/cellular_batch.hpp"
#include "evo/server/lifecycle/primitive_batch.hpp"
#include <cstddef>
#include <cstdint>
@@ -0,0 +1,153 @@
#include "evo/persistence/checkpoint_store.hpp"
#include "evo/persistence/sha256.hpp"
#include <algorithm>
#include <utility>
namespace evo::persistence {
namespace {
using std::chrono::milliseconds;
CellularPublicationFailure failure(PublicationError kind, ObjectError object = {},
LeaseStoreFailure store = {},
CellularPreparedCommit prepared = {}) {
return {kind, store, object, std::move(prepared)};
}
std::expected<void, CellularPublicationFailure>
put_verified(const FilesystemObjectStore& objects, std::span<const std::uint8_t> bytes,
VerifiedObject expected) {
auto stored = objects.put(bytes);
if (!stored)
return std::unexpected{failure(PublicationError::upload, stored.error())};
auto readback = objects.get(*stored);
if (!readback || *stored != expected || !std::ranges::equal(*readback, bytes))
return std::unexpected{failure(PublicationError::verify,
readback ? ObjectError::corrupt : readback.error())};
return {};
}
std::expected<CommittedCheckpoint, CellularPublicationFailure>
confirm(const RegionHead& head, const CellularPreparedCommit& prepared,
const CheckpointManifest& manifest) {
const auto digest = sha256(prepared.manifest);
if (!digest || head.digest != *digest || head.sequence != manifest.sequence ||
head.commit != manifest.commit || head.end_tick != manifest.end_tick ||
head.frame != manifest.frame)
return std::unexpected{failure(PublicationError::receipt)};
return CommittedCheckpoint{head, prepared.manifest};
}
bool prepared_is_canonical(const CellularPreparedCommit& prepared,
const CheckpointManifest& manifest,
const server::CellularBatch& trace) {
const auto decoded_actions = server::decode_cellular_action_block(
prepared.actions, manifest.begin_tick, manifest.end_tick);
const auto encoded_actions = decoded_actions
? server::encode_cellular_action_block(*decoded_actions, manifest.begin_tick,
manifest.end_tick)
: std::expected<std::vector<std::uint8_t>, server::CellularActionBlockError>{
std::unexpected{server::CellularActionBlockError::format}};
const auto digest = sha256(prepared.trace);
return manifest.profile == CheckpointProfile::cellular_lives_v1 &&
manifest.key == prepared.key && manifest.commit == prepared.commit &&
manifest.sequence > 0 && manifest.batch.trace_bytes == prepared.trace.size() &&
digest && *digest == manifest.batch.trace_sha256 &&
manifest.batch.decision_count == 0 &&
manifest.batch.decisions_sha256 == Sha256Digest{} && trace.header.world == prepared.key.world &&
trace.header.region == prepared.key.region && trace.header.frame == manifest.frame &&
trace.header.predecessor == manifest.predecessor_commit &&
trace.header.predecessor_digest == manifest.predecessor_digest &&
trace.header.begin_tick == manifest.begin_tick && trace.header.end_tick == manifest.end_tick &&
encoded_actions && *encoded_actions == prepared.actions;
}
} // namespace
std::expected<CommittedCheckpoint, CellularPublicationFailure>
retry_cellular_prepared(Postgres& db, const CellularPreparedCommit& prepared,
milliseconds timeout) {
auto manifest = decode_manifest(prepared.manifest);
auto trace = server::decode_cellular_batch(prepared.trace);
if (!manifest || !trace || !prepared_is_canonical(prepared, *manifest, *trace))
return std::unexpected{failure(PublicationError::candidate)};
auto result = publish_cellular_checkpoint(db, prepared.manifest, {}, prepared.actions, timeout);
if (!result) {
return std::unexpected{failure(
result.error().kind == LeaseStoreError::unknown ? PublicationError::unknown
: PublicationError::rejected,
{}, result.error(), prepared)};
}
return confirm(*result, prepared, *manifest);
}
std::expected<ResolvedCommit, LeaseStoreFailure>
resolve_cellular_prepared(Postgres& db, const CellularPreparedCommit& prepared,
milliseconds timeout) {
auto manifest = decode_manifest(prepared.manifest);
auto trace = server::decode_cellular_batch(prepared.trace);
if (!manifest || !trace || !prepared_is_canonical(prepared, *manifest, *trace))
return std::unexpected{LeaseStoreFailure{LeaseStoreError::invalid, {}}};
return resolve_cellular_region_commit(db, prepared.key, prepared.commit, prepared.manifest,
{}, prepared.actions, timeout);
}
std::expected<PublishedCellularBatch, CellularPublicationFailure>
publish_cellular_batch(Postgres& db, const FilesystemObjectStore& objects,
const LeaseGrant& lease, const CommittedCheckpoint& predecessor,
const server::CellularBatch& batch, const server::CellularLives& candidate,
Uuid commit, const Sha256Digest& build, milliseconds timeout,
const PublishHooks& hooks,
std::span<const server::CellularActionRecord> actions) {
const auto previous = decode_manifest(predecessor.manifest);
const auto predecessor_digest = sha256(predecessor.manifest);
const auto trace = server::encode_cellular_batch(batch);
const auto action_block = server::encode_cellular_action_block(
actions, batch.header.begin_tick, batch.header.end_tick);
if (!previous || !predecessor_digest || !trace || !action_block ||
previous->profile != CheckpointProfile::cellular_lives_v1 ||
predecessor.head.digest != *predecessor_digest ||
previous->sequence != predecessor.head.sequence || previous->commit != predecessor.head.commit ||
previous->end_tick != predecessor.head.end_tick || batch.header.world != previous->key.world ||
batch.header.region != previous->key.region || batch.header.frame != previous->frame ||
batch.header.predecessor != previous->commit ||
batch.header.predecessor_digest != *predecessor_digest ||
batch.header.begin_tick != previous->end_tick || candidate.world() != previous->key.world ||
candidate.region() != previous->key.region || candidate.frame() != previous->frame ||
candidate.tick() != batch.header.end_tick)
return std::unexpected{failure(PublicationError::candidate)};
auto predecessor_bytes = objects.get({previous->payload_sha256, previous->payload_bytes});
if (!predecessor_bytes)
return std::unexpected{failure(PublicationError::predecessor, predecessor_bytes.error())};
auto base = restore_cellular_payload(*predecessor_bytes, *previous);
if (!base)
return std::unexpected{failure(PublicationError::predecessor)};
auto replay = server::replay_cellular_batch(**base, batch);
const auto payload = candidate.snapshot();
if (!replay || (*replay)->snapshot() != payload ||
!server::match_cellular_actions_to_trace(**base, batch, actions))
return std::unexpected{failure(PublicationError::candidate)};
const auto payload_digest = sha256(payload);
const auto trace_digest = sha256(*trace);
if (!payload_digest || !trace_digest)
return std::unexpected{failure(PublicationError::digest)};
auto manifest = successor_manifest(*previous, *predecessor_digest, commit, lease,
batch.header.end_tick, build, payload.size(), *payload_digest,
{trace->size(), *trace_digest, 0, {}});
auto manifest_bytes = manifest ? encode_manifest(*manifest)
: std::expected<std::vector<std::uint8_t>, CheckpointError>{
std::unexpected{CheckpointError::format}};
if (!manifest || !manifest_bytes || !restore_cellular_payload(payload, *manifest))
return std::unexpected{failure(PublicationError::candidate)};
if (auto stored = put_verified(objects, payload, {*payload_digest, payload.size()}); !stored)
return std::unexpected{stored.error()};
if (auto stored = put_verified(objects, *trace, {*trace_digest, trace->size()}); !stored)
return std::unexpected{stored.error()};
CellularPreparedCommit prepared{previous->key, commit, *manifest_bytes, *trace, *action_block};
if (hooks.before_transaction)
hooks.before_transaction();
auto committed = retry_cellular_prepared(db, prepared, timeout);
if (!committed)
return std::unexpected{committed.error()};
return PublishedCellularBatch{std::move(*committed)};
}
} // namespace evo::persistence
+10 -1
View File
@@ -93,13 +93,22 @@ bool structural(const CheckpointManifest& m) noexcept {
m.end_tick - m.begin_tick <= max_checkpoint_ticks;
if (m.manifest_version == 1)
return link && no_batch;
const auto ticks = m.end_tick - m.begin_tick;
const auto cellular_min = server::cellular_batch_header_bytes +
ticks * server::cellular_batch_tick_header_bytes;
const auto cellular_max = server::cellular_batch_header_bytes +
ticks * (server::cellular_batch_tick_header_bytes +
biology::max_cellular_tick_lives *
server::cellular_batch_intent_bytes);
const bool trace_shape = primitive
? b.trace_bytes >= server::primitive_batch_header_bytes &&
b.trace_bytes <= server::max_primitive_batch_bytes &&
(b.trace_bytes - server::primitive_batch_header_bytes) %
server::primitive_birth_bytes ==
0
: b.trace_bytes == trace_bytes_for(m.end_tick - m.begin_tick);
: cellular ? b.trace_bytes >= cellular_min &&
b.trace_bytes <= cellular_max
: b.trace_bytes == trace_bytes_for(ticks);
return link && trace_shape && !zero(b.trace_sha256) &&
b.decision_count <= max_batch_decisions &&
(b.decision_count == 0) == zero(b.decisions_sha256);
+9
View File
@@ -430,6 +430,15 @@ void cellular_profile() {
filled<32>(0xA6), batch_for(1, 1));
CHECK(successor && supported(*successor) &&
successor->cellular_versions == cellular_v1_versions);
auto short_cellular_trace = *successor;
short_cellular_trace.batch.trace_bytes =
cellular_batch_header_bytes + cellular_batch_tick_header_bytes - 1;
CHECK(!encode_manifest(short_cellular_trace));
auto large_cellular_trace = *successor;
large_cellular_trace.batch.trace_bytes = cellular_batch_header_bytes +
cellular_batch_tick_header_bytes +
max_cellular_tick_lives * cellular_batch_intent_bytes + 1;
CHECK(!encode_manifest(large_cellular_trace));
const auto clone = clone_manifest(*manifest, filled<32>(0xA3), {8, 3}, filled<16>(0x03),
payload.size(), filled<32>(0xA4), {});
CHECK(!clone && clone.error() == CheckpointError::unsupported);