Implement M7.4a checkpoint publication and verified recovery

ADR-0061 publishes environmental-v1 full checkpoints. Manifest v1 is a
fixed 280-byte record (identity, chain, lease, versions, build, zero
RNG/history declarations, durability policy, payload length/SHA-256)
that PostgreSQL parses and digests itself. Schema v3 adds region_commits
with a digest CHECK and a composite head FK, owner-managed retained
builds, fenced publish/load functions and owner-only genesis; it drops
the bare v2 head kernel and fails closed while v2 heads exist.

Linux evo::region_store adds OpenSSL SHA-256, an immutable
content-addressed filesystem object store (temp+fsync+link, no
overwrite/delete, O_NOFOLLOW exact-length verified reads), and
publication (validate -> upload -> read back -> fenced transaction ->
receipt check) and recovery (re-digest, allowlist, verify, full restore).
Portable evo::region_checkpoint holds the codec and payload validation.

Validated: Windows CTest 56/56; WSL Release, clang-tidy and ASan/UBSan
private-PostgreSQL suites; four SQL and one host mutation each caught.
M7.4 stays open for 7.4b (lineage archive codec and history binding).
No Docker/Gitea CI, remote store, process kill or deployment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-09-28 15:25:27 -04:00
co-authored by Claude Opus 5.5
parent 07116446c6
commit 5d4f59f901
32 changed files with 2253 additions and 145 deletions
@@ -0,0 +1,68 @@
#pragma once
#include "evo/persistence/object_store.hpp"
#include "evo/persistence/region_checkpoint.hpp"
#include "evo/persistence/region_lease_store.hpp"
#include <chrono>
#include <expected>
#include <memory>
#include <span>
// Linux publication and recovery of environmental-v1 checkpoints (ADR-0061).
// Order: validate candidate -> immutable upload -> verified read-back -> fenced
// manifest transaction. Nothing before the transaction has authority; a failure
// there leaves the previous head and at most an unreferenced orphan object.
namespace evo::persistence {
enum class PublicationError : std::uint8_t {
candidate, // State/manifest not a valid successor; nothing uploaded.
digest, // Digest provider failure; nothing uploaded.
upload, // Object write failed; no manifest was sent.
verify, // Read-back differs from the candidate; no manifest was sent.
rejected, // Database refused (see store); the upload is an orphan.
unknown, // Outcome unknown (see store); stay paused and reconcile the head.
receipt, // Committed receipt disagrees with the local manifest: fail closed.
};
struct PublicationFailure {
PublicationError kind{};
LeaseStoreFailure store{}; // Meaningful for rejected/unknown.
ObjectError object{}; // Meaningful for upload/verify.
};
enum class RecoveryError : std::uint8_t {
store, // Database unavailable/failed (see store).
not_provisioned, // No genesis: recovery never creates one.
manifest, // Manifest bytes disagree with the committed digest/head.
unsupported, // Profile/versions/build not in this host's retained allowlist.
missing_payload, // Referenced object absent: quarantine, no older head.
corrupt_payload, // Length/digest mismatch: quarantine, no older head.
payload, // Verified bytes fail full state validation: quarantine.
};
struct RecoveryFailure {
RecoveryError kind{};
LeaseStoreFailure store{};
};
struct RecoveredCheckpoint {
CommittedCheckpoint committed;
CheckpointManifest manifest;
std::unique_ptr<processes::EnvironmentalCommunity> state;
};
// Owner/operator connection only: uploads a tick-0 state and records genesis.
[[nodiscard]] std::expected<CommittedCheckpoint, PublicationFailure> provision_environmental_region(
Postgres& owner, const FilesystemObjectStore& objects, RegionKey key, std::uint64_t frame,
const processes::EnvironmentalCommunity& opening, Uuid commit, const Sha256Digest& build,
std::size_t restore_budget, std::chrono::milliseconds timeout);
// Publishes `state` as the successor of `predecessor` under `lease`. The caller
// must hold the committed predecessor it simulated from and a lease that may
// begin a commit (LeaseAuthority::may_begin_commit).
[[nodiscard]] std::expected<CommittedCheckpoint, PublicationFailure>
publish_environmental_checkpoint(Postgres& db, const FilesystemObjectStore& objects,
const LeaseGrant& lease, const CommittedCheckpoint& predecessor,
const processes::EnvironmentalCommunity& state, Uuid commit,
const Sha256Digest& build, std::size_t restore_budget,
std::chrono::milliseconds timeout);
// Loads and fully verifies the committed head. Readiness additionally requires a
// fresh valid lease acquired before this call (M7.7); this does not grant one.
[[nodiscard]] std::expected<RecoveredCheckpoint, RecoveryFailure>
recover_environmental_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
std::span<const Sha256Digest> retained_builds,
std::size_t restore_budget, std::chrono::milliseconds timeout);
} // namespace evo::persistence
+57
View File
@@ -0,0 +1,57 @@
#pragma once
#include "evo/persistence/region_lease.hpp"
#include <cstddef>
#include <cstdint>
#include <expected>
#include <filesystem>
#include <memory>
#include <span>
#include <vector>
// Immutable content-addressed payload store on a trusted local/mounted volume
// (ADR-0061). Linux only. One namespace directory per world/region; the object
// key is the lowercase hex SHA-256 of its bytes. There is no delete or overwrite:
// unreferenced uploads are orphans and remain until a reachability-proven GC exists.
namespace evo::persistence {
enum class ObjectError : std::uint8_t {
invalid, // Bad root, namespace or object reference; nothing done.
missing, // No object with that key.
corrupt, // Length or digest mismatch, or not a regular file: quarantine.
bounds, // Larger than the configured maximum; rejected before allocation.
io, // Write/sync/link failure; the object is absent or unverified.
};
struct VerifiedObject {
Sha256Digest sha256{};
std::uint64_t bytes{};
friend bool operator==(const VerifiedObject&, const VerifiedObject&) = default;
};
class FilesystemObjectStore {
public:
// trusted_root must be an existing real directory (not a symlink). Namespace
// directories w<world>/r<region> are created 0700 and their parents synced.
[[nodiscard]] static std::expected<std::unique_ptr<FilesystemObjectStore>, ObjectError>
open(const std::filesystem::path& trusted_root, RegionKey key, std::size_t max_object_bytes);
FilesystemObjectStore(const FilesystemObjectStore&) = delete;
FilesystemObjectStore& operator=(const FilesystemObjectStore&) = delete;
FilesystemObjectStore(FilesystemObjectStore&&) = delete;
FilesystemObjectStore& operator=(FilesystemObjectStore&&) = delete;
~FilesystemObjectStore();
// Writes a private temporary file, fsyncs it, links it to its key without
// replacing anything, then fsyncs the directory. Identical bytes already stored
// are verified and reused; a differing existing object is corrupt, never replaced.
[[nodiscard]] std::expected<VerifiedObject, ObjectError>
put(std::span<const std::uint8_t> bytes) const;
// Reads only an exact-length regular file and verifies its SHA-256.
[[nodiscard]] std::expected<std::vector<std::uint8_t>, ObjectError>
get(const VerifiedObject& object) const;
[[nodiscard]] const std::filesystem::path& directory() const noexcept {
return directory_;
}
private:
FilesystemObjectStore(std::filesystem::path directory, int descriptor, std::size_t maximum);
std::filesystem::path directory_;
int descriptor_;
std::size_t maximum_;
};
} // namespace evo::persistence
@@ -0,0 +1,89 @@
#pragma once
#include "evo/persistence/region_lease.hpp"
#include "evo/processes/environmental_community.hpp"
#include <cstddef>
#include <cstdint>
#include <expected>
#include <memory>
#include <span>
#include <vector>
// Region checkpoint manifest v1 and environmental-v1 payload validation (ADR-0061,
// schemas/region-commit.md). Portable and storage-free: digests are computed and
// objects are stored by the Linux adapters; this only encodes and validates.
namespace evo::persistence {
// Encoded as u32 on the wire; values above 255 decode as unknown (0), never truncated.
enum class CheckpointProfile : std::uint8_t { unknown = 0, environmental_v1 = 1 };
// Declared durability, verified at deployment. Development: one volume, no host-loss claim.
enum class DurabilityPolicy : std::uint8_t { unknown = 0, single_volume_development = 1 };
// Exact versions of every nested format/rule the payload depends on. The interval
// adapter carries no payload state but its version is still pinned.
struct VersionBundle {
std::uint32_t rules{2}, envelope{2}, resources{1}, fields{1}, terrain{1}, coordinate{1},
generator{1}, material{1}, interval_adapter{1};
friend bool operator==(const VersionBundle&, const VersionBundle&) = default;
};
inline constexpr VersionBundle environmental_v1_versions{};
struct CheckpointManifest {
CheckpointProfile profile{CheckpointProfile::environmental_v1};
RegionKey key{};
std::uint64_t frame{};
Uuid commit{};
std::uint64_t sequence{};
Uuid predecessor_commit{}; // Zero only at genesis.
Sha256Digest predecessor_digest{}; // SHA-256 of the predecessor manifest bytes.
std::uint64_t begin_tick{}, end_tick{};
Uuid holder{}; // Zero only at genesis (provisioning, not a lease).
std::uint64_t epoch{};
std::uint64_t tick_microseconds{};
VersionBundle versions{};
Sha256Digest build{}; // Retained immutable build artifact identity.
std::uint32_t rng_streams{}; // environmental-v1 rules use no randomness: 0.
std::uint32_t history_profile{}; // No lineage/genome state in environmental-v1: 0.
DurabilityPolicy durability{DurabilityPolicy::single_volume_development};
std::uint64_t payload_bytes{};
Sha256Digest payload_sha256{};
friend bool operator==(const CheckpointManifest&, const CheckpointManifest&) = default;
};
inline constexpr std::size_t checkpoint_manifest_bytes = 280;
inline constexpr std::size_t max_environmental_payload_bytes = std::size_t{1024} * 1024;
inline constexpr std::uint64_t max_checkpoint_ticks = 64;
enum class CheckpointError : std::uint8_t {
format, // Malformed or internally inconsistent manifest.
unsupported, // Unknown profile, version, RNG/history or durability declaration.
chain, // Not a valid successor of the committed predecessor.
payload, // Payload bytes fail full envelope validation or identity/tick checks.
};
// Structural validation of every field; versions/profile are checked separately
// so recovery can distinguish corruption from an unsupported (retained) format.
[[nodiscard]] std::expected<std::vector<std::uint8_t>, CheckpointError>
encode_manifest(const CheckpointManifest& manifest);
[[nodiscard]] std::expected<CheckpointManifest, CheckpointError>
decode_manifest(std::span<const std::uint8_t> bytes);
// Profile, exact versions, zero RNG/history declarations and durability policy.
[[nodiscard]] bool supported(const CheckpointManifest& manifest) noexcept;
// Genesis for operator provisioning only; recovery never creates one.
[[nodiscard]] std::expected<CheckpointManifest, CheckpointError>
genesis_manifest(RegionKey key, std::uint64_t frame, Uuid commit, std::uint64_t tick_microseconds,
const Sha256Digest& build, std::uint64_t payload_bytes,
const Sha256Digest& payload_sha256);
// Next link from the committed predecessor and its manifest digest under a lease.
[[nodiscard]] std::expected<CheckpointManifest, CheckpointError>
successor_manifest(const CheckpointManifest& predecessor, const Sha256Digest& predecessor_digest,
Uuid commit, const LeaseGrant& lease, std::uint64_t end_tick,
const Sha256Digest& build, std::uint64_t payload_bytes,
const Sha256Digest& payload_sha256);
// Fully restores payload bytes (envelope checksum, bounds and replay audit), then
// requires manifest identity, end tick, tick quantum, envelope format 2 and a
// byte-identical canonical re-encoding. The caller verifies length and SHA-256.
[[nodiscard]] std::expected<std::unique_ptr<processes::EnvironmentalCommunity>, CheckpointError>
restore_environmental_payload(std::span<const std::uint8_t> payload,
const CheckpointManifest& manifest, std::size_t budget);
} // namespace evo::persistence
+1
View File
@@ -13,6 +13,7 @@ using Sha256Digest = std::array<std::uint8_t, 32>;
struct RegionKey {
std::uint64_t world{};
std::uint64_t region{};
friend bool operator==(const RegionKey&, const RegionKey&) = default;
};
struct RegionHead {
std::uint64_t frame{};
+16 -9
View File
@@ -4,6 +4,8 @@
#include <chrono>
#include <cstdint>
#include <expected>
#include <span>
#include <vector>
// Linux-only calls to the fenced database functions of migration v2 (ADR-0060).
// Each call is one statement on a caller-owned connection; outside an explicit
@@ -15,6 +17,8 @@ enum class LeaseStoreError : std::uint8_t {
head_mismatch, // Expected predecessor is not the committed head.
not_provisioned, // No genesis; runtime can never create one.
exhausted, // Epoch or commit sequence would wrap: fail closed.
unsupported, // Profile/version/build not admitted, or rules/quantum changed.
commit_conflict, // Commit ID already used by another manifest (resolve at M7.5).
invalid, // Rejected arguments; nothing changed.
store, // Other SQL/configuration failure; nothing committed.
unknown, // May have executed; reconnect and reconcile before reuse.
@@ -23,10 +27,10 @@ struct LeaseStoreFailure {
LeaseStoreError kind{};
PgFailure cause{};
};
struct HeadAdvance {
Uuid commit{};
Sha256Digest digest{};
std::uint64_t end_tick{};
// Committed head and the exact manifest bytes its digest covers.
struct CommittedCheckpoint {
RegionHead head{};
std::vector<std::uint8_t> manifest;
};
[[nodiscard]] std::expected<LeaseGrant, LeaseStoreFailure>
@@ -41,10 +45,13 @@ release_region_lease(Postgres& db, RegionKey key, const LeaseGrant& lease,
std::chrono::milliseconds timeout);
[[nodiscard]] std::expected<RegionHead, LeaseStoreFailure>
load_region_head(Postgres& db, RegionKey key, std::chrono::milliseconds timeout);
// Locks the lease then the head; requires unexpired holder/epoch and the exact
// expected predecessor. head_mismatch after unknown: load the head and compare.
[[nodiscard]] std::expected<CommittedCheckpoint, LeaseStoreFailure>
load_region_checkpoint(Postgres& db, RegionKey key, std::chrono::milliseconds timeout);
// Canonical manifest v1 bytes; the database parses identity, lease and chain from
// them, locks the lease then the head and requires the exact committed predecessor.
// The payload must already be verified in the object store. head_mismatch after
// unknown: load the head and compare its commit ID (full resolution is M7.5).
[[nodiscard]] std::expected<RegionHead, LeaseStoreFailure>
advance_region_head(Postgres& db, RegionKey key, const LeaseGrant& lease,
const RegionHead& expected, const HeadAdvance& next,
std::chrono::milliseconds timeout);
publish_region_checkpoint(Postgres& db, std::span<const std::uint8_t> manifest,
std::chrono::milliseconds timeout);
} // namespace evo::persistence
+13
View File
@@ -0,0 +1,13 @@
#pragma once
#include "evo/persistence/region_lease.hpp"
#include <expected>
#include <span>
// SHA-256 from the system OpenSSL libcrypto EVP interface (ADR-0061). Linux
// persistence adapters only; no project cryptography. Content identity, not
// authentication.
namespace evo::persistence {
enum class DigestError : std::uint8_t { provider };
[[nodiscard]] std::expected<Sha256Digest, DigestError>
sha256(std::span<const std::uint8_t> bytes) noexcept;
} // namespace evo::persistence