M12.6: derive replayed census history from immutable commits
Foundation / checks (deploy) (push) Successful in 6s
Foundation / checks (ecs) (push) Successful in 2m25s
Foundation / checks (network) (push) Successful in 20m50s
Foundation / checks (client) (push) Successful in 22m7s
Foundation / checks (tsan) (push) Successful in 24m42s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (postgres) (push) Canceled after 35m33s
Foundation / checks (checks) (push) Canceled after 35m43s
Foundation / checks (deploy) (push) Successful in 6s
Foundation / checks (ecs) (push) Successful in 2m25s
Foundation / checks (network) (push) Successful in 20m50s
Foundation / checks (client) (push) Successful in 22m7s
Foundation / checks (tsan) (push) Successful in 24m42s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (postgres) (push) Canceled after 35m33s
Foundation / checks (checks) (push) Canceled after 35m43s
This commit is contained in:
@@ -256,6 +256,7 @@ if(EVO_BUILD_POSTGRES)
|
||||
add_library(evo_region_store STATIC src/persistence/sha256.cpp src/persistence/object_store.cpp
|
||||
src/persistence/checkpoint_store.cpp src/persistence/primitive_checkpoint_store.cpp
|
||||
src/persistence/cellular_checkpoint_store.cpp src/persistence/population_checkpoint_store.cpp
|
||||
src/persistence/cellular_history_store.cpp
|
||||
src/persistence/region_clone.cpp src/persistence/cellular_migration_store.cpp)
|
||||
add_library(evo::region_store ALIAS evo_region_store)
|
||||
target_link_libraries(evo_region_store PUBLIC evo::postgres evo::region_checkpoint
|
||||
|
||||
@@ -132,7 +132,10 @@ process fixtures remain separate from organism biology.
|
||||
[Regional cellular census](schemas/ecology/regional-cellular-census-v1.md) provides
|
||||
read-only local resident/presence/cohort and exact/descendant observations from
|
||||
current cellular owners. Retained departed history is excluded from owned stock.
|
||||
Durable world history, complete coverage and terminal extinction remain later gates.
|
||||
[Committed regional history](schemas/ecology/committed-cellular-history-v1.md) now
|
||||
derives bounded, fully replayed census pages from the existing atomic biological
|
||||
journal. Complete world/branch coverage, client queries and terminal extinction
|
||||
remain later gates.
|
||||
`apps/evo-sim` owns CLI/file I/O
|
||||
and reporting. `tests` holds contract,
|
||||
fuzz and measurement programs. `src/server/lifecycle` owns funded primitive
|
||||
|
||||
@@ -1,5 +1,23 @@
|
||||
# Project status and next steps
|
||||
|
||||
## M12.6b replayed census history from committed cuts - 2026-10-18
|
||||
|
||||
Regional census history now reads exact retained biological commits as canonical
|
||||
durable events. Pages are bounded to eight commits and pinned to a fixed regional
|
||||
cut. Shared full recovery verifies raw/cohort payloads, traces, transitions and
|
||||
paired migration evidence before exposing census rows. Idle connections prevent
|
||||
a writer's own pending changes from appearing durable; failed evidence never
|
||||
becomes a partial page or zero population. No duplicate census write is added.
|
||||
|
||||
Native Release passes **79/79**; private GCC history qualification, ASan/UBSan
|
||||
history/TLS/storage checks and Clang analysis/format checks pass.
|
||||
Complete private persistence/process/backup/DR and SQL qualification passes in
|
||||
**546 s**, including the new history matrix and existing fault tests.
|
||||
[Evidence and limits](engineering/VALIDATION.md) retain world/branch reconciliation,
|
||||
cross-region/public client access and deployment-host query costs. This regional
|
||||
watermark cannot authorize terminal extinction. Linked ecology, maturation and
|
||||
later integration/soak gates remain open; no deployment or World0 action.
|
||||
|
||||
## M12.6a bounded regional census and history queries - 2026-10-18
|
||||
|
||||
The portable census now distinguishes resident individuals/cohorts and
|
||||
|
||||
@@ -61,3 +61,8 @@ Durable append-only census/history with atomic commit binding, bounded backend
|
||||
queries, branch/epoch/watermark reconciliation and client presentation remain
|
||||
subsequent M12.6/12.7 work. Linked ecological resources and the rules-8 maturation
|
||||
gap remain open independently of this observational primitive.
|
||||
|
||||
Follow-on ADR-0151 implements regional commit-bound history reads using the
|
||||
existing immutable biological journal as canonical events, without a second
|
||||
census-write window. Full world/branch reconciliation and client presentation
|
||||
remain separate from both primitives.
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# ADR-0151: Census history from retained biological commits
|
||||
|
||||
## Status
|
||||
|
||||
Accepted for scoped M12.6b implementation, 2026-10-18. This selects the existing
|
||||
append-only biological commit journal as the durable source for regional census
|
||||
history. No new SQL schema, biological format/rule or global extinction policy.
|
||||
|
||||
## Context and decision
|
||||
|
||||
ADR-0150's census is a detached observation. Publishing an independent mutable
|
||||
summary after the biological transaction would create a second-write failure
|
||||
window and a second demographic truth. Every accepted biological publication
|
||||
already retains its exact manifest and complete payload in the immutable regional
|
||||
journal, atomically with its head. Paired migration records both endpoints in one
|
||||
transaction; uncommitted uploads and destination ghosts are not journal entries.
|
||||
|
||||
Use those committed cuts as canonical history events. Derive census projections
|
||||
on read from their verified payloads and existing semantic replay evidence, rather
|
||||
than storing duplicate census rows. A commit remains the event identity and its
|
||||
digest binds the complete census source. There is no extra post-commit write,
|
||||
backfill that invents observations, or new database write per simulation tick.
|
||||
|
||||
Add explicit historical raw/population recovery entry points that load an exact
|
||||
journal sequence and run the same manifest/build, strong object digest, complete
|
||||
restore, batch/no-tick/paired replay and outcome checks as current-head recovery.
|
||||
Historical recovery is read-only; it never changes the selected head, resumes an
|
||||
older owner or grants authority. Existing hosts continue to recover only the head.
|
||||
Replay validates each selected cut's complete batch/no-tick operation from retained
|
||||
predecessor checkpoints; it does not re-execute the entire world from genesis.
|
||||
Evidence loaders retain their current regional-lease requirements; reading a
|
||||
historical checkpoint is not proof of current ownership or world completeness.
|
||||
|
||||
Add bounded forward census pages, at most eight commits, pinned to a caller's
|
||||
exact regional watermark (frame, sequence, commit ID, manifest digest and tick).
|
||||
Validate that watermark against the immutable journal before and after reading.
|
||||
Append-only head advancement cannot move the requested upper cut. Advance by
|
||||
sequence, not tick: no-tick profile/format/migration commits remain distinct.
|
||||
Every returned entry retains its exact committed manifest and detached census.
|
||||
Require an idle connection at entry; reject all caller-owned transactions so a
|
||||
publishing connection cannot present its own uncommitted writes as durable cuts.
|
||||
The libpq adapter exposes read-only transaction status for this guard. Diagnostic
|
||||
historical recovery retains the existing caller-owned transaction semantics.
|
||||
Missing, unsupported, corrupt, unreplayable or foreign evidence fails the entire
|
||||
page; never skip a cut or return a zero census for unavailable history.
|
||||
|
||||
All point lookups use the existing unique regional sequence index. Payload/state
|
||||
work is one cut at a time, with at most eight bounded census projections retained.
|
||||
Timeouts retain existing per-I/O semantics, not a page-wide service latency promise.
|
||||
This is a backend read primitive, not a new endpoint or permission to expose it
|
||||
directly to clients.
|
||||
|
||||
## Alternatives, costs and qualification
|
||||
|
||||
A separate mutable census table can disagree with the immutable biological source.
|
||||
Adding summary bytes to every publisher would version the same redundant truth
|
||||
through genesis, ordinary batches, profile/version transitions and paired commits.
|
||||
Retained exact checkpoints already provide atomic durable history, at the cost of
|
||||
object reads and semantic replay on query. No performance/capacity claim follows;
|
||||
a later cache/index must be derivable and preserve these evidence checks.
|
||||
|
||||
Require stable pagination after new publication, sequence-zero and no-tick cuts,
|
||||
raw/cohort restoration, historical migration ownership, exact retries without
|
||||
extra events, current lease replacement, missing/corrupt/false replay evidence,
|
||||
wrong cut/region/frame/build, bounded requests and unchanged heads/stock. Use
|
||||
private PostgreSQL, sanitizer, analysis and complete persistence regression.
|
||||
|
||||
This primitive is scoped to the connected database's regional history/frame.
|
||||
Its watermark is not a complete world watermark or an external recovery-branch
|
||||
identity. Cross-database/clone/rewind reconciliation, world region-set coverage,
|
||||
client presentation, actual multi-day ecology and terminal extinction remain
|
||||
M12.6/12.7 and later gates. No live database, World0 or deployment action.
|
||||
@@ -1,5 +1,7 @@
|
||||
# Architecture decision records
|
||||
|
||||
- [ADR-0151: Committed cellular census history](0151-committed-cellular-census-history.md) - derives bounded, exact-cut census pages from the existing atomic immutable biological journal, with full historical replay; no duplicate census writes or world-extinction authority.
|
||||
|
||||
- [ADR-0150: Regional cellular census projection](0150-regional-cellular-census-projection.md) - bounded local resident/representation/presence census and exact/descendant queries; retained nonresident history never counts as owned stock or world extinction.
|
||||
|
||||
- [ADR-0149: Atomic paired biological migration publication](0149-atomic-cellular-migration-publication.md) - schema 49 binds both current fences, heads and exact candidates in one immutable receipt/transaction; recovery replays the pair and hosts verify the changed head before readiness.
|
||||
|
||||
@@ -1,5 +1,15 @@
|
||||
# Persistence and region authority
|
||||
|
||||
M12.6b derives regional census history from the existing immutable biological
|
||||
commit journal. A commit and its exact payload are the canonical event, already
|
||||
atomic with the head; no duplicate census-write window or new SQL table is added.
|
||||
Explicit historical recovery reuses the full current-head replay checks. Pages
|
||||
are bounded to eight consecutive commits and pinned to an exact regional cut;
|
||||
unavailable/false evidence fails the entire page. Idle connections prevent a
|
||||
writer's uncommitted cuts from appearing durable. See
|
||||
[ADR-0151](../adr/0151-committed-cellular-census-history.md). This is regional
|
||||
history, not branch/world completeness, a client endpoint or extinction authority.
|
||||
|
||||
M12.4f defines cellular bundle 10 for owner-7 boundary accounting, with an explicit
|
||||
same-profile, no-tick bundle-9 upgrade. M12.4g/schema 46 admits it through the
|
||||
existing immutable version-upgrade journal and lease/head fence. Recovery
|
||||
|
||||
@@ -10,6 +10,10 @@ It reports local resident stock by current genome and exact/descendant scope,
|
||||
including suspended players; retained nonresident records carry no owned stock.
|
||||
This detached observation has no commit/branch binding or world coverage and
|
||||
does not implement the global evidence envelope or terminal extinction below.
|
||||
M12.6b's [committed regional history](../../schemas/ecology/committed-cellular-history-v1.md)
|
||||
binds those projections to exact retained biological commits with full replay.
|
||||
The commit journal supplies atomic durable events without a second census write.
|
||||
Its fixed regional cut still supplies no complete world or external branch proof.
|
||||
|
||||
## Identity and classification
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ Commit and push each validated scoped segment separately.
|
||||
| 12.3 | Exact population profile/trace/store and opt-in host integration implemented (12.3b) | ADR-0142, schema 44; exact publication/recovery, mutation race guards, ambiguous replies and process kills. Executable/network population fault drills and target-host qualification remain open. |
|
||||
| 12.4 | Owned intervals and durable paired migration/fault qualification implemented in scoped segments through 12.4l | ADR-0143–0149/schema 49; bounded admitted work, source freeze, staged ghost, atomic pair and exact recovery, local TLS and storage faults. Multi-host/orchestrator, linked ecology and integrated stock audits remain open. |
|
||||
| 12.5 | Blocked on 12.2-12.4 | Finite M6 process fixtures do not qualify coupled persistent cellular/cohort atmosphere/water budgets. |
|
||||
| 12.6 | Local archive plus regional resident/presence/cohort census and exact/descendant queries implemented (12.6a) | ADR-0150; local detached observations exclude departed history from stock. Commit/branch binding, atomic durable events and world-level query/client path remain required. |
|
||||
| 12.6 | Regional census/lineage queries and replayed commit-bound history pages implemented (12.6a/b) | ADR-0150/0151; existing atomic biological commits are the durable events. Fixed regional watermarks and full replay bind observations; world/branch context, cross-region query/index costs and client path remain required. |
|
||||
| 12.7 | Evidence policy defined; reconciler open | Complete durable world watermark and every viable/in-flight store remain required. |
|
||||
| 12.8 | Not qualified | Actual multi-day integrated ecology/restart/clone evidence remains required. |
|
||||
| 12.9 | Not accepted | Returning-player product evidence and expansion review remain required; World0 stays separately blocked. |
|
||||
|
||||
@@ -27,7 +27,7 @@ or authorize production enablement.
|
||||
| 12.3 | Dedicated population profile/trace/store/owning host; exact recovery and private PostgreSQL fault tests | Population-specific executable/network fault drills and target-host publication/capacity qualification. |
|
||||
| 12.4 | Owned intervals; exact transfer/accounting; durable atomic pair/recovery; independent loopback TLS path loss and owner expiry/replacement/heal; local object sync/read-failure matrix | Multi-host/kernel-network, storage-host and orchestrator drills, linked-boundary contracts and integrated ownership/stock audits at scale. |
|
||||
| 12.5 | Finite cellular contact feeding/decay and separate M6 producer/consumer resource owner | One jointly owned cellular/environment budget, explicit effective depth and source entitlements, versioned water/atmosphere species mapping and producer coefficients, plus closed/open and nutrient-exhaustion comparisons. |
|
||||
| 12.6 | Retained archive plus bounded regional resident/presence/cohort census and exact/descendant queries (12.6a) | Append-only population census/history events in the same durable commit, bounded world history queries/indexing and client presentation; no account power or invented precise statistical ancestry. |
|
||||
| 12.6 | Retained archive, bounded regional census/lineage queries and fully replayed census history pages from atomic biological commits (12.6a/b) | World/branch history context and cross-region query/index qualification, client permission/presentation path and deployment-host query costs; no account power or invented precise statistical ancestry. |
|
||||
| 12.7 | ADR-0133's viability/unknown semantics | Durable region-set/watermark reconciliation covering every owned representation, suspended/viable stock and migration escrow. Missing/stale regions must prevent terminal extinction. |
|
||||
| 12.8 | Deterministic seed/regime drivers and separate-process exact continuation | Reproducible integrated worlds running for actual days, real restarts/clone comparisons and retained accounting/diversity/cost audits. Accelerated tests cannot supply elapsed days. |
|
||||
| 12.9 | M11 feedback and current UI review retained | Returning-player samples on the integrated ecology and explicit product/3D readiness decision after soak evidence. No M12 product acceptance or World0 promotion is inferred. |
|
||||
@@ -98,9 +98,14 @@ M12.6a's [regional census projection](../../schemas/ecology/regional-cellular-ce
|
||||
independently implements local current-genome and descendant-closure queries.
|
||||
It keeps suspended stock and retained zero-carrier ancestry, separates exact cohort
|
||||
tags from additional ownership, and excludes nonresident history from living stock.
|
||||
It has no commit/branch/epoch binding or complete world coverage, so it cannot
|
||||
publish authoritative world history or accept terminal extinction. Durable atomic
|
||||
history publication and M12.7 reconciliation remain the next history boundaries.
|
||||
The local projection itself has no commit/branch/epoch binding or complete world
|
||||
coverage. M12.6b's [committed history reader](../../schemas/ecology/committed-cellular-history-v1.md)
|
||||
now derives it from the existing immutable atomic biological journal, with full
|
||||
historical replay, exact manifest binding and a fixed regional upper cut. No
|
||||
redundant census write or separate publication window is added. That regional cut
|
||||
is not an external branch identity or complete world watermark. World-level
|
||||
history context/queries, client presentation and M12.7 reconciliation remain open;
|
||||
neither primitive accepts terminal extinction.
|
||||
|
||||
Current expansion readiness: **no-go for M13 integration or World0 promotion**
|
||||
while these M12 integration/evidence gates remain open. This records the present
|
||||
|
||||
@@ -1,5 +1,19 @@
|
||||
# Validation strategy
|
||||
|
||||
M12.6b's private PostgreSQL history tests require exact fixed-watermark pagination
|
||||
after new no-tick publications, preserved genesis/profile/migration cuts, unchanged
|
||||
heads/stock and repeated event identity. Invalid limits, false cuts/regions/frames,
|
||||
unretained builds, stale evidence readers and caller-owned transactions fail.
|
||||
Missing/corrupt historical payloads and missing traces fail the entire page,
|
||||
including after an earlier valid entry. Independent connections cannot read a
|
||||
writer's pending cut. Historical raw/cohort recovery rejects canonical false
|
||||
migration biology; missing counterpart capsules fail pages. Earlier migration
|
||||
observations remain identical after return transfers under newer leases and match
|
||||
the complete portable pair oracle. Existing head recovery shares those validators;
|
||||
run sanitizer/analysis and complete private persistence regression for the refactor.
|
||||
Global branch/region-set reconciliation, public client access and target-host
|
||||
query costs remain separate gates.
|
||||
|
||||
M12.6a's regional census tests compare per-category counts and body/reserve
|
||||
inventories with direct individual observations. Cover active/lingering/suspended
|
||||
controlled lives, real funded wildlife, exact cohort conversion/restoration,
|
||||
|
||||
@@ -1,5 +1,64 @@
|
||||
# Validation report
|
||||
|
||||
## 2026-10-18 - M12.6b committed regional census history
|
||||
|
||||
ADR-0151 uses the retained immutable biological commit journal as canonical
|
||||
regional census events. That source already publishes atomically with the head;
|
||||
no duplicate census-write window, new table or schema migration is added. New
|
||||
historical raw/population recovery entry points share the unchanged complete
|
||||
validators with head recovery. History pages retain exact manifests and census
|
||||
projections for at most eight consecutive commits, pinned to an exact regional
|
||||
upper cut. See [the contract](../../schemas/ecology/committed-cellular-history-v1.md).
|
||||
|
||||
Executed locally:
|
||||
|
||||
- Native MSVC **19.44.35217.0** Release full CTest **79/79 passed**, **14.51 s**.
|
||||
- GCC 14 optional persistence/process targets rebuilt; private population/history
|
||||
qualification passed **16 s** (15 s population phase), with NDEBUG.
|
||||
- Linux Clang 19 **ASan/UBSan** combined history/TLS/storage fault matrix passed
|
||||
**67 s** (64 s population phase), without reports.
|
||||
- Explicit clang-tidy 19 on the PostgreSQL status adapter, history reader, shared
|
||||
raw/population recovery and integration tests passed with no owned warnings.
|
||||
Repository clang-format 19 and `git diff --check` passed.
|
||||
- Complete frozen-binary private PostgreSQL qualification passed with **exit 0**,
|
||||
**546 s**: C++ integration/process/load/history (434 s), combined history/TLS/
|
||||
storage faults (39 s), life reservations, backup/restore, host-loss DR, upgrade,
|
||||
migration history/rollback, runtime permission audit and checksum/future-version
|
||||
rejection. The detached runner recorded its completion code.
|
||||
|
||||
Tests retain genesis and separate no-tick profile cuts, pin pages across later
|
||||
publication and compare complete census rows with independently restored source
|
||||
payloads. Invalid limits/regions/frames/cut digests/builds and stale evidence
|
||||
readers fail. Missing/corrupt old payloads and missing traces fail an entire page,
|
||||
even after a valid genesis entry. Independent connections cannot see pending
|
||||
writer cuts. Public history requires a connected idle session; explicit writer
|
||||
and read-only transactions are refused. Closed connections after real TLS reply/
|
||||
request loss report non-idle, without a query or crash.
|
||||
|
||||
Historical raw/cohort recovery rejects a valid old object falsely declared as
|
||||
migration biology. Missing counterpart capsules fail history pages. Both endpoint
|
||||
observations match the exact portable pair, remain identical after return
|
||||
transfers and are read successfully under newer epochs than their original
|
||||
writers. Reads preserve current heads and biology; they never resume old state.
|
||||
|
||||
The first fixture draft replaced its lease before a separate test's account
|
||||
control reservation, correctly invalidating that old control. The independent
|
||||
history guard now uses a deliberately stale token; real replacement coverage
|
||||
belongs to the existing migration/host replacement matrix. Static analysis found
|
||||
an unchecked optional cursor in a test; the test now explicitly requires it.
|
||||
Review also added the transaction-idle guard to exclude a writer's own pending
|
||||
rows from the public durable-history API. No checks were suppressed.
|
||||
|
||||
Limits: this is regional backend history within the connected database's current
|
||||
frame/context, not an external branch identity or complete world watermark.
|
||||
World/branch reconciliation, cross-region/public client queries, derived indexes
|
||||
and deployment-host query latency/capacity remain open. Per-I/O deadlines are not
|
||||
a page-wide service SLO. Each selected cut uses the same complete tail/no-tick
|
||||
replay from retained predecessors as head recovery; no genesis-to-cut execution
|
||||
is claimed. No new TSan, Docker or remote-CI result is claimed. No thread/dependency,
|
||||
biological format/rule, golden, deployment or World0 changed.
|
||||
Unrelated `Build_Errors.txt` and `tash drop` remain untracked.
|
||||
|
||||
## 2026-10-18 - M12.6a bounded regional census and lineage queries
|
||||
|
||||
ADR-0150 adds a detached rules-8 regional projection, with explicit presence and
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
#pragma once
|
||||
|
||||
#include "evo/persistence/population_checkpoint_store.hpp"
|
||||
#include "evo/server/lifecycle/cellular_census.hpp"
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <expected>
|
||||
#include <optional>
|
||||
#include <span>
|
||||
#include <vector>
|
||||
|
||||
namespace evo::persistence {
|
||||
inline constexpr std::size_t max_cellular_history_page = 8;
|
||||
struct CellularCensusHistoryEntry {
|
||||
CommittedCheckpoint committed;
|
||||
server::CellularCensus census;
|
||||
};
|
||||
struct CellularCensusHistoryPage {
|
||||
RegionKey key{};
|
||||
RegionHead watermark{}; // Exact immutable regional upper cut, not world coverage.
|
||||
std::vector<CellularCensusHistoryEntry> entries;
|
||||
std::optional<std::uint64_t> next_sequence;
|
||||
};
|
||||
enum class CellularHistoryError : std::uint8_t { invalid_request, cut_mismatch, recovery };
|
||||
struct CellularHistoryFailure {
|
||||
CellularHistoryError kind{};
|
||||
RecoveryFailure recovery{}; // Meaningful only for recovery.
|
||||
};
|
||||
// Read-only forward history from the existing atomic biological commit journal.
|
||||
// Inclusive first_sequence, at most eight consecutive commits, pinned upper cut.
|
||||
// Full historical replay precedes every census. Any missing/unsupported/false
|
||||
// evidence fails the ENTIRE page; there is no zero/skip/older-head fallback.
|
||||
// Requires an idle connection: caller-owned transactions (including read-only
|
||||
// transactions) are refused so pending writer changes cannot appear durable.
|
||||
// Current grant is used by existing fenced evidence readers; output confers no
|
||||
// live authority, recovery-branch identity or world completeness. Per-I/O timeout.
|
||||
[[nodiscard]] std::expected<CellularCensusHistoryPage, CellularHistoryFailure>
|
||||
load_cellular_census_history(Postgres&, const FilesystemObjectStore&, RegionKey, const LeaseGrant&,
|
||||
const RegionHead& watermark, std::uint64_t first_sequence,
|
||||
std::size_t limit, std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout);
|
||||
} // namespace evo::persistence
|
||||
@@ -239,6 +239,12 @@ resolve_cellular_prepared(Postgres& db, const CellularPreparedCommit& prepared,
|
||||
recover_cellular_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease, std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout);
|
||||
// Read-only historical recovery with the SAME full replay/evidence validation.
|
||||
// Never select this state as a live head, resume an old owner or infer authority.
|
||||
[[nodiscard]] std::expected<RecoveredCellularCheckpoint, RecoveryFailure>
|
||||
recover_cellular_commit(Postgres&, const FilesystemObjectStore&, RegionKey, std::uint64_t sequence,
|
||||
const LeaseGrant&, std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout);
|
||||
// Leased, no-tick representation migration. The v1 payload is restored and
|
||||
// converted in memory before one immutable v1-to-v2 migration commit is written.
|
||||
[[nodiscard]] std::expected<CommittedCheckpoint, CellularProfileMigrationFailure>
|
||||
|
||||
@@ -45,6 +45,13 @@ resolve_population_prepared(Postgres&, const PopulationPreparedCommit&,
|
||||
recover_population_checkpoint(Postgres&, const FilesystemObjectStore&, RegionKey, const LeaseGrant&,
|
||||
std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout);
|
||||
// Explicit read-only historical recovery. It does not replace the current head
|
||||
// or grant ownership; all ordinary evidence/complete replay checks still apply.
|
||||
[[nodiscard]] std::expected<RecoveredPopulationCheckpoint, RecoveryFailure>
|
||||
recover_population_commit(Postgres&, const FilesystemObjectStore&, RegionKey,
|
||||
std::uint64_t sequence, const LeaseGrant&,
|
||||
std::span<const Sha256Digest> retained_builds,
|
||||
std::chrono::milliseconds timeout);
|
||||
// Used by raw cellular recovery after a population -> individual profile transition.
|
||||
[[nodiscard]] std::expected<void, RecoveryFailure>
|
||||
verify_population_profile_transition(Postgres&, const FilesystemObjectStore&, const LeaseGrant&,
|
||||
|
||||
@@ -37,6 +37,9 @@ class Postgres {
|
||||
Postgres& operator=(const Postgres&) = delete;
|
||||
[[nodiscard]] static std::expected<std::unique_ptr<Postgres>, PgFailure>
|
||||
connect(const PgOptions& options, std::chrono::milliseconds timeout);
|
||||
// True only for a connected, idle session outside an explicit transaction.
|
||||
// Read-only status inspection; no query, lock or transaction is started.
|
||||
[[nodiscard]] bool transaction_idle() const noexcept;
|
||||
[[nodiscard]] std::expected<PgRows, PgFailure> query(const std::string& sql,
|
||||
std::span<const std::string> parameters,
|
||||
std::chrono::milliseconds timeout);
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# Committed regional cellular census history v1
|
||||
|
||||
Authority: [ADR-0151](../../docs/adr/0151-committed-cellular-census-history.md).
|
||||
API: `evo::persistence::load_cellular_census_history`, Linux optional persistence
|
||||
adapter. Census semantics remain [regional projection v1](regional-cellular-census-v1.md).
|
||||
This is not a new byte codec, SQL table, network protocol or world reconciliation.
|
||||
|
||||
## Durable source and request
|
||||
|
||||
The immutable biological commit is the canonical history event. Its manifest and
|
||||
strong payload/trace bindings are retained in the existing append-only journal,
|
||||
in the same transaction that publishes the head. Atomic paired migration retains
|
||||
both endpoint cuts together. Orphan objects, prepared requests and staged ghosts
|
||||
do not create biological commit events. Exact publication retries retain the same
|
||||
commit identity rather than adding census events. No separate census write occurs.
|
||||
|
||||
A request supplies:
|
||||
|
||||
- exact regional key in the connected database and the evidence reader's lease;
|
||||
- the exact upper regional `RegionHead`: frame, sequence, commit ID, SHA-256 of
|
||||
its complete manifest, and completed tick;
|
||||
- inclusive first sequence and page limit 1–8;
|
||||
- retained build allowlist and the existing per-I/O timeout.
|
||||
|
||||
The connection must be connected and transaction-idle at entry. Explicit writer
|
||||
and read-only transactions are refused. Otherwise a publisher could observe its
|
||||
own uncommitted cut and mislabel it durable. Transaction status inspection starts
|
||||
no transaction and takes no lock. Lower-level explicit historical recovery keeps
|
||||
the existing diagnostic caller-transaction semantics and does not itself certify
|
||||
external commit visibility.
|
||||
|
||||
First sequence above the watermark, invalid limits, mismatched lease/frame, zero
|
||||
regional identity and non-positive timeouts fail before reading. The exact upper
|
||||
cut is checked against the journal, with its manifest digest and identity, before
|
||||
and after the page. New head publication cannot move that fixed upper cut. A false
|
||||
or missing cut fails; it never silently selects the latest or nearest older head.
|
||||
|
||||
## Replay and result
|
||||
|
||||
For each consecutive sequence, load its immutable manifest, select raw/profile-4
|
||||
or exact population/profile-5 recovery, and run the same complete verification as
|
||||
current-head recovery: supported versions/build, manifest digest/identity, strong
|
||||
object bindings, canonical restore, retained trace/outcomes and complete biological
|
||||
replay. No-tick upgrades and profile conversions use their recorded operations;
|
||||
paired migration replays both candidates and checks the shared receipt and local
|
||||
counterpart capsules. Recheck the recovered record against the initially loaded
|
||||
cut before pairing it with a census.
|
||||
As with head recovery, replay covers the selected commit's complete tail/no-tick
|
||||
operation from retained predecessor checkpoints, not all ticks since genesis.
|
||||
|
||||
An entry contains the exact committed record and its detached census. No-tick
|
||||
commits remain distinct even at equal completed ticks; sequence is the cursor.
|
||||
`next_sequence` exists only if another sequence remains below the same fixed upper
|
||||
cut. Reading/repeating a page changes no head, state, authority or event identity.
|
||||
Historical recovery never selects an old checkpoint for a running host.
|
||||
|
||||
Any unavailable, corrupt, unsupported, unreplayable or foreign entry fails the
|
||||
entire page. Earlier valid entries in that attempt do not escape as a partial
|
||||
page. Missing payload, trace, counterpart capsule or build support is not a zero
|
||||
population, local absence or terminal extinction. Current lease requirements of
|
||||
the evidence loaders still apply, independently of the manifests' original writers.
|
||||
Read output is not a new grant of authority.
|
||||
|
||||
The existing unique regional sequence index bounds each metadata lookup. At most
|
||||
eight census projections are retained; biological restore/replay work runs one cut
|
||||
at a time. There is no page-wide latency/capacity promise or automatic retry.
|
||||
Queries use committed publication boundaries, not fabricated intermediate-tick
|
||||
population samples; organism/genome events remain retained in the full archive.
|
||||
|
||||
## Coverage limits
|
||||
|
||||
This is a backend regional read primitive within the connected database's current
|
||||
frame/history context. Its regional watermark is not a world watermark, external
|
||||
branch identifier, complete region-set proof or client permission boundary.
|
||||
Cross-database/clone/rewind evidence must not be merged solely because numeric
|
||||
world/region/tick or an earlier shared prefix matches. Branch/world reconciliation,
|
||||
public queries, derived performance indexes/caches, client presentation and actual
|
||||
multi-day ecology remain subsequent gates. No terminal-extinction result exists.
|
||||
@@ -69,3 +69,8 @@ Durable M12.6 publication must derive this projection from a verified committed
|
||||
payload, bind its manifest/digest and branch, and append evidence atomically with
|
||||
the biological head. M12.7 must reconcile every required owner/store at a complete
|
||||
watermark. Neither operation is implemented by this local projection.
|
||||
|
||||
ADR-0151 now binds projections on read to the existing atomic immutable biological
|
||||
journal through [committed regional history](committed-cellular-history-v1.md).
|
||||
The commit itself is the canonical event; no redundant census write is added.
|
||||
Complete world/branch reconciliation remains separate.
|
||||
|
||||
@@ -624,14 +624,15 @@ verify_cellular_no_tick_checkpoint(Postgres& db, const FilesystemObjectStore& ob
|
||||
return {};
|
||||
}
|
||||
|
||||
namespace {
|
||||
std::expected<RecoveredCellularCheckpoint, RecoveryFailure>
|
||||
recover_cellular_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease, std::span<const Sha256Digest> retained_builds,
|
||||
milliseconds timeout) {
|
||||
recover_cellular_loaded(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease, std::span<const Sha256Digest> retained_builds,
|
||||
milliseconds timeout,
|
||||
std::expected<CommittedCheckpoint, LeaseStoreFailure> committed) {
|
||||
const auto fail_with = [](RecoveryError kind) {
|
||||
return std::unexpected{RecoveryFailure{kind, {}}};
|
||||
};
|
||||
auto committed = load_region_checkpoint(db, key, timeout);
|
||||
if (!committed)
|
||||
return std::unexpected{
|
||||
RecoveryFailure{committed.error().kind == LeaseStoreError::not_provisioned
|
||||
@@ -802,4 +803,19 @@ recover_cellular_checkpoint(Postgres& db, const FilesystemObjectStore& objects,
|
||||
std::move(mutation_actions_recovered),
|
||||
std::move(mutation_results_recovered)};
|
||||
}
|
||||
} // namespace
|
||||
std::expected<RecoveredCellularCheckpoint, RecoveryFailure>
|
||||
recover_cellular_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease, std::span<const Sha256Digest> retained_builds,
|
||||
milliseconds timeout) {
|
||||
return recover_cellular_loaded(db, objects, key, lease, retained_builds, timeout,
|
||||
load_region_checkpoint(db, key, timeout));
|
||||
}
|
||||
std::expected<RecoveredCellularCheckpoint, RecoveryFailure>
|
||||
recover_cellular_commit(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
std::uint64_t sequence, const LeaseGrant& lease,
|
||||
std::span<const Sha256Digest> retained_builds, milliseconds timeout) {
|
||||
return recover_cellular_loaded(db, objects, key, lease, retained_builds, timeout,
|
||||
load_region_commit(db, key, sequence, timeout));
|
||||
}
|
||||
} // namespace evo::persistence
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
#include "evo/persistence/cellular_history_store.hpp"
|
||||
#include "evo/persistence/sha256.hpp"
|
||||
|
||||
#include <algorithm>
|
||||
#include <utility>
|
||||
|
||||
namespace evo::persistence {
|
||||
namespace {
|
||||
bool bound(const CommittedCheckpoint& checkpoint, RegionKey key) {
|
||||
const auto digest = sha256(checkpoint.manifest);
|
||||
const auto manifest = decode_manifest(checkpoint.manifest);
|
||||
const auto& head = checkpoint.head;
|
||||
return digest && *digest == head.digest && manifest && manifest->key == key &&
|
||||
manifest->frame == head.frame && manifest->sequence == head.sequence &&
|
||||
manifest->commit == head.commit && manifest->end_tick == head.end_tick;
|
||||
}
|
||||
auto failed(RecoveryFailure failure) {
|
||||
return std::unexpected(CellularHistoryFailure{CellularHistoryError::recovery, failure});
|
||||
}
|
||||
auto failed(RecoveryError kind) {
|
||||
return failed(RecoveryFailure{kind, {}});
|
||||
}
|
||||
std::expected<CellularCensusHistoryEntry, CellularHistoryFailure>
|
||||
observe_commit(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease, std::uint64_t sequence,
|
||||
std::span<const Sha256Digest> builds, std::chrono::milliseconds timeout) {
|
||||
auto cut = load_region_commit(db, key, sequence, timeout);
|
||||
if (!cut)
|
||||
return failed(RecoveryFailure{RecoveryError::store, cut.error()});
|
||||
if (!bound(*cut, key) || cut->head.sequence != sequence)
|
||||
return failed(RecoveryError::manifest);
|
||||
const auto manifest = decode_manifest(cut->manifest);
|
||||
if (!manifest)
|
||||
return failed(RecoveryError::manifest);
|
||||
auto observation = [&]() -> std::expected<server::CellularCensus, RecoveryFailure> {
|
||||
if (manifest->profile == CheckpointProfile::cellular_lives_v1) {
|
||||
auto recovered =
|
||||
recover_cellular_commit(db, objects, key, sequence, lease, builds, timeout);
|
||||
if (!recovered)
|
||||
return std::unexpected(recovered.error());
|
||||
if (recovered->committed.head != cut->head ||
|
||||
recovered->committed.manifest != cut->manifest)
|
||||
return std::unexpected(RecoveryFailure{RecoveryError::manifest, {}});
|
||||
auto census = server::CellularCensus::observe(*recovered->state);
|
||||
if (census)
|
||||
return std::move(*census);
|
||||
return std::unexpected(
|
||||
RecoveryFailure{census.error() == server::CellularCensusError::unsupported
|
||||
? RecoveryError::unsupported
|
||||
: RecoveryError::payload,
|
||||
{}});
|
||||
}
|
||||
if (manifest->profile == CheckpointProfile::cellular_population_v1) {
|
||||
auto recovered =
|
||||
recover_population_commit(db, objects, key, sequence, lease, builds, timeout);
|
||||
if (!recovered)
|
||||
return std::unexpected(recovered.error());
|
||||
if (recovered->committed.head != cut->head ||
|
||||
recovered->committed.manifest != cut->manifest)
|
||||
return std::unexpected(RecoveryFailure{RecoveryError::manifest, {}});
|
||||
auto census = server::CellularCensus::observe(*recovered->state);
|
||||
if (census)
|
||||
return std::move(*census);
|
||||
return std::unexpected(
|
||||
RecoveryFailure{census.error() == server::CellularCensusError::unsupported
|
||||
? RecoveryError::unsupported
|
||||
: RecoveryError::payload,
|
||||
{}});
|
||||
}
|
||||
return std::unexpected(RecoveryFailure{RecoveryError::unsupported, {}});
|
||||
}();
|
||||
if (!observation)
|
||||
return failed(observation.error());
|
||||
return CellularCensusHistoryEntry{std::move(*cut), std::move(*observation)};
|
||||
}
|
||||
} // namespace
|
||||
std::expected<CellularCensusHistoryPage, CellularHistoryFailure> load_cellular_census_history(
|
||||
Postgres& db, const FilesystemObjectStore& objects, RegionKey key, const LeaseGrant& lease,
|
||||
const RegionHead& watermark, std::uint64_t first_sequence, std::size_t limit,
|
||||
std::span<const Sha256Digest> retained_builds, std::chrono::milliseconds timeout) {
|
||||
if (!db.transaction_idle() || key.world == 0 || key.region == 0 || watermark.frame == 0 ||
|
||||
watermark.frame != lease.frame || first_sequence > watermark.sequence || limit == 0 ||
|
||||
limit > max_cellular_history_page || timeout <= std::chrono::milliseconds::zero())
|
||||
return std::unexpected(CellularHistoryFailure{CellularHistoryError::invalid_request, {}});
|
||||
auto upper = load_region_commit(db, key, watermark.sequence, timeout);
|
||||
if (!upper)
|
||||
return failed(RecoveryFailure{RecoveryError::store, upper.error()});
|
||||
if (upper->head != watermark || !bound(*upper, key))
|
||||
return std::unexpected(CellularHistoryFailure{CellularHistoryError::cut_mismatch, {}});
|
||||
CellularCensusHistoryPage page{key, watermark, {}, {}};
|
||||
page.entries.reserve(limit);
|
||||
const auto last = first_sequence + std::min(static_cast<std::uint64_t>(limit - 1),
|
||||
watermark.sequence - first_sequence);
|
||||
for (auto sequence = first_sequence;; ++sequence) {
|
||||
auto entry = observe_commit(db, objects, key, lease, sequence, retained_builds, timeout);
|
||||
if (!entry)
|
||||
return std::unexpected(entry.error());
|
||||
page.entries.push_back(std::move(*entry));
|
||||
if (sequence == last)
|
||||
break;
|
||||
}
|
||||
if (last < watermark.sequence)
|
||||
page.next_sequence = last + 1;
|
||||
const auto confirmed = load_region_commit(db, key, watermark.sequence, timeout);
|
||||
if (!confirmed)
|
||||
return failed(RecoveryFailure{RecoveryError::store, confirmed.error()});
|
||||
if (confirmed->head != watermark || confirmed->manifest != upper->manifest)
|
||||
return std::unexpected(CellularHistoryFailure{CellularHistoryError::cut_mismatch, {}});
|
||||
return page;
|
||||
}
|
||||
} // namespace evo::persistence
|
||||
@@ -248,12 +248,13 @@ verify_population_profile_transition(Postgres& db, const FilesystemObjectStore&
|
||||
return fail(RecoveryError::unsupported);
|
||||
return {};
|
||||
}
|
||||
namespace {
|
||||
std::expected<RecoveredPopulationCheckpoint, RecoveryFailure>
|
||||
recover_population_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease,
|
||||
std::span<const Sha256Digest> retained_builds, milliseconds timeout) {
|
||||
recover_population_loaded(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease, std::span<const Sha256Digest> retained_builds,
|
||||
milliseconds timeout,
|
||||
std::expected<CommittedCheckpoint, LeaseStoreFailure> committed) {
|
||||
const auto fail = [](RecoveryError kind) { return std::unexpected(RecoveryFailure{kind, {}}); };
|
||||
auto committed = load_region_checkpoint(db, key, timeout);
|
||||
if (!committed)
|
||||
return std::unexpected(
|
||||
RecoveryFailure{committed.error().kind == LeaseStoreError::not_provisioned
|
||||
@@ -319,4 +320,19 @@ recover_population_checkpoint(Postgres& db, const FilesystemObjectStore& objects
|
||||
}
|
||||
return RecoveredPopulationCheckpoint{std::move(*committed), *m, std::move(*population)};
|
||||
}
|
||||
} // namespace
|
||||
std::expected<RecoveredPopulationCheckpoint, RecoveryFailure>
|
||||
recover_population_checkpoint(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
const LeaseGrant& lease,
|
||||
std::span<const Sha256Digest> retained_builds, milliseconds timeout) {
|
||||
return recover_population_loaded(db, objects, key, lease, retained_builds, timeout,
|
||||
load_region_checkpoint(db, key, timeout));
|
||||
}
|
||||
std::expected<RecoveredPopulationCheckpoint, RecoveryFailure>
|
||||
recover_population_commit(Postgres& db, const FilesystemObjectStore& objects, RegionKey key,
|
||||
std::uint64_t sequence, const LeaseGrant& lease,
|
||||
std::span<const Sha256Digest> retained_builds, milliseconds timeout) {
|
||||
return recover_population_loaded(db, objects, key, lease, retained_builds, timeout,
|
||||
load_region_commit(db, key, sequence, timeout));
|
||||
}
|
||||
} // namespace evo::persistence
|
||||
|
||||
@@ -119,6 +119,10 @@ struct Postgres::Impl {
|
||||
};
|
||||
Postgres::Postgres(std::unique_ptr<Impl> impl) : impl_(std::move(impl)) {}
|
||||
Postgres::~Postgres() = default;
|
||||
bool Postgres::transaction_idle() const noexcept {
|
||||
return impl_->connection && PQstatus(impl_->connection.get()) == CONNECTION_OK &&
|
||||
PQtransactionStatus(impl_->connection.get()) == PQTRANS_IDLE;
|
||||
}
|
||||
std::expected<PgOptions, PgError> read_pg_service(const std::string& file,
|
||||
std::string_view section) {
|
||||
constexpr std::size_t limit = 4096;
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
#include "evo/persistence/account_control_store.hpp"
|
||||
#include "evo/persistence/cellular_history_store.hpp"
|
||||
#include "evo/persistence/population_checkpoint_store.hpp"
|
||||
#include "evo/persistence/population_interval_store.hpp"
|
||||
#include "evo/persistence/sha256.hpp"
|
||||
@@ -12,6 +13,7 @@
|
||||
#include <cstdlib>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
#include <limits>
|
||||
#include <stdexcept>
|
||||
#include <sys/wait.h>
|
||||
#include <thread>
|
||||
@@ -159,7 +161,7 @@ void migration_transport_commit(Fixture& f, const CellularMigrationCommitPrepare
|
||||
!lost && lost.error().kind == PublicationError::unknown &&
|
||||
lost.error().prepared.source_manifest == prepared.source_manifest &&
|
||||
lost.error().prepared.destination_manifest == prepared.destination_manifest &&
|
||||
relay.dropped_requests() > 0 && relay.healthy() &&
|
||||
relay.dropped_requests() > 0 && relay.healthy() && !db->transaction_idle() &&
|
||||
take(load_region_checkpoint(*f.db, sk, 5s), "lost request source head").head ==
|
||||
source_before &&
|
||||
take(load_region_checkpoint(*f.db, dk, 5s), "lost request destination head").head ==
|
||||
@@ -175,7 +177,7 @@ void migration_transport_commit(Fixture& f, const CellularMigrationCommitPrepare
|
||||
!lost && lost.error().kind == PublicationError::unknown &&
|
||||
lost.error().prepared.source_manifest == prepared.source_manifest &&
|
||||
lost.error().prepared.destination_manifest == prepared.destination_manifest &&
|
||||
relay.dropped_responses() > 0 && relay.healthy() &&
|
||||
relay.dropped_responses() > 0 && relay.healthy() && !db->transaction_idle() &&
|
||||
take(load_region_checkpoint(*f.db, sk, 5s), "lost reply source head").manifest ==
|
||||
prepared.source_manifest &&
|
||||
take(load_region_checkpoint(*f.db, dk, 5s), "lost reply destination head")
|
||||
@@ -634,6 +636,159 @@ void missing_and_corrupt_objects(Fixture& f) {
|
||||
"corrupt population payload did not fail closed");
|
||||
recover(f);
|
||||
}
|
||||
void history_pages(Fixture& f) {
|
||||
const std::array<Sha256Digest, 1> builds{f.build};
|
||||
const auto upper = f.head;
|
||||
const auto biology = f.raw->snapshot();
|
||||
const auto read = [&](const RegionHead& watermark, std::uint64_t first, std::size_t limit) {
|
||||
return load_cellular_census_history(*f.db, *f.objects, f.key, f.lease, watermark, first,
|
||||
limit, builds, 5s);
|
||||
};
|
||||
auto first = take(read(upper.head, 0, 2), "history first page");
|
||||
require(first.entries.size() == 2 && first.next_sequence == 2 &&
|
||||
first.entries.front().committed.head.sequence == 0 &&
|
||||
first.entries.front().census.rows().empty(),
|
||||
"history lost genesis or changed sequence pagination");
|
||||
// Append two distinct no-tick cuts between requests; the old watermark stays pinned.
|
||||
const auto population =
|
||||
take(migrate_cellular_to_population(*f.db, *f.objects, f.lease, f.head, f.id(), f.build,
|
||||
{false, 1200, 1800}, 5s),
|
||||
"history population cut");
|
||||
f.head = take(
|
||||
materialize_population_profile(*f.db, *f.objects, f.lease, population, f.id(), f.build, 5s),
|
||||
"history raw cut");
|
||||
std::uint64_t next = take(first.next_sequence, "history continuation cursor");
|
||||
while (next <= upper.head.sequence) {
|
||||
auto page = take(read(upper.head, next, 2), "pinned historical page");
|
||||
require(page.watermark == upper.head && !page.entries.empty() && page.entries.size() <= 2,
|
||||
"history changed its pinned upper cut");
|
||||
for (const auto& entry : page.entries) {
|
||||
const auto stored =
|
||||
take(load_region_commit(*f.db, f.key, next, 5s), "history oracle cut");
|
||||
const auto manifest = take(decode_manifest(stored.manifest), "history oracle manifest");
|
||||
auto state =
|
||||
take(restore_cellular_payload(
|
||||
take(f.objects->get({manifest.payload_sha256, manifest.payload_bytes}),
|
||||
"history oracle payload"),
|
||||
manifest),
|
||||
"history oracle biology");
|
||||
const auto oracle = take(CellularCensus::observe(*state), "history oracle census");
|
||||
require(entry.committed.head == stored.head &&
|
||||
entry.committed.manifest == stored.manifest &&
|
||||
std::ranges::equal(entry.census.rows(), oracle.rows()) &&
|
||||
entry.census.boundary() == oracle.boundary(),
|
||||
"history rebound old census to a newer or different committed cut");
|
||||
++next;
|
||||
}
|
||||
require(
|
||||
page.next_sequence ==
|
||||
(next <= upper.head.sequence ? std::optional<std::uint64_t>{next} : std::nullopt),
|
||||
"history cursor skipped or invented a commit");
|
||||
}
|
||||
const auto transitions = take(read(f.head.head, upper.head.sequence + 1, 8), "no-tick history");
|
||||
require(transitions.entries.size() == 2 && !transitions.next_sequence &&
|
||||
transitions.entries[0].committed.head == population.head &&
|
||||
transitions.entries[1].committed.head == f.head.head &&
|
||||
transitions.entries[0].census.boundary().population_format == 1 &&
|
||||
transitions.entries[1].census.boundary().population_format == 0 &&
|
||||
population.head.end_tick == upper.head.end_tick &&
|
||||
f.head.head.end_tick == upper.head.end_tick,
|
||||
"history collapsed distinct no-tick events or lost representation identity");
|
||||
const auto repeat = take(read(upper.head, 0, 2), "repeat pinned history");
|
||||
require(repeat.entries[0].committed.head == first.entries[0].committed.head &&
|
||||
repeat.entries[1].committed.head == first.entries[1].committed.head &&
|
||||
f.raw->snapshot() == biology &&
|
||||
take(load_region_checkpoint(*f.db, f.key, 5s), "read-only history head").head ==
|
||||
f.head.head,
|
||||
"history read changed biology, head or event identity");
|
||||
for (const auto limit :
|
||||
{std::size_t{0}, max_cellular_history_page + 1, std::numeric_limits<std::size_t>::max()})
|
||||
require(!read(upper.head, 0, limit) && read(upper.head, 0, limit).error().kind ==
|
||||
CellularHistoryError::invalid_request,
|
||||
"unbounded history request admitted");
|
||||
require(!read(upper.head, upper.head.sequence + 1, 1), "history read beyond pinned cut");
|
||||
auto wrong = upper.head;
|
||||
wrong.digest.back() ^= 1;
|
||||
require(!read(wrong, 0, 1) &&
|
||||
read(wrong, 0, 1).error().kind == CellularHistoryError::cut_mismatch,
|
||||
"history accepted a false watermark digest");
|
||||
wrong = upper.head;
|
||||
++wrong.end_tick;
|
||||
require(!read(wrong, 0, 1), "history accepted a false watermark tick");
|
||||
wrong = upper.head;
|
||||
++wrong.frame;
|
||||
require(!read(wrong, 0, 1), "history merged foreign frames");
|
||||
require(!load_cellular_census_history(*f.db, *f.objects, {f.key.world, 999}, f.lease,
|
||||
upper.head, 0, 1, builds, 5s) &&
|
||||
!load_cellular_census_history(*f.db, *f.objects, f.key, f.lease, upper.head, 0, 1,
|
||||
{}, 5s),
|
||||
"foreign region or unretained build became an empty history");
|
||||
const auto birth = take(load_region_commit(*f.db, f.key, 1, 5s), "historical birth cut");
|
||||
const auto manifest = take(decode_manifest(birth.manifest), "historical birth manifest");
|
||||
const auto path = f.objects->directory() / hexadecimal(manifest.payload_sha256, false);
|
||||
const auto saved = path.string() + ".history-test";
|
||||
const auto bytes = take(f.objects->get({manifest.payload_sha256, manifest.payload_bytes}),
|
||||
"historical birth bytes");
|
||||
std::filesystem::rename(path, saved);
|
||||
const auto missing =
|
||||
read(upper.head, 0, 2); // Genesis succeeded; second cut must fail the page.
|
||||
std::filesystem::rename(saved, path);
|
||||
require(!missing && missing.error().kind == CellularHistoryError::recovery,
|
||||
"missing old payload returned a partial page or zero census");
|
||||
{
|
||||
std::ofstream file(path, std::ios::binary);
|
||||
file << 'X';
|
||||
require(file.good(), "corrupt old history payload fixture");
|
||||
}
|
||||
const auto corrupt = read(upper.head, 0, 2);
|
||||
{
|
||||
std::ofstream file(path, std::ios::binary);
|
||||
file.write(reinterpret_cast<const char*>(bytes.data()),
|
||||
static_cast<std::streamsize>(bytes.size()));
|
||||
require(file.good(), "restore old history payload fixture");
|
||||
}
|
||||
require(!corrupt && corrupt.error().kind == CellularHistoryError::recovery,
|
||||
"corrupt historical object returned a census");
|
||||
const auto trace_path =
|
||||
f.objects->directory() / hexadecimal(manifest.batch.trace_sha256, false);
|
||||
const auto trace_saved = trace_path.string() + ".history-test";
|
||||
std::filesystem::rename(trace_path, trace_saved);
|
||||
const auto absent_trace = read(upper.head, 1, 1);
|
||||
std::filesystem::rename(trace_saved, trace_path);
|
||||
require(!absent_trace && absent_trace.error().recovery.kind == RecoveryError::tail,
|
||||
"history trusted a payload without its replay evidence");
|
||||
take(f.db->query("BEGIN", {}, 5s), "uncommitted history transaction");
|
||||
const auto transactional = read(upper.head, 0, 1);
|
||||
require(!f.db->transaction_idle() && !transactional &&
|
||||
transactional.error().kind == CellularHistoryError::invalid_request,
|
||||
"history admitted a caller-owned transaction");
|
||||
const auto orphan =
|
||||
take(migrate_cellular_to_population(*f.db, *f.objects, f.lease, f.head, f.id(), f.build,
|
||||
{false, 1200, 1800}, 5s),
|
||||
"uncommitted history cut");
|
||||
auto observer = connection();
|
||||
const auto invisible = load_cellular_census_history(
|
||||
*observer, *f.objects, f.key, f.lease, orphan.head, orphan.head.sequence, 1, builds, 5s);
|
||||
take(f.db->query("ROLLBACK", {}, 5s), "rollback uncommitted history cut");
|
||||
require(f.db->transaction_idle() && !invisible &&
|
||||
take(load_region_checkpoint(*f.db, f.key, 5s), "orphan history head").head ==
|
||||
f.head.head,
|
||||
"uncommitted object/ghost became a durable history event");
|
||||
auto old_lease = f.lease;
|
||||
--old_lease.epoch;
|
||||
const auto stale = load_cellular_census_history(*f.db, *f.objects, f.key, old_lease, upper.head,
|
||||
1, 1, builds, 5s);
|
||||
require(!stale && stale.error().recovery.store.kind == LeaseStoreError::fenced &&
|
||||
take(read(upper.head, 0, 2), "current history reader").entries[1].committed.head ==
|
||||
birth.head,
|
||||
"historical evidence ignored a stale fence or rewrote original writers");
|
||||
take(f.db->query("BEGIN READ ONLY", {}, 5s), "read-only history transaction");
|
||||
const auto readonly = read(upper.head, 0, 1);
|
||||
take(f.db->query("ROLLBACK", {}, 5s), "close read-only history transaction");
|
||||
require(!readonly && readonly.error().kind == CellularHistoryError::invalid_request,
|
||||
"history transaction requirement changed for read-only callers");
|
||||
std::puts("committed census history pagination, replay, transaction and failure guards passed");
|
||||
}
|
||||
void boundary_upgrades(Fixture& f) {
|
||||
const std::array<Sha256Digest, 1> builds{f.build};
|
||||
const auto original = f.head;
|
||||
@@ -1063,6 +1218,7 @@ void migration_publication(Fixture& f, const RegionKey destination_key,
|
||||
destination_head, f.id(), f.build, {true, 1200, 1800}, 5s),
|
||||
"paired population destination");
|
||||
}
|
||||
std::optional<CellularCensusHistoryEntry> outgoing_history, incoming_history;
|
||||
for (unsigned direction = 0; direction < 2; ++direction) {
|
||||
auto& sh = direction == 0 ? f.head : destination_head;
|
||||
auto& dh = direction == 0 ? destination_head : f.head;
|
||||
@@ -1221,8 +1377,25 @@ void migration_publication(Fixture& f, const RegionKey destination_key,
|
||||
const auto false_recovery = verify_cellular_migration_checkpoint(
|
||||
*f.db, so, sl, take(load_region_checkpoint(*f.db, sk, 5s), "false paired head"), sb,
|
||||
5s);
|
||||
const auto false_head =
|
||||
take(load_region_checkpoint(*f.db, sk, 5s), "false history cut");
|
||||
const auto false_history = [&]() -> std::expected<void, RecoveryFailure> {
|
||||
if (representation == 0) {
|
||||
auto recovered = recover_cellular_commit(
|
||||
*f.db, so, sk, false_head.head.sequence, sl, builds, 5s);
|
||||
if (!recovered)
|
||||
return std::unexpected(recovered.error());
|
||||
} else {
|
||||
auto recovered = recover_population_commit(
|
||||
*f.db, so, sk, false_head.head.sequence, sl, builds, 5s);
|
||||
if (!recovered)
|
||||
return std::unexpected(recovered.error());
|
||||
}
|
||||
return {};
|
||||
}();
|
||||
take(f.db->query("ROLLBACK", {}, 5s), "rollback false paired biology fixture");
|
||||
require(!false_recovery && false_recovery.error().kind == RecoveryError::history,
|
||||
require(!false_recovery && false_recovery.error().kind == RecoveryError::history &&
|
||||
!false_history && false_history.error().kind == RecoveryError::history,
|
||||
"paired recovery accepted a valid old object as the migrated biology");
|
||||
const std::array<std::string, 1> generic{hexadecimal(prepared.source_manifest)};
|
||||
require(!f.db->query("SELECT * FROM evolution.publish_region_checkpoint($1::bytea)",
|
||||
@@ -1399,13 +1572,48 @@ void migration_publication(Fixture& f, const RegionKey destination_key,
|
||||
require(check_bytes(sh, sk, so, sl) == expected_source &&
|
||||
check_bytes(dh, dk, dst, dl) == expected_destination,
|
||||
"committed biology/member/material differs from the exact pair");
|
||||
auto source_history =
|
||||
take(load_cellular_census_history(*f.db, so, sk, sl, sh.head, sh.head.sequence, 1,
|
||||
builds, 5s),
|
||||
"paired source history");
|
||||
auto destination_history =
|
||||
take(load_cellular_census_history(*f.db, dst, dk, dl, dh.head, dh.head.sequence, 1,
|
||||
builds, 5s),
|
||||
"paired destination history");
|
||||
const auto census_oracle = [&](const auto& bytes,
|
||||
const CommittedCheckpoint& checkpoint) {
|
||||
const auto manifest =
|
||||
take(decode_manifest(checkpoint.manifest), "history pair oracle manifest");
|
||||
if (representation == 0)
|
||||
return take(
|
||||
CellularCensus::observe(*take(restore_cellular_payload(bytes, manifest),
|
||||
"history raw pair oracle")),
|
||||
"history raw census oracle");
|
||||
return take(
|
||||
CellularCensus::observe(*take(restore_population_payload(bytes, manifest),
|
||||
"history cohort pair oracle")),
|
||||
"history cohort census oracle");
|
||||
};
|
||||
require(source_history.entries.size() == 1 && destination_history.entries.size() == 1 &&
|
||||
std::ranges::equal(source_history.entries[0].census.rows(),
|
||||
census_oracle(expected_source, sh).rows()) &&
|
||||
std::ranges::equal(destination_history.entries[0].census.rows(),
|
||||
census_oracle(expected_destination, dh).rows()),
|
||||
"historical migration census differs from exact paired biology");
|
||||
if (direction == 0) {
|
||||
outgoing_history = std::move(source_history.entries.front());
|
||||
incoming_history = std::move(destination_history.entries.front());
|
||||
}
|
||||
const auto capsule_path = so.directory() / hexadecimal(d.payload_sha256, false);
|
||||
const auto backup = capsule_path.string() + ".paired-test";
|
||||
std::filesystem::rename(capsule_path, backup);
|
||||
const auto missing =
|
||||
verify_cellular_migration_checkpoint(*f.db, so, sl, sh, expected_source, 5s);
|
||||
const auto missing_history = load_cellular_census_history(
|
||||
*f.db, so, sk, sl, sh.head, sh.head.sequence, 1, builds, 5s);
|
||||
std::filesystem::rename(backup, capsule_path);
|
||||
require(!missing && missing.error().kind == RecoveryError::tail &&
|
||||
require(!missing && missing.error().kind == RecoveryError::tail && !missing_history &&
|
||||
missing_history.error().recovery.kind == RecoveryError::tail &&
|
||||
!verify_cellular_migration_checkpoint(*f.db, so, sl, sh, sb, 5s),
|
||||
"pair recovery ignored missing counterpart evidence or false local biology");
|
||||
source_host->drain();
|
||||
@@ -1427,6 +1635,29 @@ void migration_publication(Fixture& f, const RegionKey destination_key,
|
||||
dl = take(acquire_region_lease(*f.db, dk, f.build, 60000ms, 5s),
|
||||
"return paired destination authority");
|
||||
}
|
||||
const auto outgoing = take(std::move(outgoing_history), "retained source historical cut");
|
||||
const auto incoming =
|
||||
take(std::move(incoming_history), "retained destination historical cut");
|
||||
const auto old_source =
|
||||
take(load_cellular_census_history(*f.db, *f.objects, f.key, f.lease, f.head.head,
|
||||
outgoing.committed.head.sequence, 1, builds, 5s),
|
||||
"source history after return");
|
||||
const auto old_destination =
|
||||
take(load_cellular_census_history(*f.db, destination_objects, destination_key,
|
||||
destination_lease, destination_head.head,
|
||||
incoming.committed.head.sequence, 1, builds, 5s),
|
||||
"destination history after return");
|
||||
require(
|
||||
old_source.entries[0].committed.head == outgoing.committed.head &&
|
||||
old_destination.entries[0].committed.head == incoming.committed.head &&
|
||||
take(decode_manifest(outgoing.committed.manifest), "historical source writer")
|
||||
.epoch < f.lease.epoch &&
|
||||
take(decode_manifest(incoming.committed.manifest), "historical destination writer")
|
||||
.epoch < destination_lease.epoch &&
|
||||
std::ranges::equal(old_source.entries[0].census.rows(), outgoing.census.rows()) &&
|
||||
std::ranges::equal(old_destination.entries[0].census.rows(),
|
||||
incoming.census.rows()),
|
||||
"return migration rewrote earlier committed ownership observations");
|
||||
migration_owner_partitions(f, destination_key, destination_objects, destination_lease,
|
||||
destination_head, genome);
|
||||
}
|
||||
@@ -2341,6 +2572,7 @@ void host_ownership_and_admission(Fixture& f) {
|
||||
void run_population_publication_tests() {
|
||||
Fixture fixture;
|
||||
funded_source(fixture);
|
||||
history_pages(fixture);
|
||||
conversions_and_retries(fixture);
|
||||
interval_contract(fixture);
|
||||
ambiguous_publication(fixture, false);
|
||||
|
||||
Reference in New Issue
Block a user