Complete M7.7 recovery verification before readiness
ADR-0065: before any readiness, recovery now verifies the head's tail. The trace object must exist with its manifest digest, its header must bind profile, region, predecessor and range; the predecessor (schema v6 load_region_commit) must hash to the digest the head names; replaying the trace from the predecessor's verified payload must reproduce every report and the exact head bytes; and trace-derived decisions must equal the head's durable decisions. Missing, corrupt or non-reproducing tails quarantine (RecoveryError::tail); nothing older or empty is served. Test-only crash seams (HostConfig::crash_hook, PublishHooks) and evo-server --crash-at deliver real SIGKILL at before-upload, before-transaction, after-commit and after-ack. The kill matrix, a SIGSTOP/SIGCONT stale owner and damaged-head startups all behave as the durability contract requires; successors verify before readiness. Validated: PostgreSQL suites under GCC Release, clang-tidy and ASan/UBSan; tail-skip and replay-equality mutations each caught. No Docker/Gitea CI, production-lease kill test or deployment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -228,6 +228,8 @@ template <typename State> persistence::Uuid RegionHost<State>::next_commit_id()
|
||||
|
||||
template <typename State>
|
||||
void RegionHost<State>::confirmed(const persistence::PublishedBatch& published, SchedulerTime now) {
|
||||
if (config_.crash_hook)
|
||||
config_.crash_hook(CrashPoint::after_commit);
|
||||
const auto& head = published.committed.head;
|
||||
if (!batch_ || !batch_->confirm(head.end_tick)) {
|
||||
lose_authority(Unacknowledged::quarantined, HostPhase::quarantined);
|
||||
@@ -255,9 +257,15 @@ template <typename State> void RegionHost<State>::commit(SchedulerTime now) {
|
||||
return;
|
||||
const auto commit_id = next_commit_id();
|
||||
batch_->seal();
|
||||
const auto& crash = config_.crash_hook;
|
||||
if (crash)
|
||||
crash(CrashPoint::before_upload);
|
||||
persistence::PublishHooks hooks;
|
||||
if (crash)
|
||||
hooks.before_transaction = [&crash] { crash(CrashPoint::before_transaction); };
|
||||
auto published =
|
||||
Ops<State>::publish(*db_, *objects_, *grant, *committed_, *batch_, commit_id, config_.build,
|
||||
config_.restore_budget, config_.lease.commit_timeout);
|
||||
config_.restore_budget, config_.lease.commit_timeout, hooks);
|
||||
if (published) {
|
||||
confirmed(*published, now);
|
||||
return;
|
||||
|
||||
Reference in New Issue
Block a user