Add M10.8d two-process handoff fault matrix and recovery fixes
Foundation / checks (deploy) (push) Successful in 3m20s
Foundation / checks (ecs) (push) Successful in 1m57s
Foundation / checks (client) (push) Failing after 10m4s
Foundation / checks (network) (push) Successful in 13m3s
Foundation / checks (tsan) (push) Successful in 7m53s
Foundation / checks (checks) (push) Successful in 32m18s
Foundation / checks (postgres) (push) Successful in 43m26s
Foundation / postgres-tools (push) Skipped
Foundation / server-image (push) Skipped
Foundation / image (push) Skipped
Foundation / checks (deploy) (push) Successful in 3m20s
Foundation / checks (ecs) (push) Successful in 1m57s
Foundation / checks (client) (push) Failing after 10m4s
Foundation / checks (network) (push) Successful in 13m3s
Foundation / checks (tsan) (push) Successful in 7m53s
Foundation / checks (checks) (push) Successful in 32m18s
Foundation / checks (postgres) (push) Successful in 43m26s
Foundation / postgres-tools (push) Skipped
Foundation / server-image (push) Skipped
Foundation / image (push) Skipped
ADR-0096. Two real evo-server processes on PostgreSQL (regions 7/161 and 7/162, adjacent areas) run crossings and crossings back, SIGKILL of the source after prepare and of the destination after stage and after commit, duplicate and stale ticket retries, a client that never arrives and a destination that is down. After every scenario SQL checks one owner per probe, the conserved inventory and frozen-iff-open-transfer. The matrix found two recovery gaps, now fixed: - a source restarted after a prepare left the probe frozen for good (its abort deadline was in memory): schema v14 overdue_probe_transfers and a session-layer sweep abort open outgoing transfers past the ticket lifetime plus 10 s by the database clock; - a ticket replayed at a restarted destination after the probe moved on took control back: admission of a committed transfer now requires the record as that commit left it. M10.8 is complete. Refreshes the dev session window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -748,4 +748,27 @@ incoming_probe_transfers(Postgres& db, RegionKey key, const LeaseGrant& lease,
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::expected<std::vector<Uuid>, ProbeStoreFailure>
|
||||
overdue_probe_transfers(Postgres& db, RegionKey key, const LeaseGrant& lease,
|
||||
std::chrono::seconds age, milliseconds timeout) {
|
||||
if (age.count() < 1 || age.count() > 86400)
|
||||
return std::unexpected{ProbeStoreFailure{ProbeStoreError::invalid, {}}};
|
||||
const std::array<std::string, 5> p{std::to_string(key.world), std::to_string(key.region),
|
||||
uuid_text(lease.holder), std::to_string(lease.epoch),
|
||||
std::to_string(age.count())};
|
||||
auto rows = db.query("SELECT * FROM evolution.overdue_probe_transfers($1::evolution.u64, "
|
||||
"$2::evolution.u64, $3::uuid, $4::evolution.u64, $5::integer)",
|
||||
p, timeout);
|
||||
if (!rows)
|
||||
return std::unexpected{probe_failure(rows.error())};
|
||||
std::vector<Uuid> out;
|
||||
for (const auto& r : *rows) {
|
||||
const auto transfer = r.size() == 1 ? parse_uuid(r[0]) : std::nullopt;
|
||||
if (!transfer)
|
||||
return std::unexpected{ProbeStoreFailure{ProbeStoreError::unknown, {}}};
|
||||
out.push_back(*transfer);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
} // namespace evo::persistence
|
||||
|
||||
Reference in New Issue
Block a user