ADR-0065: before any readiness, recovery now verifies the head's tail.
The trace object must exist with its manifest digest, its header must
bind profile, region, predecessor and range; the predecessor (schema v6
load_region_commit) must hash to the digest the head names; replaying
the trace from the predecessor's verified payload must reproduce every
report and the exact head bytes; and trace-derived decisions must equal
the head's durable decisions. Missing, corrupt or non-reproducing tails
quarantine (RecoveryError::tail); nothing older or empty is served.
Test-only crash seams (HostConfig::crash_hook, PublishHooks) and
evo-server --crash-at deliver real SIGKILL at before-upload,
before-transaction, after-commit and after-ack. The kill matrix,
a SIGSTOP/SIGCONT stale owner and damaged-head startups all behave as
the durability contract requires; successors verify before readiness.
Validated: PostgreSQL suites under GCC Release, clang-tidy and
ASan/UBSan; tail-skip and replay-equality mutations each caught. No
Docker/Gitea CI, production-lease kill test or deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review P1 (object durability): a name installed by a put whose directory
fsync failed was later reused as success without syncing again, so a
publication could acknowledge an object that might vanish on power
loss. Reuse (existing and lost-link-race paths) now fsyncs the verified
file and the directory before succeeding, and namespace opens always
sync their parent. A test-only ObjectStoreFaults hook injects directory
sync failures; tests cover failed sync then retries, namespace creation
syncs, and publication refusing to proceed until durability holds.
Review P1 (pre-v5 history): v5 recovery requires one durable decision
per terminal actor, but v3/v4 published checkpoints carry none, so an
accepted upgrade could make regions unrecoverable. pre_v5_guard.sql now
runs before 0005 in every upgrade path and refuses (EVL10) when any
published pre-v5 checkpoint exists; genesis-only regions pass. 0005 is
unchanged. CI publishes a real v4 checkpoint and proves the upgrade is
refused with nothing changed; a committed death without a decision is
shown to recover as quarantined history.
Validated: PostgreSQL suites Release x3, clang-tidy, ASan/UBSan x2;
reverting each fix is caught by its test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0063: manifest v2 (360 bytes) binds a per-tick validation trace
object and a canonical decision block by SHA-256. Publication replays the
trace from the committed predecessor's verified payload and requires
every report and the candidate bytes, derives deaths deterministically,
uploads payload and trace with verified read-back, then sends manifest
and decisions to one fenced statement. RegionBatch seals at a terminal
transition or 64 ticks and refuses ticks until durable confirmation.
Schema v5 records the checkpoint, its decisions and outbox rows
atomically; a unique index keeps one terminal outcome per actor
(EVL08), triggers make commits/decisions immutable (EVL09), an exact
retry of a committed ID returns the original receipt, changed bytes
conflict, and read-only resolve_region_commit answers committed/absent/
conflict. Outbox pending/acknowledge give idempotent at-least-once
delivery. Recovery reconciles durable deaths with restored lifecycle
(and archive deaths for the lineage profile).
Validated: Windows CTest 58/58; Linux GCC Debug/Release, clang-tidy and
ASan/UBSan CTest 58/58 each; PostgreSQL suites Release, analysis, ASan
(lost response, committed-after-timeout, terminated orphan retry,
changed bytes, repeated/partial/reordered decisions, gap, outbox
redelivery, history quarantine); five mutations each caught. No
Docker/Gitea CI, process kill, broker or deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0062 adds a canonical LineageArchive codec (parents before children,
organisms by id, FNV) that decodes by replaying admit/record_birth/
record_death and requires byte-identical re-encoding; it never overwrites
an existing archive. LineageRegion binds the environmental producer and
consumer accounts to archived organisms with identity high-waters,
enforces closure (lifecycle vs deaths, exact living set, tick bounds)
and records deaths in the tick that causes them.
Checkpoint profile 2 (environmental-lineage-v1, history profile 1)
publishes that container; schema v4 admits it with a 4,268,144-byte cap
and requires each commit to keep its predecessor's profile, RNG, history
and durability declarations. Linux adds provision/publish/recover for
the profile on the shared publication path.
Qualification found WSL2 stepping CLOCK_REALTIME back ~0.5 s every ~32 s,
which made two 1 s-lease tests flaky under ASan. Tests now wait on
database-time expiry; ADR-0060 records the slewed-clock requirement.
Validated: Windows CTest 57/57; Linux GCC Debug/Release, clang-tidy and
ASan/UBSan CTest 57/57 each; PostgreSQL suites 2x Release, 2x analysis,
5x ASan; SQL and C++ mutations each caught. No Docker/Gitea CI, remote
store, process kill or deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0061 publishes environmental-v1 full checkpoints. Manifest v1 is a
fixed 280-byte record (identity, chain, lease, versions, build, zero
RNG/history declarations, durability policy, payload length/SHA-256)
that PostgreSQL parses and digests itself. Schema v3 adds region_commits
with a digest CHECK and a composite head FK, owner-managed retained
builds, fenced publish/load functions and owner-only genesis; it drops
the bare v2 head kernel and fails closed while v2 heads exist.
Linux evo::region_store adds OpenSSL SHA-256, an immutable
content-addressed filesystem object store (temp+fsync+link, no
overwrite/delete, O_NOFOLLOW exact-length verified reads), and
publication (validate -> upload -> read back -> fenced transaction ->
receipt check) and recovery (re-digest, allowlist, verify, full restore).
Portable evo::region_checkpoint holds the codec and payload validation.
Validated: Windows CTest 56/56; WSL Release, clang-tidy and ASan/UBSan
private-PostgreSQL suites; four SQL and one host mutation each caught.
M7.4 stays open for 7.4b (lineage archive codec and history binding).
No Docker/Gitea CI, remote store, process kill or deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review P1: the v2 attribute-only runtime role check accepted a login that
inherits pg_write_all_data, which can UPDATE lease rows directly and
bypass fencing. apply.sql now audits, on every migrate run, the runtime
group and all transitive members: privileged attributes, any membership
outside the group (inherited or SET-able), table privileges in evolution
(including PUBLIC), CREATE rights and EXECUTE beyond the five fenced
functions. 0002 and its checksum are unchanged. CI reproduces the bypass
and proves each configuration is rejected and rolled back.
Review P2: valid(LeasePolicy) bounds every term by the duration before
arithmetic; LeaseAuthority validates durations before deadline math and
renewal_unknown ignores impossible (>60s) and fences non-positive requests.
Extreme min/max tests reproduce the old UBSan overflow and now pass.
Validated: Windows CTest 55/55; WSL Release, clang-tidy and ASan/UBSan
private-PostgreSQL suites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0060 and schema v2 add database-time region leases and a fenced head
advance. Acquire/renew/release/advance lock the lease row, check holder,
epoch and expiry with clock_timestamp() after the lock wait, and the
advance compares the exact predecessor. Holders are DB-generated tokens;
epochs never wrap or repeat. The runtime group role gets EXECUTE on five
SECURITY DEFINER functions only - no table, provisioning or CREATE rights.
Migration history admits clean->v2, v1->v2 and a v2 no-op, and refuses
privileged runtime roles.
Host side: portable LeaseAuthority fences from send-time deadlines on
CLOCK_BOOTTIME, keeps the earlier deadline on unknown renewal and stays
fenced until a newer epoch. Selected timings: 20s lease, 5s renew, 2s
margin, 2s renew and 5s commit deadlines. Linux evo::postgres wraps the
functions with typed LeaseStoreError results.
Validated: Windows CTest 55/55; WSL Release, clang-tidy and ASan/UBSan
private-PostgreSQL suites (contenders, expiry, paused/queued stale
writers, unknown renewal, exhaustion, bypass denial, upgrade path).
Three SQL mutations were each caught. Docker/Gitea CI, remote TLS and
cluster runs were not executed. No director, host or deployment added.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>