EXPERIMENT (ADR-0094). RegionSessions owns the probe markers: admitted
intentions schedule them, each poll advances them to the simulated tick, and
snapshots carry them. evo-server sets the motion area from the region's
habitat sites; the probe prints its own marker.
The engineering client predicts its marker with the adaptive input lead,
rebases on each snapshot, resends a lost or refused intention, smooths
corrections, draws markers and follows its own with camera and interest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RegionSessions owns the replication server: interest and acks per session, a
host-set default interest, and per-tick snapshots sized to each connection's
datagram limit. evo-server builds each interest's view from the region's own
habitat sites (default: the first site, radius 7). RegionSessionClient assembles,
acknowledges and presents snapshots and can change its interest; the probe prints
them. Loopback and real-process evidence. ADR-0093.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
evo-server gains an opt-in M10 listener (--m10-*, QUIC builds only): RegionSessions
with the PostgreSQL control and action ledgers on their own connection under the
host's lease, carried actions submitted to the host and its acknowledgements and
drops turned into outcomes. evo-session-probe is a dev-only headless client. Real
process tests cover intentions, durable action outcomes, SIGKILL before and after
an action's commit, duplicate retries and the probe. The simulated watermark now
reaches every session every tick, so clients always know the server tick.
ADR-0092.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each batch's actions are recorded in the same statement as its checkpoint: one
region_actions row per (world, account, action id) and the block in an append-only
table, compared on exact retries and resolution. The region host attaches actions to
the ticks they ride and reports acknowledgements and drops, keeping unknown-outcome
batches unresolved. Outcomes come from the durable record via ActionLedger, so
crashes before or after a commit, unknown publications and cross-region moves all
resolve to one answer; a repeated id keeps its first record instead of stalling a
region. ADR-0092.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session protocol v1 (not yet deployed, amended in place) gains realtime intentions on
datagrams with receipts and a cumulative simulated watermark, and account-scoped
actions with stable ids, receipts, final outcomes and queries. IntentAdmission
enforces session binding, rate, authority, epoch, vector, sequence, tick window,
duplicates and the unresolved bound; RegionSessions and the client carry it over
QUIC. Intentions and actions have no biological effect yet. ADR-0092.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Migration v10 keeps, per world and account, the highest accepted grant serial and
the region session in control with a fencing generation and database-time expiry,
reachable only through claim/renew/release functions under the caller's region
lease. Serials survive restarts and apply across regions; a cross-region takeover
returns a wait so control never overlaps. RegionSessions takes durable control via
a ControlLedger before admitting a session, delays Welcome by the wait, renews, and
closes sessions whose control is fenced or cannot be confirmed in time.
PostgresControlLedger implements the ledger. ADR-0091.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session protocol v1 session messages (Hello with grant, Welcome, Refresh,
RefreshResult, Close with reasons) on the QUIC reliable stream. RegionSessions
verifies grants with the issuer public keys and admits them through SessionControl:
a newer grant closes the account's old connection as superseded, expiry and epoch
advances close sessions, every close is announced and flushed. RegionSessionClient
logs in, refreshes in place and reports why a session ended. ADR-0091.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first dev drill showed a full volume driving ~140 refused publications a
second. Commit retries now back off 100 ms doubling to a 2 s cap, reset on the
next confirmed commit. fault-drill.sh reports the empty-volume pod's exit code,
restarts and logs, and prints node-specific iptables rules for db-outage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An AEvolution-only Prometheus 3.13.3 and Alertmanager 0.34.1, pinned by
digest and delivered by Argo (ADR-0072), scrape each region's operations
Service through static targets (no API access, no RBAC); the region
namespaces admit port 9100 from Prometheus only. Sixteen alert rules
cover authority, lease, commits, backlog and object capacity and are
proven by promtool injection tests in CI with tools copied from the same
pinned images. Two new gauges expose object volume free space and the
reserve. Alerts route to a null receiver until a destination is chosen;
no broker is adopted and the database outbox stays.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for M7.6-M7.9. Schema v9 applies one lock order: the
barrier takes a database-wide upgrade lock exclusively, while
acquisitions, lease checks, provisioning and the retired-build trigger
take it shared first, so a queued acquisition can no longer regain
authority for a just-retired build and concurrent barriers cannot
retire the last current build. Provisioning takes a per-world lock
before classifying a world as original or a clone of one source. The
operations listener makes at most four accept attempts per serve().
Race regressions prove each contender is blocked before commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0066: evo-server --ops-listen serves /livez (loop responding, never
authority), /readyz (200 only while the host holds the lease with a
verified recovered head) and /metrics (Prometheus text: 30 fixed
series for readiness, phase, lease epoch/remaining, simulated/committed
ticks, commit lag, checkpoint age, tick debt, queue use and bounds, and
counters; the only labels are a 6-value phase and 4-value drop reason).
The dependency-free OpsServer runs on the host loop with no thread:
GET on exact paths, 405/404/400/431 responses, 8 connections with excess
refused on accept, 4 KiB requests, 1 s deadlines, 16 KiB responses.
HostStatus/phase types moved to the portable evo::ops library with a
Windows-tested renderer. evo-server now refuses to start without its
required identity and connection settings.
Adds the server-runtime Docker target (evo-server + pinned libpq,
UID 10001, no credentials), a CI server-image job with smoke script and
optional publishing, and a Kubernetes example outside Argo.
Validated: Windows CTest 59/59; Linux Debug/Release/clang-tidy/ASan
CTest 59/59; PostgreSQL suites under Release, clang-tidy and ASan with
real process probes; two ops mutations caught. Docker build/smoke and
Gitea CI not executed locally.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0065: before any readiness, recovery now verifies the head's tail.
The trace object must exist with its manifest digest, its header must
bind profile, region, predecessor and range; the predecessor (schema v6
load_region_commit) must hash to the digest the head names; replaying
the trace from the predecessor's verified payload must reproduce every
report and the exact head bytes; and trace-derived decisions must equal
the head's durable decisions. Missing, corrupt or non-reproducing tails
quarantine (RecoveryError::tail); nothing older or empty is served.
Test-only crash seams (HostConfig::crash_hook, PublishHooks) and
evo-server --crash-at deliver real SIGKILL at before-upload,
before-transaction, after-commit and after-ack. The kill matrix,
a SIGSTOP/SIGCONT stale owner and damaged-head startups all behave as
the durability contract requires; successors verify before readiness.
Validated: PostgreSQL suites under GCC Release, clang-tidy and
ASan/UBSan; tail-skip and replay-equality mutations each caught. No
Docker/Gitea CI, production-lease kill test or deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR-0064: RegionHost<State> (environmental and lineage profiles) is a
single-owner, single-threaded state machine advanced by step(now):
acquire the fenced lease and recover the verified head, run, resolve
ambiguous commits, drain and stop, or quarantine. Admission is
provisional and bounded (64 inputs, 64 KiB) with backpressure; FinalAck
and FinalOutcome are emitted only after a confirmed commit, and inputs
that will never be acknowledged are reported with a reason. Ticks follow
the caller's scheduling clock with bounded catch-up and measured tick
debt, never invented inputs. A death seals the batch and commits in the
same step, so no dependent tick uses an uncommitted outcome. Storage
outages pause and retry, unknown outcomes resolve by exact bytes,
fencing discards unconfirmed work and re-acquires; readiness fails
closed. Drain commits pending ticks and releases the lease.
evo-server hosts one region with a bounded stdin input feed, stdout
event lines, a readiness file and SIGTERM drain (exit 0/3/4).
Validated: Windows CTest 58/58; PostgreSQL suites under GCC Release,
clang-tidy and ASan/UBSan including real-process SIGTERM drain and
SIGKILL with successor recovery; four host mutations each caught. No
gateway, metrics, Docker/Gitea CI or deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>