Fix M9.4-M9.7 review findings in the dev session stack
- TLS: replace send() with a bounded queue/flush; an interrupted record is retried with identical arguments and later frames wait behind it, so no queued frame is lost or reordered; close() flushes first. - Carriers: server-issued 'I','N' | 48-bit epoch | 64-bit ordinal ids replace the aliasing low-32-bit packing; client sequences are kept separately for replies; out-of-range epochs are refused, never truncated. - Grants: a durable, fsynced, fail-closed consumed-grant ledger (--session-grant-ledger, /objects/dev-session-grants.v1 in dev) keeps single-use grants consumed across process and pod restarts. - Liveness: protocol v1 (undeployed) gains Heartbeat and the timeout close reason; both sides heartbeat when idle and treat 5 s of silence as dead. - Client sends are bounded and unresolved tracking starts at send time. - ci-client configures with an explicit C compiler. Each finding has a regression test; ADR-0083, the schema, operations doc and VALIDATION record the corrections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -28,9 +28,12 @@ Findings that constrain the design:
|
||||
**Session server in the region process.** `SessionServer` (src/network/session) is
|
||||
host-agnostic and non-blocking; `evo-server` calls it from its existing single-thread
|
||||
loop (as it does the ops server). It never mutates host state. Per connection:
|
||||
bounded output (newest view replaces an unsent one; overflow drops the connection),
|
||||
1 s Close flush deadline, 5 s accept-to-Welcome deadline, 64 KiB/s input limit,
|
||||
session bound to its connection (a connection can only speak for its own session).
|
||||
all output goes through the TLS channel's bounded queue (1 MiB; an interrupted TLS
|
||||
record is retried with identical arguments, later frames wait behind it; a full queue
|
||||
drops the slow consumer), the newest view replaces an unsent one, 1 s Close flush
|
||||
deadline, 5 s accept-to-Welcome deadline, 64 KiB/s input limit, heartbeat after 1 s
|
||||
without output and `timeout` close after 5 s without input, session bound to its
|
||||
connection (a connection can only speak for its own session).
|
||||
At most 4 connections/sessions. New epochs close existing sessions (`stale_epoch`) and
|
||||
discard intentions not yet carried.
|
||||
|
||||
@@ -42,21 +45,37 @@ committed tick as the durable watermark. Every session starts from the latest vi
|
||||
|
||||
**Intentions.** Sessions require `--synthetic-inputs development-v1`. An admitted
|
||||
intention is queued and rides the next synthetic input whose tick is at or after its
|
||||
intended tick: the input's request id becomes `INT1 | epoch | session | sequence`
|
||||
(low 32 bits each); its **content stays the host-chosen development input**. The
|
||||
intended tick under a fresh **server-issued carrier id** `'I','N' | epoch (48 bits)
|
||||
| ordinal (64 bits)`, never derived from client values; the client sequence is kept
|
||||
separately for replies. Epochs beyond 48 bits are refused at Hello (`capacity`) and
|
||||
ordinal exhaustion stops carrying; nothing wraps or truncates. The input's **content
|
||||
stays the host-chosen development input**. The
|
||||
host's FinalAck for that request yields `IntentOutcome{committed_no_effect, tick,
|
||||
commit_sequence}`; a dropped input yields `not_committed`. The driver looks ahead 8
|
||||
ticks (instead of 64) when sessions are on, matching the admission window.
|
||||
|
||||
**Protocol v1 amendment.** Before any v1 peer was deployed, v1 gained message type 8
|
||||
`IntentOutcome` and close reason 9 `capacity` (schemas/view/dev-session-v1.md). Once
|
||||
a dev deployment runs, further changes need a new version.
|
||||
**Protocol v1 amendment.** Before any v1 peer was deployed, v1 gained message types 8
|
||||
`IntentOutcome` and 9 `Heartbeat` (empty) and close reasons 9 `capacity` and 10
|
||||
`timeout` (schemas/view/dev-session-v1.md). Once a dev deployment runs, further
|
||||
changes need a new version.
|
||||
|
||||
**Grant consumption survives restarts.** Consumed grant ids are appended to a durable
|
||||
ledger (`--session-grant-ledger`; `/objects/dev-session-grants.v1` in dev, outside the
|
||||
object namespaces and ignored by backups) and fsynced **before** Welcome; a failed
|
||||
write refuses the session and keeps the grant consumed in memory. A replacement
|
||||
process loads unexpired entries into admission at startup. Corrupt content, I/O
|
||||
failure or more than 65536 entries fail closed. The ledger does not survive a
|
||||
restore-to-bundle (a rewind) or a lost volume; grants expire within an hour.
|
||||
|
||||
**Client.** `SessionClient` connects, verifies the pinned dev CA, consumes one
|
||||
single-use grant per Hello, and on Welcome resets both layers (fresh baseline). On
|
||||
loss it keeps the last presentation marked stale, reports outstanding admitted
|
||||
intentions as `unknown`, and retries with capped exponential backoff until grants run
|
||||
out (explicit `exhausted`). Intended tick = newest view tick + 4. The desktop app
|
||||
single-use grant per Hello (except after `server_shutdown`/`capacity`, which the
|
||||
server refuses before recording the grant), and on Welcome resets both layers (fresh
|
||||
baseline). It heartbeats when idle and treats 5 s without any server frame as loss.
|
||||
On loss it keeps the last presentation marked stale, reports every sent-but-unresolved
|
||||
intention as `unknown` (sent without a result, or admitted without an outcome), and
|
||||
retries with capped exponential backoff until grants run out (explicit `exhausted`).
|
||||
`send` is bounded: at most 256 unresolved intentions and the channel queue's capacity;
|
||||
beyond that it refuses rather than buffers. Intended tick = newest view tick + 4. The desktop app
|
||||
sends an intention only when its vector changes, at most once per view tick.
|
||||
`evo-client --connect ... --ca ... --grants ...` replaces the local fixture.
|
||||
|
||||
@@ -103,6 +122,14 @@ tokens: anyone holding an unused grant (or the dev key) can open a dev session.
|
||||
Restart gaps: an intention admitted just before a crash has an unknown outcome until
|
||||
the client reconnects (reported as such, never assumed).
|
||||
|
||||
## Review corrections (2026-10-02)
|
||||
The M9.4-M9.7 review (`.cache/reviews/M9.4-M9.7-review.md`, not in Git) reproduced
|
||||
and this revision fixes: TLS retry with changed arguments losing queued frames;
|
||||
carrier ids aliasing sequences 2^32 apart (the original low-32-bit packing above was
|
||||
unsafe and is withdrawn); consumed grants becoming valid again after a restart; no
|
||||
liveness after Welcome; and the `ci-client` configure missing its C compiler. Each has
|
||||
a regression test (engineering/VALIDATION.md).
|
||||
|
||||
## Consequences
|
||||
M9.8 packages the client and qualifies the presentation. M10 replaces the carrier
|
||||
with real intention semantics once organisms move, and reopens transport choice.
|
||||
|
||||
@@ -3509,3 +3509,58 @@ were not executed remotely. Port-forward reachability of the pod-loopback listen
|
||||
is the documented design, not yet observed. M8-dev/stage remains not passed. The
|
||||
connected desktop run was hidden/software-rendered; no human visual check of remote
|
||||
mode. Intentions have no biological effect by design.
|
||||
|
||||
## M9.4–M9.7 review corrections — 2026-10-02
|
||||
|
||||
Responds to the review of `bb0412c..8e31932` (`.cache/reviews/M9.4-M9.7-review.md`,
|
||||
not in Git). All five findings were confirmed against the code and fixed:
|
||||
|
||||
1. **[P1] TLS retry lost queued frames.** `TlsChannel::send` is replaced by a bounded
|
||||
(1 MiB) `queue` + `flush`: an interrupted record is retried with identical
|
||||
arguments, later frames wait behind it, `close()` flushes before close_notify, a
|
||||
full queue returns `backpressure` without queuing. Server and client route every
|
||||
frame through it. Regression (`interrupted_writes`): zero progress and partial
|
||||
progress then zero, frames queued during the stall, both directions, exact ordered
|
||||
sequences; close while stalled; backpressure.
|
||||
2. **[P1] Carrier ids aliased.** Carriers are server-issued `'I','N' | 48-bit epoch |
|
||||
64-bit ordinal`; the client sequence is kept separately. Epochs beyond 48 bits are
|
||||
refused at Hello; ordinal exhaustion stops carrying. ADR-0083's low-32-bit packing
|
||||
is withdrawn. Regression (`carriers_never_alias`): sequences 1 and 2^32+1 get
|
||||
distinct ids; `not_committed` and `committed` reach their own sequences.
|
||||
3. **[P2] Grants reusable after restart.** New durable `GrantLedger` (fsync before
|
||||
Welcome, atomic prune on open, fail closed on corruption/I-O/65536 entries), loaded
|
||||
into admission by a replacement process; `--session-grant-ledger` (dev:
|
||||
`/objects/dev-session-grants.v1`). Regressions: restart with the same ledger
|
||||
refuses the consumed grant and admits a fresh one; corrupt ledger refuses server
|
||||
creation; expired entries pruned; and in the end-to-end rehearsal the successor of
|
||||
a SIGKILLed evo-server refused the first, consumed grant (`closed reason=3`).
|
||||
4. **[P2] No liveness after Welcome.** Protocol v1 (still undeployed) gains Heartbeat
|
||||
(type 9) and Close `timeout` (10). Both sides heartbeat after 1 s without output and
|
||||
treat 5 s of silence as dead (server closes `timeout`, freeing the subject; client
|
||||
marks stale, reports unknown outcomes, reconnects). Regressions (`liveness`): idle
|
||||
but healthy session survives 1.5 s with no ticks; a silent server is detected and
|
||||
reconnected to a fresh baseline; a silent client is closed and its subject reused.
|
||||
5. **[P1, inherited] ci-client C compiler.** `-DCMAKE_C_COMPILER=clang-19` added; all
|
||||
other C-enabling fresh configures already name it.
|
||||
|
||||
Follow-ups taken: `SessionClient::send` is bounded (256 unresolved, channel queue
|
||||
capacity) and refuses instead of buffering; unresolved tracking now starts at send,
|
||||
so a loss before IntentResult is also reported as unknown. The abuse test now floods
|
||||
through a raw TLS peer (the client can no longer flood) and checks the server's
|
||||
`rate_limited` decision. Client event-vector bounds remain M9.8 work.
|
||||
|
||||
Executed after the fixes: Windows MSVC `network` 1/1 and `client-network` 6/6; Linux
|
||||
GCC 14 network suite and the **full** PostgreSQL integration suite (all suites,
|
||||
including the session process test with the ledger and new carrier ids); Clang 19
|
||||
ASan/UBSan network suite; clang-tidy 19 (warnings as errors) on all changed network,
|
||||
client and server units (one `string_view::data` finding fixed in the ledger parser);
|
||||
libFuzzer on the amended codec, 523,479 runs in 61 s, no crash or canonicality abort;
|
||||
repository format, CI selector fixtures and deploy policy (67 objects, 0 failures).
|
||||
End-to-end rehearsal: durable ack, SIGKILL, successor Ready in 3472 ms at epoch 2,
|
||||
grants kept across two `server_shutdown` refusals, second durable ack, consumed grant
|
||||
refused by the successor, Windows evo-client welcomed (`session=2 epoch=2`), successor
|
||||
drained with `exit 0`.
|
||||
|
||||
Not done: the reviewer's own reproduction binaries target the removed `send` API and
|
||||
were not rebuilt; equivalent regressions above replace them. The `ci-client` Docker
|
||||
image and Gitea pipeline were not rerun here, and no cluster operation was performed.
|
||||
|
||||
@@ -37,9 +37,13 @@ kubectl -n evolution-dev create secret generic evolution-dev-session \
|
||||
--from-file=grant.key=$HOME/aevolution-dev-session/grant.key
|
||||
```
|
||||
|
||||
The Deployment mounts it optionally (0440). Rotation: create new credentials,
|
||||
replace the Secret, restart evo-region, redistribute `dev-ca.pem`. Old grants stop
|
||||
verifying with the new key.
|
||||
The Deployment mounts it optionally (0440). Consumed grants are recorded in
|
||||
`/objects/dev-session-grants.v1` on the region's persistent volume (fsynced before
|
||||
each Welcome), so a restarted pod refuses grants its predecessor used. Corrupt or
|
||||
unwritable ledgers fail closed (`error session-grant-ledger`, or sessions refused).
|
||||
|
||||
Rotation: create new credentials, replace the Secret, restart evo-region and
|
||||
redistribute `dev-ca.pem`. Old grants stop verifying with the new key.
|
||||
|
||||
## 4. Session drill (enables sessions temporarily, then restores Git values)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user