M10.9d: record the fifth evolution-dev reference run
Foundation / checks (checks) (push) Successful in 5s
Foundation / checks (client) (push) Successful in 4s
Foundation / checks (ecs) (push) Successful in 4s
Foundation / checks (network) (push) Successful in 4s
Foundation / checks (postgres) (push) Successful in 4s
Foundation / checks (tsan) (push) Successful in 4s
Foundation / checks (deploy) (push) Successful in 3m32s
Foundation / postgres-tools (push) Successful in 5s
Foundation / server-image (push) Successful in 5s
Foundation / image (push) Successful in 5s

Image c597c0ba (batched durable lookups). Clients within budget at 20-32 sessions; server
within budgets v2 at 24, 28 and 32 (the configured session cap): work p95 <= 12.9 ms,
commit ticks <= 44.3 ms, session p99 <= 13.3 ms, no tick debt. At 20, four refusals during
the login ramp and per-session p99 0.554 ms. Saturation beyond the cap and native-client
observation remain open. Refresh the dev session window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-10-09 07:41:25 -04:00
co-authored by Claude Opus 5.5
parent 07b85b9fb3
commit 24a2d4baaf
5 changed files with 72 additions and 4 deletions
@@ -12,7 +12,7 @@ spec:
env:
- {name: REGION_SESSION, value: "dev"}
- {name: REGION_SESSION_LISTEN, value: "0.0.0.0:7400"}
- {name: REGION_SESSION_WINDOW, value: "dev-20261009T100326:1791554606"}
- {name: REGION_SESSION_WINDOW, value: "dev-20261009T113452-96b1:1791559492"}
- {name: REGION_M10_MAX_GRANT_LIFETIME, value: "14400"}
ports:
# QUIC (ADR-0090): UDP only; nothing listens on TCP 7400.
+2 -1
View File
@@ -550,7 +550,8 @@ slow peer and connect/disconnect storm with honest-service checks done; 10.9c-2
drain lease loss with 24 bots (all back within 5 s of the successor) done; no standalone
raw-UDP sender (abuse stays in-process/loopback; datagram-cap eviction cost analysed).
10.9d drill harness (load-drill.sh, server load-report, bundle-mode evo-load, Mbed TLS
crypto lock) done; the evolution-dev reference run follows.
crypto lock) done; reference runs 3-5 done (budgets v2, watermark coalescing, batched durable
lookups; within budget to the 32-session cap); saturation and native-client observation open.
**Entry:** M9 client/session evidence and M7/M8 dev/stage durability. Read
NETWORK_ARCHITECTURE, PERSISTENCE_ARCHITECTURE and ADR-0017. The M9 transport is
+14 -2
View File
@@ -128,8 +128,20 @@ logins, and responses stay at 106 ms p95. M10.9d adds the evolution-dev load dri
`evo-load` bots through the public route, and matches each step with the server's 10 s
`m10 load-report` log lines. Building it found that Mbed TLS (built without its threading
layer) raced when bots verified connections on several threads; one process-wide crypto
lock fixed it. Next: the reference run, which needs a promoted image, a load bundle and
the maintainer's go-ahead.
lock fixed it.
Reference runs on evolution-dev:
- **Third:** 24 sessions within the client budgets. It led to budgets v2 (commit ticks
≤ 50 ms, measured apart from the rest) and to watermark coalescing.
- **Fourth:** the first run whose actions committed. It exposed one synchronous store
round trip per action on the simulation thread. Lookups are now batched: one per poll
and one per commit, with no schema change.
- **Fifth:** clients within budget at 20, 24, 28 and 32 sessions. The server was within
budgets v2 at 24–32 (work p95 ≤ 12.9 ms, commit ticks ≤ 44.3 ms, no tick debt) and
reached the configured 32-session cap without saturating. At 20, the login ramp drew 4
refusals and per-session p99 was 0.554 ms (budget 0.5).
Open before 10.9 completes: saturation beyond the cap, and native-client observation.
## Previous gate - 2026-10-02
@@ -418,3 +418,45 @@ commit about every 1.05 s).
- The store cost is now at most one round trip per poll plus one per commit, whatever
the number of sessions or actions.
- **Next:** a fifth run with this fix, then the declaration.
## 10.9d fifth reference run (2026-10-09, image `c597c0ba…`)
Drill at 11:28–11:33 UTC, restored. All actions were new, and about 12 committed per
commit.
| Sessions | Client budgets | Work p95 (≤ 25) | Commit ticks (≤ 50) | Other ticks | Session p99 | Per session p99 (≤ 0.5) | Governor |
|---|---|---|---|---|---|---|---|
| 20 | met (4 ramp refusals) | 11.1 ms | 36.4 ms | 11.1 ms | 11.1 ms | 0.554 ms | refused during the login ramp only |
| 24 | met | 10.4 ms | 36.9 ms | 12.9 ms | 11.3 ms | 0.471 ms | calm |
| 28 | met | 12.5 ms | 44.3 ms | 12.8 ms | 13.3 ms | 0.473 ms | calm |
| 32 | met | 12.9 ms | 41.5 ms | 14.5 ms | 11.2 ms | 0.351 ms | calm |
Client budgets throughout:
- response p95 111 ms;
- snapshots 20 Hz with no gaps;
- outcome p95 1.13–1.14 s (the action waits for the commit of its batch);
- no starvation and no duplicate actions.
Notes on the run:
- **Batching worked.** Against the fourth run, commit ticks fell from 58–70 ms to
36–44 ms and session p99 from 30–39 ms to 11–13 ms. Tick debt stayed 0 throughout.
- **The N=20 refusals were the login ramp, not steady state.** Sixteen bots logged in
within 8 s, and each login does synchronous store work (control claim, probe open). The
window's p95 reached 15.2 ms and the governor refused 4 logins (11:28:25–28). Those bots
retried 10 s later and all 20 established. `evo-load` counts any refusal as saturation,
so it reported `sustainable=0 saturated_at=20`; the per-step results above are the
evidence.
- **Session teardown at the end shed briefly.** All 32 sessions closed at once (each close
does a release and a probe save; p95 35.6 ms), after the measured hold.
- **N=20 misses the per-session budget slightly** (0.554 ms against 0.5). The session
layer's fixed per-tick cost is shared by fewer sessions there.
- **32 is the configured session cap** (`SessionControl::max_sessions`, the budgets'
"32 default"). The region reached it within every per-tick budget without saturating.
Commit ticks are the tightest margin (44.3 of 50 ms).
Open for 10.9's completion criteria, a maintainer decision:
- measured saturation beyond the cap;
- native-client observation during a drill.
Follow-ups:
- login and close store work off the simulation thread, or batched;
- the commit off the tick (already recorded).
+13
View File
@@ -5278,3 +5278,16 @@ Executed:
- Windows `debug` 60/60, `quic` 5/5, `client-network` 15/15.
Not done: the fifth drill (needs the new image, a fresh bundle and the go-ahead).
## 2026-10-09 — M10.9d fifth evolution-dev drill
Image `c597c0ba…`, 11:28–11:33 UTC, restored Ready. Steps 20/24/28/32, 60 s holds:
- clients within budget at every step;
- server within budgets v2 at 24, 28 and 32, the configured session cap. The worst values
were work p95 12.9 ms, commit ticks 44.3 ms and session p99 13.3 ms, with no tick debt.
- At 20: 4 login refusals during the ramp, and per-session p99 0.554 ms (budget 0.5).
- Details are in ADR-0097, fifth run. Drill files were kept locally (load and server logs).
Not done:
- saturation beyond the 32-session cap;
- native-client observation during a drill.