Fix load governor startup refusal and drill issues from the first dev run
Foundation / checks (deploy) (push) Successful in 3m24s
Foundation / checks (ecs) (push) Successful in 2m0s
Foundation / checks (client) (push) Successful in 12m24s
Foundation / checks (network) (push) Successful in 13m35s
Foundation / checks (tsan) (push) Successful in 7m58s
Foundation / checks (checks) (push) Successful in 32m32s
Foundation / checks (postgres) (push) Successful in 45m25s
Foundation / postgres-tools (push) Successful in 17s
Foundation / image (push) Successful in 2m32s
Foundation / server-image (push) Successful in 4m15s

ADR-0097. The first evolution-dev load drill (image 6c96629e) did not
measure capacity: the governor refused six seconds after the restart with
no session (p95 over a window of only startup ticks), and the bots spent
their grants retrying every 2 s. Reference numbers with one session: tick
work p95 4.0 ms with 33 ms spikes, session layer p95 0.68 ms.

- The governor acts on work only over a full 40-tick window; tick debt
  still counts at once.
- evo-load backs off 10 s after a load refusal and, in bundle mode,
  reuses the grant the region refused before using it.
- The drill waits for a calm load-report before the bots start, takes the
  drill pod's whole log before restoring (the live follow died), and
  confirms the restore against a pod other than the drill pod.

Refreshes the dev session window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-10-08 11:01:02 -04:00
co-authored by Claude Opus 5.5
parent 9b868b3ae2
commit d173f076ea
7 changed files with 120 additions and 21 deletions
+13
View File
@@ -154,6 +154,19 @@ void load_governor() {
CHECK(!LoadGovernor::create({19'000, 15'000, 20'000, 10'000, 1'000'000, 5'000'000, 3}));
CHECK(!LoadGovernor::create({25'000, 15'000, 20'000, 10'000, 0, 5'000'000, 3}));
CHECK(!LoadGovernor::create({25'000, 15'000, 20'000, 10'000, 1'000'000, 5'000'000, 4}));
// After a (re)start, work counts only once the window is full (seen on evolution-dev:
// the first slow ticks alone tripped refusal before any session). Tick debt counts at once.
{
auto fresh = *LoadGovernor::create({});
std::uint64_t t = 0;
for (int i = 0; i < 39; ++i)
CHECK(!fresh.observe(30'000, 1'000, 0, t += 50'000) && !fresh.state().refusing);
CHECK(fresh.state().p95_us == 30'000);
CHECK(fresh.observe(30'000, 1'000, 0, t += 50'000) && fresh.state().refusing &&
fresh.state().shed == 1);
auto debt = *LoadGovernor::create({});
CHECK(debt.observe(1'000, 100, 2, 50'000) && debt.state().refusing);
}
auto made = LoadGovernor::create({});
CHECK(made.has_value());
auto& g = *made;