M11-PT-001: log why a cellular tick cannot be built
Foundation / checks (network) (push) Canceled after 0s
Foundation / checks (postgres) (push) Canceled after 0s
Foundation / checks (tsan) (push) Canceled after 0s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (checks) (push) Canceled after 5s
Foundation / checks (client) (push) Canceled after 5s
Foundation / checks (deploy) (push) Canceled after 5s
Foundation / checks (ecs) (push) Successful in 2m26s

A tick whose batch the builder refuses was dropped silently and retried every 50 ms,
so a region could stop advancing with nothing in its log: after a restart DEV region 6
stayed at tick 7144 while its owner was connected. The server now prints each distinct
refusal once ("cellular tick stalled tick=N at=create|append error=E") until a tick
builds again.

Also covers a founder that buds, is suspended while its wildlife bud lives on, and
resumes on its owner's return (rules 6); that path resumes correctly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-10-16 11:35:51 -04:00
co-authored by Claude Opus 5.5
parent 926537ef3a
commit 7929dea2fc
3 changed files with 74 additions and 1 deletions
+15
View File
@@ -86,6 +86,7 @@
#include <string_view>
#include <thread>
#include <unistd.h>
#include <utility>
#include <vector>
namespace {
@@ -928,6 +929,17 @@ int serve_cellular(Options& o) {
std::optional<server::HostPhase> phase;
std::optional<std::chrono::steady_clock::time_point> next_tick;
std::vector<network::CarriedAction> awaiting_outcomes;
// A tick that cannot be built is retried every 50 ms; name each distinct cause once.
std::optional<std::pair<std::string_view, server::CellularSessionBatchError>> stalled;
const auto report_stall = [&](std::string_view where, server::CellularSessionBatchError error,
std::uint64_t tick) {
if (stalled && stalled->first == where && stalled->second == error)
return;
stalled.emplace(where, error);
std::printf("cellular tick stalled tick=%llu at=%.*s error=%u\n",
static_cast<unsigned long long>(tick), static_cast<int>(where.size()),
where.data(), static_cast<unsigned>(error));
};
const auto set_ready = [&](bool now) {
if (ready != now) {
ready = now;
@@ -1095,6 +1107,7 @@ int serve_cellular(Options& o) {
auto builder =
server::CellularSessionBatchBuilder::create(*state, host.committed()->head);
if (!builder) {
report_stall("create", builder.error(), tick);
bridge->dropped(actions, true);
} else {
auto appended = builder->append_with_birth_refusals(
@@ -1116,8 +1129,10 @@ int serve_cellular(Options& o) {
tick, baseline, actions, held_births, host.queued_mutations());
}
if (!appended) {
report_stall("append", appended.error(), tick);
bridge->dropped(actions, true);
} else {
stalled.reset();
awaiting_outcomes.insert(awaiting_outcomes.end(), actions.begin(),
actions.end());
auto published = host.publish(builder->batch(), builder->actions(),