Record cellular births in replay traces
Foundation / checks (deploy) (push) Successful in 6s
Foundation / checks (checks) (push) Failing after 39s
Foundation / checks (ecs) (push) Successful in 2m0s
Foundation / checks (postgres) (push) Failing after 52s
Foundation / checks (tsan) (push) Failing after 33s
Foundation / checks (network) (push) Failing after 3m4s
Foundation / checks (client) (push) Failing after 4m22s
Foundation / postgres-tools (push) Skipped
Foundation / server-image (push) Skipped
Foundation / image (push) Skipped

This commit is contained in:
dingo
2026-10-11 00:28:49 -04:00
parent 81866a6897
commit b4fe9e7628
13 changed files with 296 additions and 21 deletions
+9
View File
@@ -45,3 +45,12 @@ deterministic and bounded, while the complete post-tick checkpoint remains the
authoritative recovery state. Profile 1–3 trace formats and manifest size rules
remain unchanged. Process-level uncertain-commit and restart tests are still
required before M11.3d can close.
ADR-0114 extends the profile-4 trace to format 2. It retains the 112-byte
header and per-tick life intentions, then records a bounded birth set sorted by
account and request. Each birth record is 40 bytes: account (16 bytes), request
ID, control generation, and allocator-issued identity (u64 LE each). The birth
is replayed after that tick's existing-life step. Format-1 traces remain
decodable for already committed checkpoints. The new maximum is
`112 + ticks * (8 + 256 * (24 + 40))` bytes; migration 22 adjusts the profile-4
manifest bound without changing other profiles.
+8
View File
@@ -310,6 +310,14 @@ scoped change before starting the next segment.
following tick. This remains in-memory: canonical birth trace, durable
publication/recovery and session scheduling remain open, so 11.3e is not
playable yet.
- 11.3d-7h: profile-4 replay trace format 2 adds canonically ordered bounded
birth records after each tick's existing-life intentions. Replay applies each
birth at the end tick, derives all biology and placement from the predecessor,
and verifies that request identity, generation and allocator identity are
nonzero and unique. The decoder still accepts format-1 traces. C++ manifest
validation and migration 22 raise the profile-4 trace ceiling; PostgreSQL and
end-to-end birth publication tests are still pending, so this is not yet a
durable birth path or a playable 11.3e slice.
## Exit boundary
+19
View File
@@ -6398,3 +6398,22 @@ jobs remain stalled without output and do not count as validation. The current
Windows environment also lacks `clang-format`. Durable birth trace and server
integration remain unimplemented, so this does not make the cellular stage
playable.
## 2026-10-11 — M11.3d-7h cellular birth replay trace (database pending)
Profile-4 cellular trace format 2 now records a bounded birth set after each
tick's existing-life intentions. Recovery deterministically applies births at
the end tick, and the decoder retains format-1 support for old committed traces.
The C++ manifest bound was expanded; migration 22 updates the PostgreSQL
profile-4 trace ceiling and the migration runner/CI history checks.
The focused lifecycle test passes under GCC 13.2 in debug and `-DNDEBUG` builds,
including trace encode/decode, version-1 decoding, funded end-of-tick birth
replay, and malformed generation rejection. Strict-warning syntax compilation
of `src/persistence/region_checkpoint.cpp` and `git diff --check` pass. This
Windows environment has no `psql`, so migration execution and PostgreSQL
integration are pending CI. CTest, sanitizer checks, clang-format and CI have
not run for this segment. The Windows `bash.exe` shim is access-denied, and the
migration test translation unit cannot compile here because its process fixture
requires POSIX `poll.h`. The previously noted unrelated initializer warning in
`cellular_region_step.cpp:29` is demoted only for the manual focused test build.
@@ -21,6 +21,14 @@ struct CellularBatchHeader {
struct CellularTickIntents {
// One intention for each life alive at the start of this tick, sorted by life_id.
std::vector<CellularLifeIntent> lives;
// Owner-authorized births are sorted by (account, request). Biology and
// placement are derived during replay after the existing-life step.
struct Birth {
LifeAccount account{};
std::uint64_t request{}, generation{}, identity{};
friend bool operator==(const Birth&, const Birth&) = default;
};
std::vector<Birth> births;
friend bool operator==(const CellularTickIntents&, const CellularTickIntents&) = default;
};
struct CellularBatch {
@@ -35,14 +43,17 @@ struct CellularActionRecord {
friend bool operator==(const CellularActionRecord&, const CellularActionRecord&) = default;
};
inline constexpr std::uint32_t cellular_batch_format = 1;
inline constexpr std::uint32_t cellular_batch_format = 2;
inline constexpr std::uint32_t cellular_batch_legacy_format = 1;
inline constexpr std::size_t cellular_batch_header_bytes = 112;
inline constexpr std::size_t cellular_batch_tick_header_bytes = 4;
inline constexpr std::size_t cellular_batch_tick_header_bytes = 8;
inline constexpr std::size_t cellular_batch_intent_bytes = 24;
inline constexpr std::size_t cellular_batch_birth_bytes = 40;
inline constexpr std::size_t max_cellular_batch_ticks = 64;
inline constexpr std::size_t max_cellular_batch_bytes = cellular_batch_header_bytes +
max_cellular_batch_ticks * (cellular_batch_tick_header_bytes +
biology::max_cellular_tick_lives * cellular_batch_intent_bytes);
biology::max_cellular_tick_lives *
(cellular_batch_intent_bytes + cellular_batch_birth_bytes));
enum class CellularBatchError : std::uint8_t { format, version, scope, state };
inline constexpr std::size_t cellular_action_record_bytes = 48;
inline constexpr std::size_t max_cellular_action_records = 1'024;
@@ -0,0 +1,104 @@
-- M11.3d-7h, ADR-0114: allow canonical cellular birth records in bounded traces.
CREATE OR REPLACE FUNCTION evolution.parse_checkpoint_manifest(m bytea)
RETURNS evolution.checkpoint_manifest LANGUAGE plpgsql IMMUTABLE
SET search_path = pg_catalog, pg_temp AS $$
DECLARE
r evolution.checkpoint_manifest;
zero16 constant bytea := '\x00000000000000000000000000000000';
zero32 constant bytea := decode(repeat('00', 32), 'hex');
history numeric;
cap numeric;
ticks numeric;
BEGIN
IF m IS NULL OR octet_length(m) < 12 OR substring(m FROM 1 FOR 8) <> '\x4145564f52434d31' THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
IF evolution.manifest_uint(m, 8, 4) <> 2 THEN
RAISE EXCEPTION 'unsupported checkpoint manifest' USING ERRCODE = 'EVL06';
END IF;
IF octet_length(m) <> 360 THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
r.profile := evolution.manifest_uint(m, 12, 4);
history := evolution.manifest_uint(m, 232, 4);
IF NOT (
(((r.profile = 1 AND history = 0) OR (r.profile = 2 AND history = 1))
AND substring(m FROM 161 FOR 36) = '\x020000000200000001000000010000000100000001000000010000000100000001000000')
OR (r.profile = 3 AND history = 2
AND substring(m FROM 161 FOR 36) = '\x010000000100000001000000000000000000000001000000010000000100000001000000')
OR (r.profile = 4 AND history = 3
AND substring(m FROM 161 FOR 36) = '\x010000000100000001000000010000000100000002000000010000000100000000000000')
)
OR evolution.manifest_uint(m, 228, 4) <> 0 OR evolution.manifest_uint(m, 236, 4) <> 1 THEN
RAISE EXCEPTION 'unsupported checkpoint manifest' USING ERRCODE = 'EVL06';
END IF;
cap := CASE r.profile WHEN 1 THEN 1048576 WHEN 2 THEN 4268144
WHEN 4 THEN 16777216 ELSE 3266800 END;
r.world_id := evolution.manifest_uint(m, 16, 8);
r.region_id := evolution.manifest_uint(m, 24, 8);
r.frame_id := evolution.manifest_uint(m, 32, 8);
r.commit_id := encode(substring(m FROM 41 FOR 16), 'hex')::uuid;
r.commit_sequence := evolution.manifest_uint(m, 56, 8);
r.predecessor_commit := encode(substring(m FROM 65 FOR 16), 'hex')::uuid;
r.predecessor_digest := substring(m FROM 81 FOR 32);
r.begin_tick := evolution.manifest_uint(m, 112, 8);
r.end_tick := evolution.manifest_uint(m, 120, 8);
r.holder := encode(substring(m FROM 129 FOR 16), 'hex')::uuid;
r.epoch := evolution.manifest_uint(m, 144, 8);
r.tick_microseconds := evolution.manifest_uint(m, 152, 8);
r.versions := substring(m FROM 161 FOR 36);
r.build_identity := substring(m FROM 197 FOR 32);
r.payload_bytes := evolution.manifest_uint(m, 240, 8);
r.payload_sha256 := substring(m FROM 249 FOR 32);
r.trace_bytes := evolution.manifest_uint(m, 280, 8);
r.trace_sha256 := substring(m FROM 289 FOR 32);
r.decision_count := evolution.manifest_uint(m, 320, 4);
r.decisions_sha256 := substring(m FROM 329 FOR 32);
IF r.world_id = 0 OR r.region_id = 0 OR r.frame_id = 0
OR substring(m FROM 41 FOR 16) = zero16 OR r.tick_microseconds = 0
OR r.build_identity = zero32 OR r.payload_bytes < 1 OR r.payload_bytes > cap
OR r.payload_sha256 = zero32 OR evolution.manifest_uint(m, 324, 4) <> 0 THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
IF r.profile IN (3, 4) AND (r.tick_microseconds <> 50000 OR
(r.profile = 3 AND (r.decision_count <> 0 OR r.decisions_sha256 <> zero32)) OR
(r.commit_sequence = 0 AND
(substring(m FROM 65 FOR 16) <> zero16 OR r.predecessor_digest <> zero32))) THEN
RAISE EXCEPTION 'invalid lifecycle checkpoint' USING ERRCODE = '22023';
END IF;
IF r.commit_sequence = 0 THEN
IF substring(m FROM 129 FOR 16) <> zero16 OR r.epoch <> 0 OR r.begin_tick <> r.end_tick
OR r.trace_bytes <> 0 OR r.trace_sha256 <> zero32 THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
IF substring(m FROM 65 FOR 16) = zero16 AND r.predecessor_digest = zero32 THEN
IF r.end_tick <> 0 OR r.decision_count <> 0 OR r.decisions_sha256 <> zero32 THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
r.predecessor_commit := NULL;
ELSIF substring(m FROM 65 FOR 16) = zero16 OR r.predecessor_digest = zero32
OR r.predecessor_commit = r.commit_id OR r.profile IN (3, 4)
OR r.decision_count > 8
OR (r.decision_count = 0) <> (r.decisions_sha256 = zero32) THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
r.holder := NULL;
ELSE
ticks := r.end_tick - r.begin_tick;
IF substring(m FROM 65 FOR 16) = zero16 OR r.predecessor_commit = r.commit_id
OR r.predecessor_digest = zero32 OR substring(m FROM 129 FOR 16) = zero16
OR r.epoch = 0 OR r.end_tick <= r.begin_tick OR ticks > 64
OR (r.profile IN (1, 2) AND r.trace_bytes <> 96 + 304 * ticks)
OR (r.profile = 3 AND (r.trace_bytes < 112 OR r.trace_bytes > 12400
OR mod(r.trace_bytes - 112, 48) <> 0))
OR (r.profile = 4 AND (r.trace_bytes < 112 + 4 * ticks
OR r.trace_bytes > 112 + 16400 * ticks))
OR r.trace_sha256 = zero32 OR r.decision_count > 8
OR (r.decision_count = 0) <> (r.decisions_sha256 = zero32) THEN
RAISE EXCEPTION 'malformed checkpoint manifest' USING ERRCODE = '22023';
END IF;
END IF;
RETURN r;
END $$;
INSERT INTO evolution.schema_migrations(version, sha256) VALUES (22, :'migration_sha_22');
+25 -1
View File
@@ -95,27 +95,35 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
:'history' = '1:' || :'migration_sha' || ',2:' || :'migration_sha_2' || ',3:' || :'migration_sha_3' || ',4:' || :'migration_sha_4' || ',5:' || :'migration_sha_5' || ',6:' || :'migration_sha_6' || ',7:' || :'migration_sha_7' || ',8:' || :'migration_sha_8' || ',9:' || :'migration_sha_9' || ',10:' || :'migration_sha_10' || ',11:' || :'migration_sha_11' || ',12:' || :'migration_sha_12' || ',13:' || :'migration_sha_13' || ',14:' || :'migration_sha_14' || ',15:' || :'migration_sha_15' || ',16:' || :'migration_sha_16' || ',17:' || :'migration_sha_17' || ',18:' || :'migration_sha_18' || ',19:' || :'migration_sha_19' AS at_v19,
:'history' = '1:' || :'migration_sha' || ',2:' || :'migration_sha_2' || ',3:' || :'migration_sha_3' || ',4:' || :'migration_sha_4' || ',5:' || :'migration_sha_5' || ',6:' || :'migration_sha_6' || ',7:' || :'migration_sha_7' || ',8:' || :'migration_sha_8' || ',9:' || :'migration_sha_9' || ',10:' || :'migration_sha_10' || ',11:' || :'migration_sha_11' || ',12:' || :'migration_sha_12' || ',13:' || :'migration_sha_13' || ',14:' || :'migration_sha_14' || ',15:' || :'migration_sha_15' || ',16:' || :'migration_sha_16' || ',17:' || :'migration_sha_17' || ',18:' || :'migration_sha_18' || ',19:' || :'migration_sha_19' || ',20:' || :'migration_sha_20' AS at_v20,
:'history' = '1:' || :'migration_sha' || ',2:' || :'migration_sha_2' || ',3:' || :'migration_sha_3' || ',4:' || :'migration_sha_4' || ',5:' || :'migration_sha_5' || ',6:' || :'migration_sha_6' || ',7:' || :'migration_sha_7' || ',8:' || :'migration_sha_8' || ',9:' || :'migration_sha_9' || ',10:' || :'migration_sha_10' || ',11:' || :'migration_sha_11' || ',12:' || :'migration_sha_12' || ',13:' || :'migration_sha_13' || ',14:' || :'migration_sha_14' || ',15:' || :'migration_sha_15' || ',16:' || :'migration_sha_16' || ',17:' || :'migration_sha_17' || ',18:' || :'migration_sha_18' || ',19:' || :'migration_sha_19' || ',20:' || :'migration_sha_20' || ',21:' || :'migration_sha_21' AS at_v21 \gset
\if :at_v21
SELECT right(:'history', length(',22:' || :'migration_sha_22')) = ',22:' || :'migration_sha_22' AS at_v22 \gset
\if :at_v22
\elif :at_v21
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v20
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v19
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v18
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v17
\ir 0018_cellular_profile_admission.sql
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v16
\ir 0017_life_request_outcomes.sql
\ir 0018_cellular_profile_admission.sql
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v15
\ir 0016_primitive_publication.sql
\ir 0017_life_request_outcomes.sql
@@ -123,6 +131,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v14
\ir 0015_life_reservations.sql
\ir 0016_primitive_publication.sql
@@ -131,6 +140,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v13
\ir 0014_probe_overdue.sql
\ir 0015_life_reservations.sql
@@ -140,6 +150,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v12
\ir 0013_probe_incoming.sql
\ir 0014_probe_overdue.sql
@@ -150,6 +161,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v11
\ir 0012_probe_handoff.sql
\ir 0013_probe_incoming.sql
@@ -161,6 +173,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v10
\ir 0011_region_actions.sql
\ir 0012_probe_handoff.sql
@@ -173,6 +186,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v9
\ir 0010_account_control.sql
\ir 0011_region_actions.sql
@@ -186,6 +200,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v8
\ir 0009_serialized_authority.sql
\ir 0010_account_control.sql
@@ -200,6 +215,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v7
\ir 0008_upgrade_barrier.sql
\ir 0009_serialized_authority.sql
@@ -215,6 +231,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v6
\ir 0007_region_clones.sql
\ir 0008_upgrade_barrier.sql
@@ -231,6 +248,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v5
\ir 0006_commit_lookup.sql
\ir 0007_region_clones.sql
@@ -248,6 +266,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v4
\ir pre_v5_guard.sql
\ir 0005_batch_decisions.sql
@@ -267,6 +286,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v3
\ir 0004_lineage_profile.sql
\ir pre_v5_guard.sql
@@ -287,6 +307,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v2
\ir 0003_region_checkpoints.sql
\ir 0004_lineage_profile.sql
@@ -308,6 +329,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\elif :at_v1
\ir 0002_region_fencing.sql
\ir 0003_region_checkpoints.sql
@@ -330,6 +352,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\else
\echo 'Unknown or changed migration history'
DO $$ BEGIN RAISE EXCEPTION 'unknown or changed migration history'; END $$;
@@ -357,6 +380,7 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
\ir 0019_cellular_action_records.sql
\ir 0020_cellular_publication.sql
\ir 0021_cellular_trace_and_recovery.sql
\ir 0022_cellular_birth_trace_bound.sql
\endif
-- Effective-privilege audit on every run, including no-ops (ADR-0060). Attributes
-- alone are insufficient: predefined roles such as pg_write_all_data, PUBLIC
+3 -3
View File
@@ -94,12 +94,12 @@ bool structural(const CheckpointManifest& m) noexcept {
if (m.manifest_version == 1)
return link && no_batch;
const auto ticks = m.end_tick - m.begin_tick;
const auto cellular_min = server::cellular_batch_header_bytes +
ticks * server::cellular_batch_tick_header_bytes;
const auto cellular_min = server::cellular_batch_header_bytes + ticks * 4;
const auto cellular_max = server::cellular_batch_header_bytes +
ticks * (server::cellular_batch_tick_header_bytes +
biology::max_cellular_tick_lives *
server::cellular_batch_intent_bytes);
(server::cellular_batch_intent_bytes +
server::cellular_batch_birth_bytes));
const bool trace_shape = primitive
? b.trace_bytes >= server::primitive_batch_header_bytes &&
b.trace_bytes <= server::max_primitive_batch_bytes &&
+70 -7
View File
@@ -5,6 +5,7 @@
#include <bit>
#include <optional>
#include <ranges>
#include <set>
#include <tuple>
#include <utility>
@@ -39,8 +40,11 @@ bool valid_intent(const CellularLifeIntent& life) {
step.movement.y <= movement_limit;
}
bool valid_ticks(std::span<const CellularTickIntents> ticks) {
std::set<std::uint64_t> identities;
std::set<LifeAccount> accounts;
for (const auto& tick : ticks) {
if (tick.lives.size() > biology::max_cellular_tick_lives)
if (tick.lives.size() > biology::max_cellular_tick_lives ||
tick.births.size() > biology::max_cellular_tick_lives)
return false;
for (std::size_t i = 0; i < tick.lives.size(); ++i) {
const auto& life = tick.lives[i];
@@ -48,6 +52,17 @@ bool valid_ticks(std::span<const CellularTickIntents> ticks) {
(i > 0 && tick.lives[i - 1].life_id >= life.life_id))
return false;
}
for (std::size_t i = 0; i < tick.births.size(); ++i) {
const auto& birth = tick.births[i];
if (all_zero(birth.account) || birth.request == 0 || birth.generation == 0 ||
birth.identity == 0 ||
(i > 0 && std::tie(tick.births[i - 1].account,
tick.births[i - 1].request) >=
std::tie(birth.account, birth.request)))
return false;
if (!identities.insert(birth.identity).second || !accounts.insert(birth.account).second)
return false;
}
}
return true;
}
@@ -107,7 +122,8 @@ encode_cellular_batch(const CellularBatch& batch) {
return std::unexpected{CellularBatchError::format};
std::size_t size = cellular_batch_header_bytes;
for (const auto& tick : batch.ticks) {
const auto records = tick.lives.size() * cellular_batch_intent_bytes;
const auto records = tick.lives.size() * cellular_batch_intent_bytes +
tick.births.size() * cellular_batch_birth_bytes;
if (records > max_cellular_batch_bytes - size - cellular_batch_tick_header_bytes)
return std::unexpected{CellularBatchError::format};
size += cellular_batch_tick_header_bytes + records;
@@ -129,8 +145,15 @@ encode_cellular_batch(const CellularBatch& batch) {
put(out, 0, 4);
for (const auto& tick : batch.ticks) {
put(out, tick.lives.size(), 4);
put(out, tick.births.size(), 4);
for (const auto& life : tick.lives)
put_intent(out, life);
for (const auto& birth : tick.births) {
out.insert(out.end(), birth.account.begin(), birth.account.end());
put(out, birth.request, 8);
put(out, birth.generation, 8);
put(out, birth.identity, 8);
}
}
if (out.size() != size)
return std::unexpected{CellularBatchError::format};
@@ -142,7 +165,9 @@ decode_cellular_batch(std::span<const std::uint8_t> bytes) {
if (bytes.size() < cellular_batch_header_bytes || bytes.size() > max_cellular_batch_bytes ||
!std::ranges::equal(bytes.first(magic.size()), magic))
return std::unexpected{CellularBatchError::format};
if (get(bytes, 8, 4) != cellular_batch_format || get(bytes, 12, 4) != 4)
const auto format = get(bytes, 8, 4);
if ((format != cellular_batch_format && format != cellular_batch_legacy_format) ||
get(bytes, 12, 4) != 4)
return std::unexpected{CellularBatchError::version};
if (get(bytes, 108, 4) != 0)
return std::unexpected{CellularBatchError::format};
@@ -166,12 +191,21 @@ decode_cellular_batch(std::span<const std::uint8_t> bytes) {
std::size_t offset = cellular_batch_header_bytes;
out.ticks.reserve(static_cast<std::size_t>(tick_count));
for (std::uint64_t tick_index = 0; tick_index < tick_count; ++tick_index) {
if (bytes.size() - offset < cellular_batch_tick_header_bytes)
const auto tick_header_bytes = format == cellular_batch_format
? cellular_batch_tick_header_bytes
: sizeof(std::uint32_t);
if (bytes.size() - offset < tick_header_bytes)
return std::unexpected{CellularBatchError::format};
const auto count = get(bytes, offset, 4);
offset += cellular_batch_tick_header_bytes;
offset += 4;
std::uint64_t birth_count{};
if (format == cellular_batch_format) {
birth_count = get(bytes, offset, 4);
offset += 4;
}
if (count > biology::max_cellular_tick_lives ||
count > (bytes.size() - offset) / cellular_batch_intent_bytes)
count > (bytes.size() - offset) / cellular_batch_intent_bytes ||
birth_count > biology::max_cellular_tick_lives)
return std::unexpected{CellularBatchError::format};
CellularTickIntents step;
step.lives.reserve(static_cast<std::size_t>(count));
@@ -198,6 +232,25 @@ decode_cellular_batch(std::span<const std::uint8_t> bytes) {
step.lives.push_back(life);
offset += cellular_batch_intent_bytes;
}
if (birth_count > (bytes.size() - offset) / cellular_batch_birth_bytes)
return std::unexpected{CellularBatchError::format};
step.births.reserve(static_cast<std::size_t>(birth_count));
for (std::uint64_t i = 0; i < birth_count; ++i) {
const auto record = bytes.subspan(offset, cellular_batch_birth_bytes);
CellularTickIntents::Birth birth;
std::ranges::copy(record.first(16), birth.account.begin());
birth.request = get(record, 16, 8);
birth.generation = get(record, 24, 8);
birth.identity = get(record, 32, 8);
if (all_zero(birth.account) || birth.request == 0 || birth.generation == 0 ||
birth.identity == 0 ||
(!step.births.empty() &&
std::tie(step.births.back().account, step.births.back().request) >=
std::tie(birth.account, birth.request)))
return std::unexpected{CellularBatchError::format};
step.births.push_back(birth);
offset += cellular_batch_birth_bytes;
}
out.ticks.push_back(std::move(step));
}
if (offset != bytes.size())
@@ -217,9 +270,19 @@ replay_cellular_batch(const CellularLives& base, const CellularBatch& batch) {
base.tick());
if (!replay)
return std::unexpected{CellularBatchError::state};
for (const auto& tick : batch.ticks)
for (const auto& tick : batch.ticks) {
if (!(*replay)->advance_tick(tick.lives))
return std::unexpected{CellularBatchError::state};
for (const auto& birth : tick.births) {
const LifeRequest request{birth.account, birth.request};
const LifeAllocation allocation{birth.identity, birth.identity, birth.identity,
birth.identity, {(*replay)->tick()}};
auto born = (*replay)->birth_candidate(request, allocation);
if (!born)
return std::unexpected{CellularBatchError::state};
*replay = std::move(*born);
}
}
return std::move(*replay);
}
+32
View File
@@ -94,6 +94,38 @@ void reserved_birth_is_funded_and_detached() {
CHECK((*born)->advance_tick(next_tick) && (*born)->tick() == 2 &&
(*born)->lives().size() == 1);
auto trace_base = empty_cellular_owner(founder_price(1), {{0, 0}});
CellularBatch birth_batch;
birth_batch.header = {trace_base->world(), trace_base->region(), trace_base->frame(), {}, {},
trace_base->tick(), trace_base->tick() + 1};
birth_batch.header.predecessor.fill(3);
birth_batch.header.predecessor_digest.fill(4);
CellularTickIntents birth_tick;
birth_tick.births.push_back({request.account, request.action, 1, 801});
birth_batch.ticks.push_back(birth_tick);
const auto birth_bytes = encode_cellular_batch(birth_batch);
CHECK(birth_bytes);
const auto decoded_births = decode_cellular_batch(*birth_bytes);
CHECK(decoded_births && *decoded_births == birth_batch);
const auto replayed_birth = replay_cellular_batch(*trace_base, *decoded_births);
CHECK(replayed_birth && (*replayed_birth)->tick() == 1 &&
(*replayed_birth)->lives().size() == 1 &&
(*replayed_birth)->lives()[0].identity.life == 801 &&
(*replayed_birth)->lives()[0].account == request.account &&
(*replayed_birth)->archive().recorded() == 1 &&
(*replayed_birth)->current_total() == (*replayed_birth)->opening_total());
auto legacy_batch = birth_batch;
legacy_batch.ticks[0].births.clear();
auto legacy_bytes = encode_cellular_batch(legacy_batch);
CHECK(legacy_bytes);
(*legacy_bytes)[8] = static_cast<std::uint8_t>(cellular_batch_legacy_format);
legacy_bytes->erase(legacy_bytes->begin() + 116, legacy_bytes->begin() + 120);
const auto decoded_legacy = decode_cellular_batch(*legacy_bytes);
CHECK(decoded_legacy && decoded_legacy->ticks[0].births.empty());
auto invalid_birth_batch = birth_batch;
invalid_birth_batch.ticks[0].births[0].generation = 0;
CHECK(encode_cellular_batch(invalid_birth_batch).error() == CellularBatchError::format);
auto unfunded = empty_cellular_owner({}, {{0, 0}});
CHECK(unfunded->advance_tick(no_lives));
const auto unfunded_snapshot = unfunded->snapshot();
+3 -1
View File
@@ -187,6 +187,7 @@ void upgrade_across_migration_and_barrier() {
" -v migration_sha_19=$(s 0019_cellular_action_records.sql)"
" -v migration_sha_20=$(s 0020_cellular_publication.sql)"
" -v migration_sha_21=$(s 0021_cellular_trace_and_recovery.sql)"
" -v migration_sha_22=$(s 0022_cellular_birth_trace_bound.sql)"
" -v runtime_role=evo_runtime_test -f migrations/apply.sql"
" > /dev/null 2>&1") != 0);
CHECK(history(*owner_db) == "1,2,3,4,5,6,7");
@@ -202,7 +203,8 @@ void upgrade_across_migration_and_barrier() {
CHECK(shell("PGDATABASE=" + database +
" EVO_PG_RUNTIME_ROLE=evo_runtime_test bash \"$EVO_SOURCE_ROOT/"
"tools/migrate-postgres.sh\" > /dev/null") == 0);
CHECK(history(*owner_db) == "1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21");
CHECK(history(*owner_db) ==
"1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22");
CHECK(scalar(*owner_db, "SELECT count(*) FROM information_schema.columns WHERE "
"table_schema = 'evolution' AND table_name = 'region_actions' AND "
"column_name IN ('input_bytes', 'result')") == "2");
+5 -5
View File
@@ -69,7 +69,7 @@ PGDATABASE=upgrade_path psql -X -v ON_ERROR_STOP=1 -1 -v "migration_sha=$v1_sha"
-f "$root/migrations/0001_region_bootstrap.sql" > /dev/null
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',') FROM evolution.schema_migrations")" = 1
PGDATABASE=upgrade_path bash "$root/tools/migrate-postgres.sh"
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22
# v2 heads have no payload binding: v2->v3 fails closed and leaves history at v2.
v2_sha="$(sha256sum "$root/migrations/0002_region_fencing.sql")"
v2_sha="${v2_sha%% *}"
@@ -90,7 +90,7 @@ PGDATABASE=v3_commits psql -X -v ON_ERROR_STOP=1 \
-c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \
-c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '01000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32), 'hex'))" > /dev/null
PGDATABASE=v3_commits bash "$root/tools/migrate-postgres.sh"
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 21:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) || ':' || string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 22:1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22
test "$(PGDATABASE=v3_commits psql -XAtc "SELECT count(*) FROM evolution.region_commits")" = 1
# Review P1: a v4 database holding a published (possibly terminal) checkpoint has
# no durable decisions for it. The v5 upgrade must refuse before changing anything.
@@ -144,7 +144,7 @@ PGDATABASE=v6_populated psql -X -v ON_ERROR_STOP=1 \
-c "INSERT INTO evolution.retained_builds VALUES (decode(repeat('b1', 32), 'hex'))" \
-c "SELECT evolution.provision_region_checkpoint(decode('4145564f52434d31' || '02000000' || '01000000' || '0100000000000000' || '0100000000000000' || '0900000000000000' || repeat('11', 16) || repeat('00', 8) || repeat('00', 16) || repeat('00', 32) || repeat('00', 16) || repeat('00', 16) || repeat('00', 8) || '50c3000000000000' || '020000000200000001000000010000000100000001000000010000000100000001000000' || repeat('b1', 32) || '00000000' || '00000000' || '01000000' || '6400000000000000' || repeat('a0', 32) || repeat('00', 80), 'hex'))" > /dev/null
PGDATABASE=v6_populated bash "$root/tools/migrate-postgres.sh"
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT count(*) FROM evolution.region_commits c, LATERAL evolution.parse_checkpoint_manifest(c.manifest) m WHERE m.commit_sequence = 0 AND m.predecessor_commit IS NULL")" = 1
test "$(PGDATABASE=v6_populated psql -XAtc "SELECT string_agg(conname, ',') FROM pg_constraint WHERE conrelid = 'evolution.region_heads'::regclass AND contype = 'c' AND pg_get_constraintdef(oid) LIKE '%end_tick%'")" = region_heads_published_tick
# Inherited privileges bypass fencing, so every migrate run audits effective rights.
@@ -190,9 +190,9 @@ expect_audit_failure 'GRANT SELECT ON evolution.account_controls TO evo_runtime_
expect_audit_failure 'REVOKE EXECUTE ON FUNCTION evolution.claim_account_control(evolution.u64, evolution.u64, uuid, evolution.u64, uuid, evolution.u64, evolution.u64, integer) FROM evo_runtime_test' 'GRANT EXECUTE ON FUNCTION evolution.claim_account_control(evolution.u64, evolution.u64, uuid, evolution.u64, uuid, evolution.u64, evolution.u64, integer) TO evo_runtime_test'
bash "$root/tools/migrate-postgres.sh" # Restored configuration passes the audit again.
# Unknown future versions and checksum drift must not be repaired silently.
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=22 WHERE version=21'
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=23 WHERE version=22'
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=21 WHERE version=22"
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET version=22 WHERE version=23"
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=14"
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
psql -X -v ON_ERROR_STOP=1 -c "UPDATE evolution.schema_migrations SET sha256=repeat('0',64) WHERE version=13"
+1 -1
View File
@@ -61,7 +61,7 @@ SQL
[ "$code" != 0 ] && grep -q 'division by zero' "$work/migration.log" || fail "the rehearsal migration did not fail"
[ "$(q -c "SELECT count(*) FROM information_schema.columns WHERE table_name = 'region_commits' AND column_name = 'rehearsal_column'")" = 0 ] ||
fail "the failed migration left a column"
[ "$(q -c 'SELECT max(version) FROM evolution.schema_migrations')" = 21 ] || fail "schema version moved"
[ "$(q -c 'SELECT max(version) FROM evolution.schema_migrations')" = 22 ] || fail "schema version moved"
wait_ack "$work/b1.log" $((before + 40)) || fail "b1 did not resume after the failed migration"
kill -0 "$b1_pid" 2> /dev/null || fail "b1 exited during the failed migration"
bash "$root/tools/migrate-postgres.sh" > /dev/null || fail "migrate after the failure"
+3
View File
@@ -54,6 +54,8 @@ digest_20="$(sha256sum "$root/migrations/0020_cellular_publication.sql")"
digest_20="${digest_20%% *}"
digest_21="$(sha256sum "$root/migrations/0021_cellular_trace_and_recovery.sql")"
digest_21="${digest_21%% *}"
digest_22="$(sha256sum "$root/migrations/0022_cellular_birth_trace_bound.sql")"
digest_22="${digest_22%% *}"
psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \
--set="migration_sha_2=$digest_2" --set="migration_sha_3=$digest_3" \
--set="migration_sha_4=$digest_4" --set="migration_sha_5=$digest_5" \
@@ -68,5 +70,6 @@ psql -X --no-password --set=ON_ERROR_STOP=1 --set="migration_sha=$digest" \
--set="migration_sha_19=$digest_19" \
--set="migration_sha_20=$digest_20" \
--set="migration_sha_21=$digest_21" \
--set="migration_sha_22=$digest_22" \
--set="runtime_role=$runtime_role" \
--file="$root/migrations/apply.sql"