Files
aevolution/docs/engineering/TEST_STRATEGY.md
T
dingo 0515702964
Foundation / checks (deploy) (push) Successful in 5s
Foundation / checks (ecs) (push) Successful in 2m25s
Foundation / checks (network) (push) Successful in 20m44s
Foundation / checks (client) (push) Successful in 21m57s
Foundation / checks (tsan) (push) Successful in 22m46s
Foundation / postgres-tools (push) Canceled after 0s
Foundation / server-image (push) Canceled after 0s
Foundation / image (push) Canceled after 0s
Foundation / checks (checks) (push) Canceled after 30m2s
Foundation / checks (postgres) (push) Canceled after 29m56s
M12.4: retain read-only destination migration stages
2026-10-18 09:32:50 -04:00

6.9 KiB

Validation strategy

M12.4f's checkpoint contract exercises explicit portable bundle-9 to 10 upgrades for raw owners and a real nonempty cohort with retained policy/sequence. It requires complete candidate equality, canonical restore, continued batch-2 biology and bundle-preserving representation changes. Negative cases cover old/new payload confusion, malformed source/lease/digest bindings, exhaustion, repeated upgrade and new genesis. The private PostgreSQL population suite must continue refusing these portable candidates through schema 45 and existing prepared/generic publication paths without advancing the head. Durable admission and upgrade crash/retry recovery remain separate gates.

M12.4g's private PostgreSQL suite upgrades raw/profile-5 bundle-9 sources using schema 46. It rehearses raw upgrade/trace-3 continuation and rollback, retains a nonempty cohort, refuses pending-interval upgrades and wrong publication paths, and recovers after a pre-transaction SIGKILL and an ambiguous exact retry. Changed requests and stale writers fail. Missing/corrupt objects quarantine; a canonical but unequal cohort with matching biological totals fails full upgrade replay. Batch-1/2 cross-binding is refused in both directions. The process-loss and host-recreation/materialization matrix also runs with batch 2. Full historical migration/backup/DR/permission regression remains required for schema admission.

M12.4h's private freeze matrix uses funded player-derived wildlife and both bundle-10 representations. It requires unchanged source/destination heads and stock, exact request retries, conflicting/new-work refusal, pending-choice/ interval guards, new-epoch recovery, explicit abort without reopening, and real SIGKILL before/after preparation. Unknown preparation/abort replies pause the owning host through exact reconciliation; drain/recreation retains the freeze. Missing and same-length corrupt capsules fail, and canonical false material fails full biological replay. SQL checksum rejection is checked independently of the C++ decoder. The metric has one fixed unlabelled series and remains bounded. These are preparation/abort checks, not proof of destination activation or atomic paired publication; full historical persistence regression is also required.

M12.4i's private staging matrix covers both bundle-10 representations and actual player-origin wildlife. It requires unchanged biological heads, complete ghost replay, exact/conflicting retries, refusal of moved destination heads and pending intervals, publication/profile/life-reservation guards and exclusive endpoint roles. Stale destination epochs fail; replacement retains the read-only ghost. Missing source capsules, missing/same-length corrupt ghosts and valid but false ghost objects fail recovery. Real lock-timeout ambiguity, SIGKILL before/after stage commit and independent-connection stage/abort races preserve one outcome. The owning destination host retains exact uncertain work across restart and resumes only its original biology after confirmed source abort. Full schema, permission, backup/DR and historical regression remain required. These tests do not qualify paired commit, destination activation or linked ecological movement.

Tests defend contracts, not directory coverage. All checks remain active in Release builds. Current proof has no actual network, biology or distributed recovery, so those categories below are milestone gates rather than passing test labels attached to empty targets.

Category Foundation evidence Future gate
Unit PRNG reference vectors, entity lifecycle, capacity/error boundaries Genome development and energetic transfer rules
Deterministic regression Frozen canonical fingerprint plus independent reference calculation; uninterrupted vs restored execution Versioned ecosystem baseline ensembles
Property-style Many explicit seeds and tick split points; coordinate and ID invariants Conservation, nonnegative mass/population and phenotype prerequisites
Serialization Exact byte round-trip, malformed lengths/versions/checksum/IDs/coordinates; validate before publication Compatible migrations, verified object archive and replay range
Integration CLI seed/ticks and replay verification; file save/load comparison Durable death/reproduction acknowledgment and worker process kill
Fuzz Optional libFuzzer snapshot decoder with bounded input and sanitizer instrumentation Authenticated protocol/parser and migration fuzzing
Static/sanitizers clang-format/tidy; ASan/UBSan and separate TSan configurations TSan on real scheduler and I/O lifecycle
Benchmark/load Seed/count/tick/time/compiler report for kernel; no noisy portable time threshold 20+ clients, region load, interest bandwidth, recovery backlog
Biological regression Absent until M2; do not call synthetic particles an ecosystem Million-tick balances, predator/prey cycles, starvation, diversity and extinction
Disaster recovery Absent until persistent services exist Off-host verified backup restore; database/object consistency; old owner fencing

Reference fixture, rules/format 1: seed 123456, create 8 particles at (0,0), then 1,000 ticks yields canonical payload FNV-1a d61ad5bf0c928c76. Expected positions by ID: (49,10), (985,993), (39,33), (1008,0), (16,1), (7,3), (1004,1007), (995,15). The initial value was computed independently in JavaScript with explicit wrapping BigInt arithmetic, not copied from the C++ test result. SplitMix64 seed-zero vectors begin e220a8397b1dcdaf, 6e789e6aa1b965f4, 06c45d188009454f, f88bb8a8724c81ec.

Changing a golden requires a stated rule/format change, compatibility decision, independent review and updated documentation. A new compiler or optimization mode must reproduce existing integer fixtures. Floating-point biology later needs error envelopes and ensemble comparisons in addition to exact fixtures.

Unexecuted configurations are not validation. Record exact tools, commands, results and limitations in VALIDATION.md at the end of each foundation change.

The cohort qualification runs its complete matrix under sanitizers; its wall timeout is a bounded CI execution guard, not a portable performance claim. Budgets are 180 s normally, 600 s with ASan/UBSan and 900 s with TSan. The larger migration contract gets 180/360/600 s respectively. Cases, seeds, checks and instrumentation remain identical. The checks job allows 90 minutes for the full cold/shared-runner build, instrumented suites and remaining fuzz/benchmark steps.

Population and migration restart scripts honor the runtime EVO_TEST_TMPDIR override and create distinct random fixture subdirectories there. Without the override they retain the configured build-directory output paths. The non-root, networkless, read-only TSan container sets the override to its existing bounded /tmp tmpfs; no source/build mount is made writable and no sandbox permission is broadened. Fixture path randomness has no biological or test-result meaning.