Files
aevolution/docs/engineering/VALIDATION.md
T

9.3 KiB

Validation report

Recorded 2026-09-19. Latest verified Linux delivery: run 1402 passed on 4c2cc04ac47d37c6191a9ef02d4bbba716a930dc. Native checks apply to base commit 8ab48d3 plus the foundation-delivery working changes committed with this report. Simulation rules and snapshot format remain version 1; the only C++ change is benchmark metadata. Results below distinguish executed checks from pending Linux/cluster gates.

Tools and host

  • Windows NT 10.0.26200, AMD Ryzen 7 9800X3D, 8 cores / 16 logical processors.
  • MSVC 19.44.35217.0, CMake 3.31.6-msvc6, Ninja 1.12.1.
  • clang-format and clang-tidy 19.1.5 (Visual Studio LLVM tools).
  • kubectl 1.31.4 / Kustomize 5.4.2 for offline manifest rendering.

Executed native checks

Command Observed result
./tools/windows-check.ps1 -Preset debug -Action check Configure/build passed; 5/5 CTests passed
./tools/windows-check.ps1 -Preset release -Action check Configure/build passed; 5/5 CTests passed under NDEBUG
./tools/windows-check.ps1 -Preset analysis -Action check clang-tidy/warnings-as-errors build passed; 5/5 CTests passed
./tools/windows-check.ps1 -Preset asan -Action check MSVC AddressSanitizer build passed; 5/5 CTests passed; this is NOT UBSan
./tools/windows-check.ps1 -Preset debug -Action format Passed with clang-format 19; repeated twice in one shell after helper fix
./build/release/evo-sim --seed 123456 --entities 8 --ticks 1000000 --verify-replay Exit 0; replay_verified=true; hash fab22c2e8ae1c379
./build/release/evo-benchmark Exit 0; MSVC 1944 Release, seed 123456, 10000 entities, 1000 ticks, 0.0210666 seconds, hash 3bad6db1612cbd1
Git Bash bash -n tools/ci/linux-check.sh and bash -n tools/ci/publish-image.sh Syntax checks passed; no Linux execution implied
kubectl kustomize deploy/kubernetes/dev Rendered Job, ServiceAccount and NetworkPolicy in evolution-dev; final render pins the verified run-1402 image

CTest covers the independently frozen 1000-tick fingerprint d61ad5bf0c928c76, seeded replay properties, malformed/semantic snapshot rejection, entity lifecycle, capacity/overflow, telemetry and CLI file continuation/alias protection. The expected-failure harness confirms tests remain active in Release.

The benchmark is a single uncontrolled wall-clock observation, not a performance threshold or an MMO capacity estimate. Its initial compiler output accidentally used hexadecimal; the stream now returns to decimal and reports MSVC 1944.

Gitea evidence

The maintainer's run 1399 on 8ab48d3a5639121693f5881b600482e870b49f05 completed successfully. API job records confirm verify (configure/build/CTest) and publish (Docker build/push) succeeded on runner theraborn-runners-runner-0, label ubuntu-22.04. This is evidence for the original workflow, not the expanded sanitizer/analysis jobs added in this change. Exact Linux tool versions and expanded-run results appear below; a runner label is not a compiler version.

Limitations and next gates

  • Local Docker Desktop has no running Linux engine. Container checks must run on the configured Gitea runner; no local container success is claimed.
  • Expanded Linux GCC14/Clang19, ASan/UBSan, TSan and libFuzzer evidence is recorded below. TSan currently validates a sequential kernel, not future concurrency.
  • Sandbox process restrictions stalled native Ninja and blocked Git Bash signal pipes. Those attempts were stopped; checks reran successfully outside sandbox.
  • Repeated VsDevCmd initialization grew PATH beyond cmd's limit. The helper now reuses the matching x64 environment and deduplicates PATH; repeated checks pass.
  • The maintainer reports Argo setup, but this workspace has no Kubernetes context. No dev Job completion or cluster admission result is recorded yet.
  • No biology, network/load, durable server recovery or backup restore exists. Those remain M2, M7/M8 and later gates, not failing or passing foundation tests.

Update this report with run URL, tested commit, image digest and Job result when those checks execute. Never overwrite a golden merely to make CI pass.

Expanded Linux run 1400

Run 1400, commit 9e412736d9: GCC14 Debug and Release, Clang19 analysis and ASan/UBSan each passed 5/5 tests. Formatting passed. The seeded fuzz smoke completed 10000 cases with no reported failure. CMake 3.28.3, GCC 14.2.0, Clang/format/tidy 19.1.1, Linux 6.8.0-110-generic x86-64. Million-tick hash matched Windows (fab22c2e8ae1c379); benchmark hash matched too (3bad6db1612cbd1), with 0.0674131 seconds observed for 10000 entities * 1000 ticks on the CI host.

TSan compiled but all executions failed before main: its runtime could not call personality(ADDR_NO_RANDOMIZE) inside the Docker build sandbox. The image job was skipped, as required. This is not recorded as a TSan pass. The follow-up isolates execution in a container with the narrowly constrained profile from ADR-0020; its result must be recorded after execution.

Successful delivery run 1402

Run 1402, commit 4c2cc04ac4: every job succeeded. The general checks reused Docker's valid cache for unchanged checked sources; their full execution also passed in run 1401. TSan executed all five CTests successfully in 2.71 seconds using the scoped profile and writable tmpfs outputs. Run 1401 had first exposed a non-root tmpfs permission error; setting explicit mode=1777 on these isolated scratch mounts fixed it. No test was disabled or ignored.

The runtime image built, ran the restricted replay smoke test and was pushed to:

registry.aeternalisgames.com/aeternalis/aevolution:sha-4c2cc04ac47d37c6191a9ef02d4bbba716a930dc-run-6-1

Registry digest: sha256:8fce2b462c9907f38d3fb832afeb70d0bff068dd832b2171e861186dbcb8e6b9.

The dev manifest pins this digest. This proves build/test/image delivery; Argo application mapping and Kubernetes Job completion remain unverified locally.

M1.5 native ECS experiment

Opt-in target evo-ecs-experiment (EnTT4 pinned in ADR-0021), based on 1c6e6e1 plus this experiment's changes. MSVC Release contract checks passed, including 72 seed/density/count combinations, hand-calculated arithmetic, canonical equality, restore continuation, retired-ID rejection, malformed snapshot checks and allocator balance. The final clang-tidy build/test passed after tightening the process-boundary diagnostic catch. MSVC ASan passed on the final source. Baseline Release CTest remained 5/5 and format-check included the experiment. No simulation rule/golden changed.

Commands: configure release/analysis/asan with -B build/ecs[-analysis|-asan] -DEVO_BUILD_ECS_EXPERIMENT=ON, build target evo-ecs-experiment, then ctest --test-dir <directory> -R ecs-experiment --output-on-failure. Seven measured samples plus warmup are retained in results/ecs-msvc1944.jsonl; protocol, build observations and scope limits are in ECS_EVALUATION.md. The Linux experiment CI job is pending its first pushed run; no Linux experiment success is claimed yet.

M1.5 Linux evidence

Run 1404, source bf7d60b7cd: every job passed. The new ci-ecs stage explicitly prepared the pinned header/license, then built and ran the experiment under GCC14 Release, Clang19 ASan/UBSan and Clang19 analysis. All three contract invocations passed. Raw measured GCC samples are retained at results/ecs-gcc142.jsonl; all 56 fingerprints match corresponding Windows samples. No production rule or snapshot-format change occurred. This supersedes the pending Linux experiment note above. The final follow-up edits only record results and decision scope; production ECS adoption remains deferred.

M1.6 sequential scheduler (2026-09-19)

Native commands: tools/windows-check.ps1 -Preset release -Action configure, clang-format on changed C++ files, cmake --build --preset release, ctest --preset release; then tools/windows-check.ps1 -Preset analysis -Action check, tools/windows-check.ps1 -Preset asan -Action check and Release -Action format. All three configurations passed 5/5 CTests; clang-tidy and format-check passed. Tests execute under NDEBUG and cover six batch permutations across 32 seeds, an independent direct-lifecycle reference, ordered receipts, restore continuation, rejected conflicts and malformed admission, the 1,024-command boundary, full entity capacity and tick/ID exhaustion. Golden d61ad5bf0c928c76 remains unchanged.

The copy-and-commit implementation allocates before publishing world state and statically checks noexcept publication/receipt moves. Allocation-failure injection was not run. No scheduler throughput claim, concurrency validation, durable input journal or Kubernetes execution is implied.

Gitea run 1407 passed all four jobs for source b620edfd6f8097172acf055402a24620cb078358: Linux checks, the ECS experiment, separate TSan, and image smoke/publication. This closes the pending Linux validation for M1.6. The maintainer's independent dev image pin from 20e609d was preserved during rebase. Kustomize renders the three expected dev resources successfully; no cluster execution is claimed.