164 lines
9.0 KiB
Markdown
164 lines
9.0 KiB
Markdown
# M1.5 ECS evaluation protocol
|
||
|
||
Scope: synthetic composition storage, not biological rules. Public SimulationWorld,
|
||
its golden fingerprint, rules version and snapshot format remain unchanged.
|
||
|
||
## Workload and controls
|
||
|
||
Both adapters store a stable u64 ID and counter, with optional Motion and Capture
|
||
counter components. These are workload labels, not organism classes or energy.
|
||
Seed 123456 selects capabilities independently at 10% and 100% prevalence.
|
||
Run 1000 and 10000 entities, 100 update passes, one warmup and seven measured
|
||
samples. Alternate adapter order between samples to reduce simple ordering bias.
|
||
|
||
Both adapters use three ordered system passes per tick. One adds 3 to every
|
||
common counter; the next adds Motion.rate then increments that
|
||
rate by 1, and adds Capture.rate then increments it by 2. These independent
|
||
integer operations have a closed-form reference. No wall clock enters state.
|
||
After updates, remove every tenth original entity and create the same number
|
||
with new monotonic IDs. Each adapter rejects stale IDs and preserves external
|
||
identity despite storage relocation/native handle recycling.
|
||
|
||
Measure population creation (excluding the empty store constructor), updates,
|
||
churn, canonical encoding and fresh restore
|
||
separately. Export sorts by external ID and explicitly encodes fields little
|
||
endian. Decode checks exact size, bounded count, ID ordering and capability
|
||
flags. This in-memory experimental format is not production persistence.
|
||
Compare full bytes after every correctness phase and after restore/continuation;
|
||
fingerprints in reports are convenient checks, not substitutes for equality.
|
||
|
||
The row adapter extends the proof's sorted-row strategy with optional fields;
|
||
it is not a benchmark of the unmodified production binary. EnTT uses sparse
|
||
views and an external-ID-to-handle vector. That map's allocation is counted.
|
||
Neither adapter reserves ahead to conceal growth; both start empty. No owning
|
||
groups or per-tick sorting is applied because updates do not interact across
|
||
entities. A production interaction scheduler must specify stable conflict order.
|
||
|
||
EnTT receives a small allocator-traits bridge backed by the counting resource.
|
||
Direct polymorphic_allocator failed compilation because allocator-aware storage
|
||
construction received a second injected allocator. The bridge preserves normal
|
||
allocator-traits construction and tracks rebound allocations; it is benchmark
|
||
instrumentation, not a new production allocator. Lifetime-balance tests apply
|
||
to both adapters. Allocation failures abort the experiment at its host boundary;
|
||
these internal fixture adapters do not claim transactional recovery.
|
||
|
||
External IDs are capped at 200000 in this experiment. The EnTT lookup vector
|
||
retains space for retired IDs until store destruction. This is deliberately
|
||
measured, not an acceptable unbounded mapping for a persistent region. A
|
||
production EnTT adapter would need a separately evaluated bounded/sparse map.
|
||
|
||
## Memory and evidence
|
||
|
||
Track allocation calls, live requested bytes and peak requested bytes for store
|
||
containers through a dedicated std::pmr resource. Exclude canonical snapshot
|
||
scratch/output, stack, allocator metadata and process RSS. Check all tracked
|
||
bytes return to zero on adapter destruction. Do not call this total memory.
|
||
Report construction/iteration/churn/export/restore timings for every sample,
|
||
seed, count, density, compiler and build type; summarize medians and ranges.
|
||
Measure cold/warm target build time and executable size separately. Do not
|
||
compare wall times between Windows and Linux as if hosts/flags were controlled.
|
||
|
||
## Acceptance and decision
|
||
|
||
Correctness gates: hand-checkable arithmetic fixture; density/seed ensembles;
|
||
empty/singleton stores; stale/duplicate identities; churn; byte equality;
|
||
malformed codec inputs; restore then continued equality; allocator balance.
|
||
Timing has no CI threshold. Regression checks remain active under NDEBUG.
|
||
|
||
Prefer the simpler strategy when it satisfies measured needs. A candidate's
|
||
query speed cannot excuse unstable IDs, hidden iteration-order semantics,
|
||
unbounded adapter retention or more memory than the region budget supports.
|
||
Document the measured recommendation in ADR-0005; any later production adapter
|
||
needs a separate contract review and unchanged replay tests.
|
||
|
||
## Reproduction
|
||
|
||
After initializing the compiler environment (Windows: run the existing check
|
||
helper once in the same shell):
|
||
|
||
```sh
|
||
cmake -P tools/prepare-entt.cmake
|
||
cmake --preset release -B build/ecs -DEVO_BUILD_ECS_EXPERIMENT=ON
|
||
cmake --build build/ecs --target evo-ecs-experiment --parallel 2
|
||
ctest --test-dir build/ecs -R ecs-experiment --output-on-failure
|
||
./build/ecs/evo-ecs-experiment --measure
|
||
```
|
||
|
||
Preparation is the only network step. Repeating configure/build uses the verified
|
||
local header offline. Use .exe on Windows. Never place EnTT headers in include/evo.
|
||
|
||
## Windows observations, 2026-09-19
|
||
|
||
MSVC 19.44.35217.0 (/O2, NDEBUG), CMake 3.31.6, Ninja 1.12.1; Windows
|
||
10.0.26200 on Ryzen 7 9800X3D (8 cores / 16 logical processors). Seven samples
|
||
after warmup; [raw JSONL](results/ecs-msvc1944.jsonl) includes every sample and
|
||
fingerprint. Measurements were recorded after compiler jobs completed. There
|
||
is no controlled CPU isolation or portable performance threshold.
|
||
|
||
10000 entities, median elapsed milliseconds; update column is **100 ticks**,
|
||
churn removes/replaces 1000 individuals. Memory is live requested container bytes.
|
||
|
||
| Store / density | Create ms | Update ms | Churn ms | Export ms | Restore ms | Live bytes |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| Rows / 10% | 0.3366 | 0.7365 | 1.3883 | 0.3019 | 0.3056 | 485520 |
|
||
| EnTT4 / 10% | 0.4178 | 0.9868 | 0.0952 | 0.6435 | 0.5067 | 638608 |
|
||
| Rows / 100% | 0.2699 | 0.9237 | 1.3829 | 0.2873 | 0.2561 | 485520 |
|
||
| EnTT4 / 100% | 0.5777 | 7.0615 | 0.1399 | 0.6269 | 0.6842 | 866632 |
|
||
|
||
Observed update min–max: rows 10% 0.7312–0.8778 ms; EnTT 10%
|
||
0.9824–1.0396 ms; rows 100% 0.9203–0.9358 ms; EnTT 100%
|
||
7.0491–7.1502 ms. These are seven-sample ranges, not confidence intervals.
|
||
|
||
All paired samples have identical canonical bytes. EnTT wins this individual
|
||
churn case by roughly 10–15x; rows win the measured simple updates/export and
|
||
use fewer live bytes. EnTT's peak bytes can be lower than the row vector's
|
||
temporary reallocation peak: at 10% density, 638608 versus 809200. Do not equate
|
||
live bytes, peak requested bytes and process RSS.
|
||
|
||
The combined comparison executable is 99328 bytes; it contains both adapters
|
||
and verification code, so this is not an isolated EnTT binary-size delta.
|
||
Clean target build observed 2.7621 seconds, warm no-op 0.0475 seconds
|
||
(configuration excluded; OS file cache not flushed).
|
||
No claim of allocator-overhead/RSS measurement is made.
|
||
|
||
Recommendation: keep the current bounded row proof while designing the
|
||
deterministic scheduler. Preserve a future adapter boundary. Revisit with real
|
||
component sizes, dynamic capabilities, interacting systems, EnTT groups, batched
|
||
structural changes and a bounded persistent-ID mapping before production adoption.
|
||
Do not extrapolate these synthetic counts into organism/player capacity.
|
||
|
||
## Linux observations, Gitea run 1404
|
||
|
||
[Run 1404](https://git.aeternalisgames.com/dingo/AEvolution/actions/runs/1404),
|
||
source bf7d60b7cd174736d38aea42616064e55e0c5f05: GCC14 Release experiment,
|
||
Clang19 ASan/UBSan and clang-tidy contract runs all passed. Baseline regression,
|
||
TSan and image smoke/publish jobs also passed. The experiment itself remains
|
||
single-threaded; baseline TSan does not certify future ECS concurrency.
|
||
|
||
CMake 3.28.3, GCC 14.2.0, Clang 19.1.1; Linux x86-64 Docker runner. CPU model and
|
||
cgroup limits were not captured for these samples, so they are exploratory
|
||
within-runner comparisons, not a controlled hardware benchmark. All 56 sample
|
||
fingerprints match Windows. [Raw Linux JSONL](results/ecs-gcc142.jsonl).
|
||
|
||
10000 entities, medians in milliseconds; updates cover 100 ticks:
|
||
|
||
| Store / density | Create ms | Update ms | Churn ms | Export ms | Restore ms | Live bytes |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| Rows / 10% | 0.5208 | 7.6433 | 10.8217 | 1.1124 | 1.0239 | 655360 |
|
||
| EnTT4 / 10% | 1.0889 | 3.2116 | 0.1932 | 1.6101 | 1.4745 | 710392 |
|
||
| Rows / 100% | 0.8512 | 7.9037 | 10.8592 | 1.1100 | 1.0152 | 655360 |
|
||
| EnTT4 / 100% | 1.4721 | 18.9315 | 0.2737 | 1.7558 | 1.9766 | 968672 |
|
||
|
||
Update min-max: rows 10% 7.5747-7.7266 ms; EnTT 10% 3.1476-3.2407 ms;
|
||
rows 100% 7.3748-9.3729 ms; EnTT 100% 18.7963-19.0924 ms. Combined executable
|
||
size: 180256 bytes (unstripped); this is not a per-adapter size delta. Allocation
|
||
instrumentation contributes overhead to creation/churn/restore timings.
|
||
|
||
Unlike Windows, EnTT sparse updates win on this Linux run (~2.4x), as well as
|
||
churn (~40-56x). Rows retain cheaper dense updates, export and lower live requested
|
||
bytes. Different vector growth policies also change live/peak allocation totals.
|
||
These results strengthen the case for keeping the production choice open:
|
||
selecting a library from one compiler's synthetic loop would be unjustified.
|
||
The scoped ADR-0005 recommendation still retains rows for the bounded proof and
|
||
scheduler contract, with production adoption gated on realistic workloads.
|