Initial commit: architecture docs and headless engineering proof
Aeternalis Evolution, a custom-engine persistent evolutionary MMO. This first commit establishes the project skeleton: - Architecture and design docs: project vision, system overview, engineering constitution, roadmap, 19 ADRs, and the simulation, world, network, persistence and infrastructure architectures. - Headless engineering proof: deterministic synthetic particles, component lifecycle, explicit PRNG streams, canonical snapshots and restored execution. - Build and tooling: CMake 3.28+ / Ninja presets, C++23 feature check, clang-format and clang-tidy config, format check script, Windows check script, and a Docker build context. - Tests: unit tests, benchmarks, a snapshot fuzz target and CLI integration wiring. No gameplay, renderer, network service or running deployment yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,104 @@
|
||||
# Infrastructure and Kubernetes topology
|
||||
|
||||
Status: production-shaped proposal, not deployed workloads. Initial hardware is
|
||||
one R730xd with approximately 128GB RAM. Kubernetes is available per brief; actual
|
||||
CPU/storage/CNI/CSI/backup/network details remain uninspected. Replicas on one
|
||||
physical host do not create an independent failure domain.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
Registry[Harbor immutable images] --> Git[Reviewed deployment Git revision]
|
||||
Git --> Argo[Argo CD environment applications]
|
||||
CI[Build / tests / analysis / sanitizers] --> Registry
|
||||
Workflows[Argo Workflows soak / clone / migration] --> Git
|
||||
subgraph Cluster[Kubernetes cluster initially one physical node]
|
||||
subgraph Dev[evolution-dev]
|
||||
D[Isolated development workloads / quotas]
|
||||
end
|
||||
subgraph Stage[evolution-stage]
|
||||
S[Cloned world / validation / quotas]
|
||||
end
|
||||
subgraph World[evolution-world0]
|
||||
API[Modular backend and director]
|
||||
R[Disposable region workers]
|
||||
B[Background ecology / terrain jobs when needed]
|
||||
PG[(PostgreSQL and durable volumes)]
|
||||
OBJ[(S3-compatible payload storage)]
|
||||
N[JetStream if adopted]
|
||||
API --> PG
|
||||
R --> PG
|
||||
R --> OBJ
|
||||
B --> PG
|
||||
PG --> N
|
||||
end
|
||||
Obs[Prometheus / Grafana / Loki; Otel where justified]
|
||||
end
|
||||
PG --> Off[Independent backup/replication failure domain]
|
||||
OBJ --> Off
|
||||
```
|
||||
|
||||
Deploy one modular backend initially: accounts/identity adapters, history/genome
|
||||
queries and director modules. Region processes scale separately for latency and
|
||||
failure isolation. Background workers split only for measurable workload/lifecycle
|
||||
needs. No Redis unless transient caching/session/routing measurements justify it;
|
||||
no Kafka by default. Evaluate JetStream before adding a bus.
|
||||
|
||||
## Scheduling, storage and security
|
||||
|
||||
| Concern | Required shape |
|
||||
|---|---|
|
||||
| Region pods | Stateless image; memory state recovered through fenced ownership; requests/limits, bounded queues, startup/readiness/liveness probes |
|
||||
| Stateful stores | Supported operator or reviewed StatefulSet strategy; persistent volumes, retention/reclaim safeguards, backup/restore tests |
|
||||
| Future nodes | Labels for realtime, biosimulation, storage/database and optional GPU; affinity/taints and topology spread; no required separation that makes one-node dev unschedulable |
|
||||
| Network | Default-deny namespace policies with explicit DNS/service flows; protected DB/storage/metrics; external realtime UDP exposure chosen with CNI/LB measurement |
|
||||
| Identity | Dedicated service accounts; disable unnecessary token mounts; narrowly scoped RBAC and storage/DB grants |
|
||||
| Pod security | Non-root, read-only rootfs, dropped capabilities, seccomp, bounded writable scratch and resource quotas |
|
||||
| Secrets | External/cluster secret management chosen by operator; no plaintext repo secrets or credentials in images/logs |
|
||||
| Termination | Stop admission, freeze/drain, commit recoverable state and release lease; grace budget measured; crash recovery remains necessary |
|
||||
|
||||
CPU throttling can miss ticks; measure requests/limits rather than copying web
|
||||
service settings. Autoscaling uses tick debt/latency and safe region moves, not
|
||||
unbounded replica creation. Pod disruption budgets help planned maintenance but
|
||||
cannot protect against chassis loss. Local volumes constrain scheduling; adding
|
||||
compute nodes does not automatically make stateful storage portable.
|
||||
|
||||
128GB is a capacity envelope, not all available simulation memory. Inventory OS/
|
||||
Kubernetes overhead, DB cache, object storage, observability, dev/stage and recovery
|
||||
headroom before region budgets. Start measured namespace quotas and leave explicit
|
||||
recovery/compaction headroom. No unverified entity/player-per-node promises.
|
||||
|
||||
## Delivery and upgrades
|
||||
|
||||
Commit → configure/build/warnings → format/tidy → unit/property/determinism/codec
|
||||
checks → ASan/UBSan and separate TSan → applicable integration/ecology/load checks
|
||||
→ measured benchmark report → container build/scan/SBOM/signature → Harbor digest
|
||||
→ reviewed dev GitOps change → soak → staging clone/migration → manual World0
|
||||
promotion. Argo Workflows is the planned orchestrator for clone/accelerated/soak
|
||||
jobs; Argo CD reconciles reviewed manifests. CI skeleton does not imply checks
|
||||
or deployments already ran.
|
||||
|
||||
World0 uses separate application/project permissions, disabled automatic prune/
|
||||
self-heal for destructive state transitions, retained PVCs and explicit promotion.
|
||||
Routine developer jobs cannot delete namespace/data or change world versions.
|
||||
World-law upgrades use a consistent world barrier, not adjacent different-rule
|
||||
canaries. Keep infra/environment values separable so later repository extraction
|
||||
does not require rewriting simulation.
|
||||
|
||||
## Observability and adoption gates
|
||||
|
||||
Prometheus/Grafana: tick histograms, worst-region debt, entities/players, bandwidth,
|
||||
queues, checkpoint age, fencing/restore failures; add biomass/energy/nutrients,
|
||||
birth/death/diversity metrics with biology. Loki: bounded structured logs. Otel
|
||||
traces at request/I/O boundaries when useful. Avoid IDs as unbounded metric labels.
|
||||
Alert on unavailable recovery paths, disk capacity/latency, missing backups,
|
||||
restarts, packet loss and prolonged tick debt; test alerts through fault exercises.
|
||||
|
||||
S3 is the storage API boundary; backend selection remains open. MinIO's official
|
||||
repository is archived/no longer maintained, so do not assume it is an acceptable
|
||||
new production default. Evaluate maintained alternatives for license, operations,
|
||||
S3 semantics, durability and independent restore. [Official MinIO repository](https://github.com/minio/minio).
|
||||
|
||||
Before M8: validate inventory, storage class/reclaim behavior, DNS/TLS/ingress,
|
||||
Harbor/Argo integration, least privilege, resource budgets and off-host recovery.
|
||||
Then kill a region pod and demonstrate automatic fenced restore. No manifests
|
||||
with fabricated endpoints, secrets or storage classes are deployed in M0.
|
||||
@@ -0,0 +1,81 @@
|
||||
# Authoritative network architecture
|
||||
|
||||
Status: locked server authority and untrusted clients; transport/reliability are
|
||||
proposals for M10. M0 has no networking. Steam distribution follows later through
|
||||
an identity-provider adapter; core account IDs are independent of Steam IDs.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
S[Steam identity provider] --> A[Identity adapter / backend]
|
||||
C[Custom desktop client] <-->|HTTPS identity / world / history| A
|
||||
A -->|short-lived region-scoped session grant| C
|
||||
C -->|authenticated bounded intentions| R[Region network host]
|
||||
R -->|ordered commands| W[Simulation owner]
|
||||
W -->|views / outcomes| R
|
||||
R -->|perceived deltas / ACKs / durable events| C
|
||||
```
|
||||
|
||||
## Channel contracts
|
||||
|
||||
| Channel | Information | Delivery contract |
|
||||
|---|---|---|
|
||||
| Realtime | Movement/orientation intentions and superseding snapshots | Prefer datagrams; sequence/drop obsolete state; bounded loss recovery |
|
||||
| Reliable gameplay | Mutation choice, reproduction, death, region transfer | Idempotent action/event IDs; retry; outcome ACK after durable commit |
|
||||
| Backend | Identity, world selection, friends, lineage/history/admin | HTTPS or justified gRPC; provider validation and least-privilege API scopes |
|
||||
|
||||
InputFrame includes protocol/session/region epoch, sequence, intended tick,
|
||||
bounded movement/orientation, action flags and validated interaction reference.
|
||||
Clients cannot author position, damage, energy, resources, genome, reproduction
|
||||
or death. Validate lengths, numeric ranges/non-finite values, sequence window,
|
||||
target relevance, session ownership, action prerequisites, rates and expiry
|
||||
before admitting bounded commands. Reject unsupported required protocol versions.
|
||||
Do not deserialize directly into trusted native structs.
|
||||
|
||||
Snapshot includes region/epoch, server tick, acknowledged simulated input sequence,
|
||||
baseline ID, perception-filtered state and events. Recovery also exposes the
|
||||
durable commit watermark. Receipt ACK means bytes accepted; simulated ACK means
|
||||
input applied; final biological outcome means durable. These are not aliases.
|
||||
|
||||
## Prediction and interest boundaries
|
||||
|
||||
Client predicts only approved local motion using shared versioned rules. It keeps
|
||||
bounded unacknowledged intentions. On authoritative snapshot, restore local state
|
||||
at ACK, discard acknowledged inputs, replay remaining ones, and visually smooth
|
||||
correction. Remote entities interpolate buffered snapshots with bounded
|
||||
extrapolation; loss requests a fresh baseline. Death/mutation/reproduction remain
|
||||
server decisions. Bound retained baselines, input buffers and packet work.
|
||||
|
||||
Proposed interface: `PerceptionView(observer, tick)` → perceptible entity/field
|
||||
facts; `InterestPolicy(session, view, budget)` → replication set and granularity.
|
||||
Do spatial coarse filtering before biological perception. Poor vision cannot
|
||||
receive hidden exact transforms; scent may expose a directional field, echolocation
|
||||
a derived acoustic result. Rendering receives no arbitrary player/NPC/level labels.
|
||||
Social friend identification is a separate authorized policy. Implement simple
|
||||
spatial interest first; M0 does not implement sensory networking.
|
||||
|
||||
## Reliability and security experiments
|
||||
|
||||
Compare established authenticated datagram/reliable transports (including a
|
||||
library-backed UDP security channel and QUIC options) against an owned selective
|
||||
reliability layer. Measure loss/reorder/duplication, head-of-line delay, MTU/
|
||||
fragmentation, congestion, NAT routing, CPU, bandwidth and platform support.
|
||||
Do not choose raw unauthenticated UDP or invent cryptography. The final protocol
|
||||
ADR must name the maintained implementation, exact version, license, cipher
|
||||
configuration and threat review. Custom replication/game protocol does not
|
||||
require custom TLS, encryption, key exchange or random nonces.
|
||||
|
||||
Use short-lived world/region/account-scoped grants, authenticated key establishment,
|
||||
replay windows, rate/byte/work caps, server input scheduling windows and timeouts.
|
||||
Reconnect cannot duplicate control grants. Secrets never enter logs or gameplay
|
||||
snapshots. Network identities map to current ownership epoch; stale region
|
||||
responses/handoff tickets fail. Administrative APIs and storage credentials are
|
||||
separate from client sessions. Game workers have only region-scoped operations;
|
||||
a compromised process remains a threat requiring isolation and audit, not merely
|
||||
client validation.
|
||||
|
||||
Required tests before M10 acceptance: fabricated/oversized/truncated packets,
|
||||
NaN/overflow, replayed actions, sequence wrap/old baselines, duplicate/reordered
|
||||
reliable messages, speed/mutation/resource cheats, flooded sessions, dropped ACKs,
|
||||
lease/handoff races, reconnect uniqueness, sensory information leaks and bounded
|
||||
queue overload. Demonstrate 20+ simulated clients, then measure higher load with
|
||||
latency/loss and authoritative-state divergence reports.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Design decision register
|
||||
|
||||
All recommendations below are reviewable, not silent product decisions. Owner
|
||||
means the engineering/design role accountable at the gate, not an assigned
|
||||
person. ADR indexes distinguish scoped proof choices from production proposals.
|
||||
|
||||
| # | Open question | Current position / experiment | Owner; gate |
|
||||
|---|---|---|---|
|
||||
| 1 | Tick frequency | 20Hz proof only; measure latency, CPU, stiff biology | Simulation/network; before M7/M10 |
|
||||
| 2 | Fixed/variable boundaries | Fixed authoritative ticks, host wall-clock pacing; ecology integer multiples | Simulation; M1 |
|
||||
| 3 | ECS | Bounded component store proof; compare EnTT/Flecs realistic iteration | Engine; M1 |
|
||||
| 4 | Dependency manager | Evaluate vcpkg manifest vs Conan; no runtime dependency for proof | Build; first dependency |
|
||||
| 5 | Units library | Minimal explicit integer wrappers now; evaluate dimensional library before biology | Simulation; M2 |
|
||||
| 6 | Determinism | Exact integer proof; physics separately scoped | Simulation; each system ADR |
|
||||
| 7 | Mutation generation | Seeded context-conditioned candidates with validated prerequisites | Biology/design; M3/M11 |
|
||||
| 8 | Genome representation | Immutable versioned developmental capabilities, not DNA | Biology; M3 |
|
||||
| 9 | Developmental body | Typed body graph proposal | Biology/engine; M13 |
|
||||
| 10 | Heredity/reproductive mutation | Preserve ancestry, define mutation budget and costs | Biology/design; M3/M11 |
|
||||
| 11 | NPC mutation | Undecided; no silent autonomous divergence | Biology/design; M12 |
|
||||
| 12 | Species/speciation | Derived classifications; no class mechanics | Biology/design; M12 |
|
||||
| 13 | Lineage extinction | Need world-wide no-living/no-viable-reserve evidence | Ecology; M4/M12 |
|
||||
| 14 | Offline state | Park without any progression; disconnect fairness unresolved | Design/security; before player persistence |
|
||||
| 15 | Multicellular transition | Physiological and ecological criteria unresolved | Design/biology; M14 |
|
||||
| 16 | Population abstraction | Conservative cohort ledger + distributions, ensemble validation | Ecology; M4 |
|
||||
| 17 | Region dimensions | Benchmark against interest, entities and migration overhead | Engine/platform; M7 |
|
||||
| 18 | Planet coordinates | Hierarchical global address + local coordinates proposal | World; M5/M13 |
|
||||
| 19 | Spherical voxels | Prototype cubed sphere versus alternatives; no fixed topology yet | World; M13 |
|
||||
| 20 | Voxel LOD | Sparse procedural base, aggregate fields, active chunks | World; M5/M13 |
|
||||
| 21 | Surface meshing | Marching Cubes / Surface Nets / Dual Contouring comparison | Render/world; M13 |
|
||||
| 22 | Hydrology | Conserved regional reservoirs/flux before full flow physics | World; M6/later R&D |
|
||||
| 23 | Atmosphere | Aggregate mass and species budgets; no global CFD promise | Ecology/world; M12 |
|
||||
| 24 | Cross-region ownership | Fenced leases + explicit handoff state machine | Distributed systems; M7/M10 |
|
||||
| 25 | Reliability layer | Evaluate secure datagram/reliable transports before bespoke reliability | Network/security; M10 |
|
||||
| 26 | Serialization | Local bounded binary proof; production schemas undecided | Engine/platform; M7/M10 |
|
||||
| 27 | Snapshot/events | Snapshot + deterministic replay batches + durable semantic history | Persistence; M7 |
|
||||
| 28 | Recovery | Committed contiguous replay range and unique epoch | Persistence; M7/M8 |
|
||||
| 29 | Cross-version migration | Clone, migrate, accelerated comparison, approved world barrier | Simulation/platform; M7+ |
|
||||
| 30 | Steam identity | Provider subject → internal account boundary | Backend/security; before Steam |
|
||||
| 31 | Friend recognition | Separate social channel, no player/NPC labels | Design; M11 |
|
||||
| 32 | Anti-cheat | Authoritative outcomes plus authenticated bounded intentions | Security; M10 |
|
||||
| 33 | AI behavior | Shared intention interface; bounded deterministic controller | Simulation; M2/M11 |
|
||||
| 34 | Evolvable behavior | Undecided; intelligence not mandatory | Biology/design; M12/R&D |
|
||||
| 35 | Morphology limits | Development graph validates material, attachment and capability | Biology/design; M13 |
|
||||
| 36 | Procedural animation | Isolated locomotion prototypes; no authored animation dependence | Engine; M13 |
|
||||
| 37 | Retention | Permanent genome ancestry; tiered replay/snapshots/backups | Platform/design; M7/M8 |
|
||||
| 38 | DR objectives | Propose measurable RPO/RTO, validate on actual storage/off-host target | Platform; before World0 |
|
||||
|
||||
Additional blockers: actual host inventory, domain/TLS, registry and Git provider,
|
||||
storage class/backup destination, resource budgets, project licensing and World0
|
||||
promotion approvers. None blocks a local dependency-free simulation proof.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Persistence and region authority
|
||||
|
||||
Status: proposed production contract; source-of-truth PostgreSQL and disposable
|
||||
workers follow locked requirements. M0 implements only an in-memory local proof
|
||||
snapshot. It has no durable journal, database, lease or crash-consistent file writer.
|
||||
|
||||
## Data ownership and recovery
|
||||
|
||||
| Data | Proposed owner and retention |
|
||||
|---|---|
|
||||
| Accounts/provider subjects, worlds, regions | Versioned PostgreSQL relational metadata |
|
||||
| Immutable genomes and ancestry/history | PostgreSQL metadata/events; never discard extinct genomes |
|
||||
| Active simulation | Region worker memory, recoverable from committed checkpoint/replay |
|
||||
| Snapshot/chunk payloads | Immutable S3-compatible objects, strong content digest |
|
||||
| Snapshot manifests and committed replay ranges | PostgreSQL transactions; references verified objects |
|
||||
| Notifications/projections | Transactional outbox; optional JetStream, idempotent consumers |
|
||||
| Ephemeral routing/cache | In-process first; Redis only with measured need, never sole durable state |
|
||||
|
||||
Proposed durable keys: world_id, region_id, epoch, simulation_version,
|
||||
world_format_version, codec_version, generator_version, snapshot_id, tick,
|
||||
covered_sequence, content_digest. Organism, genome and runtime entity identities
|
||||
remain distinct. All cross-world keys include world identity. Migration SQL is
|
||||
versioned when introduced at M7; do not create unused tables in M0.
|
||||
|
||||
Recovery is not semantic events alone: deaths/births do not reconstruct intervening
|
||||
movement, external inputs or random decisions. Persist contiguous deterministic
|
||||
replay batches containing ordered admitted commands, exogenous inputs, tick range,
|
||||
versions and resulting state hash. Preserve required RNG/state/checkpoint data.
|
||||
Use strict append-only domain history for irreversible ancestry/lifecycle outcomes;
|
||||
use snapshots plus replay for mutable region state, not event sourcing every scalar.
|
||||
Batch transactions; no PostgreSQL write per simulation tick.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant W as Worker
|
||||
participant O as Object store
|
||||
participant D as PostgreSQL
|
||||
participant C as Client
|
||||
W->>W: Simulate ordered ticks into pending batch
|
||||
W->>O: Upload immutable replay payload if externalized / verify
|
||||
W->>D: Fenced transaction: replay range + domain outcomes + outbox
|
||||
D-->>W: Durable commit or ambiguous failure
|
||||
W-->>C: Final outcome only after confirmed/reconciled commit
|
||||
W->>O: Upload checkpoint bytes / verify digest
|
||||
W->>D: Fenced publish manifest with covered sequence
|
||||
Note over W,D: On recovery: manifest + complete replay range, verify hash before serving
|
||||
```
|
||||
|
||||
Input receipt, simulated-input acknowledgment and durable gameplay acknowledgment
|
||||
are distinct. Never present an uncommitted permanent death or reproductive success
|
||||
as final. Acknowledged death/reproduction must reconstruct after worker loss.
|
||||
Ordinary speculative movement may roll back within an explicit bounded replay
|
||||
window; that window and batching latency are M7 experiments. Database failure
|
||||
withholds irreversible outcomes, limits pending memory and pauses before budgets
|
||||
or lease safety expire. A disconnected commit may have succeeded: reconcile its
|
||||
idempotency key before retrying.
|
||||
|
||||
Decision IDs store canonical request digests: retry with changed contents is
|
||||
rejected. Track simulated_tick separately from committed_tick. Pending irreversible
|
||||
decisions cannot enable dependent actions; pending death cannot bypass logout.
|
||||
For initial M7, pause the entire region at an irreversible-outcome boundary until
|
||||
commit. A future nonblocking alternative must prove deterministic isolation of
|
||||
every affected interaction; freezing only a dying organism is insufficient.
|
||||
CommitRegionBatch requires the current lease and expected predecessor head.
|
||||
Segments identify contiguous ticks, predecessor digest, exact rules/binary/RNG
|
||||
versions, resolved exchanges and required nondeterministic outputs. If initial M7
|
||||
cannot capture complete replay, evaluate full checkpoint batches explicitly rather
|
||||
than claiming incomplete event replay is recovery.
|
||||
|
||||
## Fenced region lease
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
Unassigned --> Acquiring: director assignment hint
|
||||
Acquiring --> Recovering: transactional epoch increment / lease
|
||||
Recovering --> Active: checkpoint + replay + validation complete
|
||||
Active --> Draining: handoff or shutdown
|
||||
Active --> Fenced: renewal failure / deadline safety margin
|
||||
Draining --> Released: final committed checkpoint / release
|
||||
Fenced --> Acquiring: new valid ownership acquisition
|
||||
```
|
||||
|
||||
The director suggests assignment; PostgreSQL arbitrates. Proposed row:
|
||||
`RegionLease(world, region, holder, epoch, expires_at)`. Acquisition locks the row,
|
||||
requires release/expiry using database time, increments a never-reused epoch and
|
||||
records holder/deadline. Renewal compares holder+epoch and unexpired authority.
|
||||
Every durable region mutation transaction locks/checks that same lease row and
|
||||
epoch; DB privileges prevent workers bypassing the guarded interface. The lock
|
||||
serializes ownership acquisition against commits. A process cannot become
|
||||
authoritative merely because it has a Pod identity or receives a NATS message.
|
||||
|
||||
Workers fence themselves before their conservative local safety deadline; a
|
||||
pause/resume cannot renew a stale epoch. Incoming commands, routing grants,
|
||||
snapshots and transfer operations carry epoch/revision. Gateways/clients reject
|
||||
stale-authority output under the routing/session protocol. Two processes may
|
||||
hold stale routing views too: require bounded-validity authority registration
|
||||
and fail-closed forwarding on uncertainty. Begin with one admission gateway per
|
||||
region; partition/clock tests gate multiple gateways. Carrying an epoch alone
|
||||
does not fence a gateway unaware of a newer epoch. Two processes may
|
||||
retain memory, but cannot both commit authority. Recovery readiness requires
|
||||
validated replay and a valid current lease; liveness is not proof of ownership.
|
||||
Test long pauses, stale renewals, overlapping acquisition and delayed packets.
|
||||
|
||||
## Entity handoff across regions
|
||||
|
||||
1. Destination holds a valid compatible region lease and read-only ghost.
|
||||
2. Source freezes the authoritative organism at a tick boundary and durably
|
||||
prepares final state under a transfer ID. Ghost cannot spend/act/reproduce.
|
||||
3. Destination validates and durably stages the exact transferred state.
|
||||
4. One transaction checks both epochs and changes the organism ownership mapping
|
||||
from source to destination, recording commit and state reference.
|
||||
5. Destination activates only after observing commit and its current lease;
|
||||
source tombstones its authority. Client reroutes with a fresh grant.
|
||||
|
||||
Abort is a durable state transition permitted before commit; source resumes only
|
||||
after observing that abort. Timeout alone does not authorize a second copy.
|
||||
Retries return the recorded transfer outcome. Crash at each arrow must yield one
|
||||
owner or a temporarily unavailable organism, never duplication/loss. Epochs and
|
||||
the organism ownership mapping must be enforced inside persistence transactions.
|
||||
|
||||
## Snapshots, migrations and retention
|
||||
|
||||
Snapshots capture coherent committed state: snapshot.tick <= committed_tick with
|
||||
a matching committed replay watermark. They never publish speculative outcomes.
|
||||
Compression and I/O run outside the simulation writer on immutable buffers.
|
||||
Upload immutable bytes, verify length/digest, then
|
||||
publish metadata and replay coverage transactionally under the current fence.
|
||||
Failed publication leaves an orphan, not a checkpoint. Compaction/GC retains all
|
||||
objects reachable from live manifests, clones, backups and retained replay. Never
|
||||
delete source replay until a verified recoverable replacement and backup exist.
|
||||
|
||||
Version biology rules, world format and wire protocol independently. Unknown
|
||||
required formats/rules fail closed. Upgrade: quiesce/capture consistent world
|
||||
barrier → clone into isolated identity/subjects → migrate → accelerate → compare
|
||||
ecological distributions and histories → explicit approval → compatible world
|
||||
barrier deployment. Do not run adjacent regions under different physical laws.
|
||||
Migration failure reverts code/state before new writes; downgrade after new writes
|
||||
requires a designed reverse migration, not a blind database restore.
|
||||
|
||||
Backup/DR is detailed in OPERATIONS. A replica on the same host does not survive
|
||||
host loss. Exact guarantees for acknowledged outcomes require an explicitly
|
||||
verified durable failure domain containing the complete causal recovery chain
|
||||
and referenced payloads before acknowledgment; restoring an older backup cannot silently undo
|
||||
permadeath. That admission/DR requirement is a World0 launch gate.
|
||||
@@ -0,0 +1,109 @@
|
||||
# Simulation architecture
|
||||
|
||||
## Contract and scope
|
||||
|
||||
LOCKED: headless, composition-based simulation; explicit ticks and seeded
|
||||
randomness; stable agreed replay; no graphics or infrastructure dependency.
|
||||
RECOMMENDATION: one authoritative mutation owner per region. Systems execute in
|
||||
a documented order; later jobs operate on immutable inputs and merge in stable
|
||||
order. No component starts threads.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
Planet[Tier 3: regional / planetary ecology] <-->|versioned conserved transfers| Population[Tier 2: lineage populations / distributions]
|
||||
Population <-->|exclusive representation conversion| Entity[Tier 1: active individuals]
|
||||
Micro[2D cellular domain] <-->|energy / nutrients / biomass ledgers| Macro[3D macroscopic domain]
|
||||
Entity --> Micro
|
||||
Entity --> Macro
|
||||
```
|
||||
|
||||
Each represented organism belongs to exactly one active or aggregate population
|
||||
record. Conversions preserve genotype identity, resource totals and residuals;
|
||||
they do not manufacture newly sampled energy. Biological time advances from
|
||||
world ticks. Offline player records do not participate in any progressing tier.
|
||||
|
||||
## Initial proof interface
|
||||
|
||||
The foundation uses **synthetic particles**, not organisms. `SimulationWorld`
|
||||
stores a bounded sorted sequence of entity IDs, `Position2` components and
|
||||
per-entity `Random` components. This is a minimal composition-store experiment,
|
||||
not a general ECS framework. Maximum 100,000 active proof entities, monotonic
|
||||
64-bit IDs with no reuse, read-only inspection by span. Mutation can invalidate
|
||||
views; callers cannot hold references across mutation. Erasing an ID permanently
|
||||
invalidates it. Entity exhaustion is an explicit error.
|
||||
|
||||
```cpp
|
||||
// Contract sketch; include/evo/simulation/world.hpp is the concrete API.
|
||||
SimulationWorld world{seed};
|
||||
auto id = world.create_entity(Position2{}); // expected<EntityId, Error>
|
||||
auto result = world.tick(); // exactly one logical tick
|
||||
auto bytes = world.serialize(); // canonical local snapshot
|
||||
auto restored = SimulationWorld::restore(bytes); // expected<World, Error>
|
||||
```
|
||||
|
||||
One proof tick is provisionally 50,000 microseconds. Every particle independently
|
||||
draws two bounded values and moves -1, 0 or +1 integer millimetres per axis on a
|
||||
1,024 mm periodic test domain. Movement is deliberately synthetic; no velocity,
|
||||
biological energy, mutation, feeding or player input exists yet. Tick count
|
||||
overflow is rejected before mutation. Entity ordering is ascending ID and RNG
|
||||
state belongs to each entity, so unrelated entity removal cannot shift another
|
||||
entity's random stream.
|
||||
|
||||
## Determinism boundary
|
||||
|
||||
Exact guarantee for proof version 1: same initial bytes or seed + entity creation
|
||||
order, same API operations and tick count produce byte-identical canonical
|
||||
snapshots. Integer unsigned operations are specified; no standard-library random
|
||||
distributions, floating-point operations, addresses, wall clocks, hash-table
|
||||
iteration or thread scheduling enter the state. Tests include algorithm golden
|
||||
vectors and a frozen snapshot fingerprint, replay and mid-run recovery.
|
||||
|
||||
This does **not** promise that future third-party physics will be bit-exact
|
||||
across platforms. Such systems require their own versioned determinism scope,
|
||||
tolerance envelopes and canonical-server target. Fast-math is forbidden in
|
||||
authoritative deterministic code. Random streams for later systems will use
|
||||
stable entity/system identifiers and explicit counters; changing draw order is a
|
||||
simulation rules change.
|
||||
|
||||
## Scheduling and pacing
|
||||
|
||||
The kernel accepts no `deltaTime` argument. The executable advances an exact
|
||||
requested number of ticks. A future realtime host converts monotonic wall time
|
||||
into tick debt; it must cap catch-up work, emit overload metrics and shed region
|
||||
load or pause under an explicit policy. It must not enlarge biological timesteps
|
||||
to conceal overload. Slow ecology rates are integer multiples of base world
|
||||
ticks; cross-tier transfers use committed tick intervals.
|
||||
|
||||
## Snapshot proof format v1
|
||||
|
||||
All fields are encoded explicitly little endian with no padding. Header:
|
||||
8-byte ASCII `AEVOPRF0`, u32 format version=1, u32 rules version=1, u64 tick
|
||||
quantum in microseconds=50000, u64 world seed, u64 current tick, u64 next entity
|
||||
ID, u32 entity count. Rows: u64 ID, u32 x, u32 y, u64 PRNG state. Trailer: u64
|
||||
FNV-1a of every preceding byte (offset 14695981039346656037, prime
|
||||
1099511628211, wrapping u64). Header is 52 bytes; total is 60 + 24*count.
|
||||
|
||||
Restore checks bounded input length/count **before allocation**, exact length,
|
||||
magic, versions, tick quantum, checksum, sorted unique nonzero IDs below next ID,
|
||||
and bounded coordinates. Unknown versions fail without partial state publication.
|
||||
The fingerprint detects accidental damage and regressions; it is not an
|
||||
authentication or security primitive. Snapshot allocation may throw at the host
|
||||
boundary. There is no durable-file or crash-consistent storage claim here.
|
||||
|
||||
Production storage will wrap independently versioned simulation/world/codec
|
||||
identities with immutable object checksums and committed metadata; this local
|
||||
format cannot be used for World 0 without a migration ADR.
|
||||
|
||||
## Extension seams and failure tests
|
||||
|
||||
Future commands carry tick, entity identity, sequence and validated intention;
|
||||
duplicate/reordered input handling belongs to the host protocol boundary.
|
||||
Biology introduces separate immutable genomes, development and phenotypes before
|
||||
adding energetic systems. Entity lifecycle mutations occur at stable phase
|
||||
boundaries. Metrics sinks see counters after ticks; they cannot mutate state.
|
||||
|
||||
Tests must reject truncated, corrupt, oversized, unknown-version, duplicate-ID,
|
||||
out-of-order and invalid-coordinate snapshots; compare continuous versus
|
||||
restored execution; exercise removal, capacity and tick exhaustion; and check
|
||||
that two entities' RNG streams remain independent of population order changes.
|
||||
See ADRs 0005–0011 before extending this module.
|
||||
@@ -0,0 +1,98 @@
|
||||
# System overview
|
||||
|
||||
Status: architecture recommendation; locked product constraints are in
|
||||
[PROJECT_VISION](../PROJECT_VISION.md). Only the local proof described in
|
||||
[SIMULATION_ARCHITECTURE](SIMULATION_ARCHITECTURE.md) is implemented initially.
|
||||
|
||||
## System context
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Player[Player / Steam desktop client] -->|intentions| Game[Aeternalis world services]
|
||||
Steam[Steam identity provider] -->|validated identity| Game
|
||||
Operator[Operator] -->|approved GitOps release| Game
|
||||
Game -->|perceived state / outcomes / history| Player
|
||||
Game --> History[(Persistent world and phylogeny)]
|
||||
Game --> Telemetry[Metrics / logs / traces]
|
||||
Backups[Independent backup failure domain] <-->|verified archives / restore| History
|
||||
```
|
||||
|
||||
## Component and dependency boundaries
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Client[Custom desktop client] --> Protocol[Protocol / prediction adapter]
|
||||
Region[Region process] --> Network[Transport / validation / interest]
|
||||
Region --> Persist[Persistence adapter]
|
||||
Region --> Sim[libevo simulation]
|
||||
Sim --> Biology[Genome / development / phenotype / ecology]
|
||||
Sim --> Environment[World / resources / sparse terrain interfaces]
|
||||
Sim --> Core[IDs / units / ticks / PRNG]
|
||||
Biology --> Core
|
||||
Environment --> Core
|
||||
Protocol --> Core
|
||||
CLI[evo-sim / tests / benchmarks] --> Sim
|
||||
Backend[Modular backend: identity / history / director] --> Persist
|
||||
Persist --> Stores[PostgreSQL / object store]
|
||||
Network --> Protocol
|
||||
```
|
||||
|
||||
The renderer may consume phenotype and terrain views. Biology never consumes a
|
||||
mesh, graphics device, network session or database connection. Public simulation
|
||||
headers expose values and bounded commands, not third-party ECS/storage types.
|
||||
No Steam ID, socket, Kubernetes API, logger clock or storage handle belongs in
|
||||
the simulation state. Durable account references live in orchestration mapping.
|
||||
|
||||
| Boundary | Owns | Must not own |
|
||||
|---|---|---|
|
||||
| core | Value types, deterministic algorithms, explicit version constants | Global state, threads, I/O |
|
||||
| simulation (`evo_simulation`, alias `evo::simulation`) | State, stable tick ordering, component lifecycle, pure restore validation | Wall clock, transport, database or rendering |
|
||||
| biology | Genotype → development → phenotype; energetic transfers | Account rewards, player/NPC subclasses |
|
||||
| world | Procedural base identity, spatial/environmental fields | Dense planet allocation, gameplay authority |
|
||||
| persistence | Durable metadata, snapshots, replay batches, migration | Per-tick simulation decisions |
|
||||
| network | Input validation, authentication boundary, bounded codecs and relevance | Trust in client-provided positions or biology |
|
||||
| backend | Identity adapters, account/world metadata, history queries, directory/lease coordination | Every noun deployed as a service |
|
||||
| app / server | Tick pacing, I/O ownership, durable outcome barrier, metrics export | Alternative biological implementation |
|
||||
|
||||
Only core, simulation, local snapshot codec and `evo-sim` are source modules in
|
||||
the proof. Future rows are boundaries, not empty directories or dummy services.
|
||||
Split a library when a real independently tested contract exists.
|
||||
|
||||
## Runtime topology and data flow (target)
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Client[Client] -->|HTTPS identity / world selection| API[Backend API]
|
||||
API -->|session and routing grant| Client
|
||||
Client -->|authenticated input channel| Worker[Region worker]
|
||||
Director[Director] -->|assignment hint| Worker
|
||||
Worker -->|fenced lease / commit| PG[(PostgreSQL)]
|
||||
Worker -->|immutable snapshots| S3[(Object store)]
|
||||
Worker -->|tick-boundary commands| Sim[In-memory simulation]
|
||||
Sim -->|events and views| Worker
|
||||
Worker -->|perception-filtered snapshots / durable outcomes| Client
|
||||
PG -->|transactional outbox| Bus[Optional JetStream]
|
||||
Bus -->|idempotent projections| History[History queries / ecology jobs]
|
||||
```
|
||||
|
||||
Simulation reads an immutable ordered command batch at a tick boundary. It emits
|
||||
state and domain outcomes. The host stages durable records and marks an outcome
|
||||
committed only after the durable barrier. Projection notifications are not the
|
||||
source of authority. Simulation metrics are deterministic counters; timing and
|
||||
tracing are measured outside the simulation.
|
||||
|
||||
## Failure containment
|
||||
|
||||
- Worker crash: recover a consistent snapshot plus complete committed replay
|
||||
range under a new lease epoch. Never accept an old worker's writes.
|
||||
- Database unavailable: stop durable decisions and eventually pause simulation;
|
||||
do not continue granting irreversible uncommitted outcomes.
|
||||
- Object storage unavailable: retain/retry bounded pending snapshots; enforce
|
||||
journal/storage budgets rather than silently dropping recovery data.
|
||||
- Backend unavailable: availability degrades separately from existing regions,
|
||||
within session and lease policies. This does not grant workers perpetual
|
||||
authority.
|
||||
- Shared-host outage: all initial workloads can stop. Off-host recovery is an
|
||||
explicit operations requirement, not a benefit Kubernetes provides by itself.
|
||||
|
||||
See persistence, network and infrastructure architecture for detailed contracts.
|
||||
@@ -0,0 +1,105 @@
|
||||
# World architecture
|
||||
|
||||
Status: locked sparse, persistent, editable-world requirements; recommended
|
||||
contracts below; planetary topology, resolution and solvers remain experiments.
|
||||
These are future boundaries, not implemented environmental systems in M0.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
P[Planet seed and generation version] --> C[Planetary aggregate cells]
|
||||
C --> R[Regions with leased authority]
|
||||
R --> M[2D microscopic habitats]
|
||||
R --> V[Sparse 3D environmental chunks]
|
||||
V --> F[Derived fine detail and mesh caches]
|
||||
M <--> L[Conservative ecological transfer ledger]
|
||||
V <--> L
|
||||
B[Immutable procedural base] --> V
|
||||
D[Persisted modifications and snapshots] --> V
|
||||
```
|
||||
|
||||
Region, terrain-chunk and ecological-cell boundaries need not coincide. A world
|
||||
mapping contract resolves them; assigning another process never changes durable
|
||||
addresses. Every lineage ultimately refers to a player-influenced founder; base
|
||||
generation supplies geology, chemistry and energy, never implicit wildlife.
|
||||
|
||||
## Coordinates and contracts
|
||||
|
||||
Recommend a planet-centered right-handed reference frame for interchange,
|
||||
bounded region-local right-handed frames (+Z up) for active simulation and stable
|
||||
integer chunk addresses. Persist frame transforms, resolution, origin, units and
|
||||
versions. A microscopic habitat defines two metric axes and effective depth;
|
||||
2D area never implicitly becomes 3D biomass. Render origin shifts do not move
|
||||
authoritative state. Use coarse addresses plus local offsets, not global hot
|
||||
entity floats. Final spherical indexing, radius, region/chunk sizes and precision
|
||||
remain open. M5 starts with a small local test volume.
|
||||
|
||||
| Logical contract | Inputs / outputs and invariant |
|
||||
|---|---|
|
||||
| GenerateBaseChunk | seed, generator/material versions, address → deterministic samples; no I/O clock |
|
||||
| ReadEnvironment | footprint, requested fields, view → units, resolution, valid tick and error bound |
|
||||
| ProposeEnvironmentalTransfer | action ID, owners, quantities → reserved inventory or insufficiency |
|
||||
| CommitEnvironmentalTransfer | reservation, epoch → exactly one debit/credit; retry returns prior outcome |
|
||||
| ApplyTerrainEdit | command ID, expected revision, geometry, matter destinations → new chunk revision |
|
||||
| BuildSurface | immutable samples, neighbor revisions, mesher/LOD versions → disposable mesh/collision cache |
|
||||
|
||||
## Representation and storage
|
||||
|
||||
See ADR-0019. Evaluate palette material IDs, quantized geometry, constant uniform
|
||||
channels, sparse varying field pages and coarser chemistry grids. Material tables
|
||||
hold shared coefficients. Allocate only relevant changing fields. Concentration
|
||||
derives from inventory/volume; temperature derives from thermal energy and heat
|
||||
capacity. A material ID does not imply unlimited extractable nutrients.
|
||||
|
||||
Quantization defines units, range, rounding and residuals. Refinement partitions
|
||||
existing inventory; coarsening sums extensive quantities before deriving intensive
|
||||
ones. Fine detail cannot contribute consumable matter without a parent budget.
|
||||
Benchmarks include geometry, fields, indexes, halos, dirty journals and caches;
|
||||
stress uniform, heterogeneous and heavily excavated worlds, including deep edits.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant W as Worker
|
||||
participant G as Versioned generator
|
||||
participant S as Object store
|
||||
participant J as Durable manifest and journal
|
||||
W->>J: Read committed manifest / covered sequence
|
||||
W->>S: Read verified compacted payload if present
|
||||
W->>G: Otherwise generate exact recorded base version
|
||||
W->>J: Read complete committed edits after covered sequence
|
||||
W->>W: Replay edits and validate inventory
|
||||
W->>S: Upload immutable compacted candidate and verify
|
||||
W->>J: Atomically publish new manifest / covered sequence
|
||||
```
|
||||
|
||||
Keep generator versions required by existing territory. Never regenerate old
|
||||
chunks under new geology implicitly. Compaction records base/version/address,
|
||||
revision, covered sequence and digest. Upload alone is not a commit; deletion
|
||||
waits for clone/backup references and retention. Neighbor halos are read-only,
|
||||
revision-tagged state. Stale mesh work is discarded. Collision revision gating
|
||||
requires a later gameplay ADR; visually completed edits cannot bypass physics.
|
||||
|
||||
## Ecological coupling and fidelity
|
||||
|
||||
2D organisms/cohorts debit biomass through a transfer ledger to 3D plankton
|
||||
density. Corpses debit organic matter to decomposition/nutrient reservoirs.
|
||||
Domains do not share literal physics. Each inventory has exactly one owner;
|
||||
active detail reserves a disjoint share of its aggregate. Transfers record ID,
|
||||
source/destination, epochs, logical interval, composition, amount and causal event.
|
||||
|
||||
Across regions, durably escrow the source before idempotent destination credit.
|
||||
Recovery retries or cancels using durable transfer state; elapsed time cannot
|
||||
authorize another credit. In-transit inventory remains in conservation totals.
|
||||
Edits displace mass to soil, carried resources, debris/fluid or an explicit sink.
|
||||
Excavation cannot simply delete matter.
|
||||
|
||||
Solver intervals are explicit integer multiples of world ticks. Use committed
|
||||
boundary views and apply integrated fluxes once. Catch-up is bounded and measured;
|
||||
never silently drop intervals. Hydrology, atmospheric chemistry, erosion,
|
||||
boundary conditions and acceptable numerical residuals remain experiments. M6
|
||||
uses light, moisture and one or two nutrients, not global CFD.
|
||||
|
||||
Required tests: address round trips and seams; generator golden versions;
|
||||
duplicate edit idempotency; crash at every compaction boundary; oscillating LOD
|
||||
without resource gain; stale halo invalidation; stalled transfer escrow; corrupt
|
||||
objects/full disk rejection. No planet topology or smooth mesher is accepted
|
||||
until its comparison demonstrates precision, seam correctness and bounded cost.
|
||||
Reference in New Issue
Block a user