Files
aevolution/docs/infrastructure/POSTGRESQL.md
T
dingoandClaude Opus 5.5 dd7e8028b0 Complete M7.5 durable batch outcomes and commit resolution
ADR-0063: manifest v2 (360 bytes) binds a per-tick validation trace
object and a canonical decision block by SHA-256. Publication replays the
trace from the committed predecessor's verified payload and requires
every report and the candidate bytes, derives deaths deterministically,
uploads payload and trace with verified read-back, then sends manifest
and decisions to one fenced statement. RegionBatch seals at a terminal
transition or 64 ticks and refuses ticks until durable confirmation.

Schema v5 records the checkpoint, its decisions and outbox rows
atomically; a unique index keeps one terminal outcome per actor
(EVL08), triggers make commits/decisions immutable (EVL09), an exact
retry of a committed ID returns the original receipt, changed bytes
conflict, and read-only resolve_region_commit answers committed/absent/
conflict. Outbox pending/acknowledge give idempotent at-least-once
delivery. Recovery reconciles durable deaths with restored lifecycle
(and archive deaths for the lineage profile).

Validated: Windows CTest 58/58; Linux GCC Debug/Release, clang-tidy and
ASan/UBSan CTest 58/58 each; PostgreSQL suites Release, analysis, ASan
(lost response, committed-after-timeout, terminated orphan retry,
changed bytes, repeated/partial/reordered decisions, gap, outbox
redelivery, history quarantine); five mutations each caught. No
Docker/Gitea CI, process kill, broker or deployment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 17:30:30 -04:00

166 lines
9.7 KiB
Markdown

# PostgreSQL bootstrap (M7.2) and region fencing (M7.3)
ADR-0059 pins server/libpq 18.6. `evo::postgres` is optional, Linux-only and separate
from all simulation targets. Preparation explicitly downloads a verified source
archive; ordinary CMake configure does not fetch anything. PostgreSQL license is
permissive; source COPYRIGHT and system TLS license information must accompany
distributed tooling. No custom hashing/cryptography is introduced.
## Local isolated verification
Linux prerequisites: GCC14, CMake>=3.28, Ninja, make, bison, flex, curl, bzip2,
OpenSSL development headers and zlib development headers. Optional clang19 tools
support analysis/ASan. Run as a normal user, not root:
```sh
bash tools/prepare-postgres.sh
cmake --preset release -B build/postgres -DCMAKE_CXX_COMPILER=g++-14 \
-DEVO_BUILD_POSTGRES=ON -DPostgreSQL_ROOT="$PWD/.cache/postgres/install"
cmake --build build/postgres --target evo-postgres-tests --parallel 2
bash tools/ci/postgres-check.sh "$PWD/build/postgres/evo-postgres-tests"
```
The test script overrides all connection coordinates and creates a new private
0700 /tmp directory, private Unix socket and disposable local cluster. It creates
the `evo_runtime_test` group and the `evo_runtime_login` member, then runs fencing
tests as that real runtime login. TCP is off;
local trust applies only inside that private directory. Cluster stops on exit;
logs/data are retained there for diagnosis. It never uses production PG credentials
or deletes a pre-existing database. Do not point this script at a shared server.
## Adapter contract
One connection belongs to one I/O owner; one outstanding parameterized statement.
No hidden threads, pool, transaction replay or automatic reconnect. Connections
and queries take 1..30000ms deadlines on a steady clock. Remote hostaddr must be
numeric, host supplies TLS identity, CA file is explicit, TLS verify-full required.
Local sockets require no PGHOSTADDR/PGSERVICE override. GSS negotiation is disabled.
Query SQL <=16KiB, <=16 parameters of <=4KiB, response <=128 rows/16 columns/64KiB
decoded values; single-row mode prevents accumulating a whole result set. A single
server field is allocated by libpq before its length can be checked: this is not
an arbitrary-query memory sandbox. Only trusted bounded persistence SQL is admitted.
Server statement/lock timeouts default to 10s/5s; callers choose shorter I/O deadlines.
Admission config/bounds errors send nothing. SQL errors include only SQLSTATE and
require caller rollback when inside a transaction. Receive bounds, network errors
and deadlines close the connection and report unknown: a mutation may have
committed. Resolve durable request IDs at M7.5; never treat disconnect as rollback.
Allocation exceptions propagate to the host boundary; do not catch and reuse an
incomplete session. No PostgreSQL diagnostic strings or connection values are logged.
## Migrations and privilege boundary
`tools/migrate-postgres.sh` uses psql environment/service files, no URI arguments.
It requires exactly server 18.6 and serializes migrations with a transaction
advisory lock. It hashes each checked-in SQL file and applies everything in one
transaction. The recorded history must be an exact prefix: clean->v1->v2, v1->v2,
or a v2 no-op. Unknown versions, checksum changes and partial foreign schemas fail
closed. There is no down migration or destructive repair. The u64 domain rejects
fractional, negative and above-u64 values without signed truncation.
Use a dedicated owner/migrator login with CREATE on the database. Before
migrating, operators create a NOLOGIN runtime group role and pass its name as
`EVO_PG_RUNTIME_ROLE`. The runtime login is a separate member of that group only.
Every migrate run, including a no-op, audits effective privileges of the group and
every role transitively in it, and rolls back on any violation:
- privileged attributes or owner identity;
- membership in any other role, whether inherited or SET-able (for example
`pg_write_all_data`);
- any table privilege in `evolution`, including via PUBLIC;
- CREATE on the schema, the database or `public`;
- EXECUTE outside the ten fenced functions (schema v5).
Attribute checks alone are not enough: a login with `pg_write_all_data` can update
lease rows directly. Rerun the migrate Job after adding or changing runtime logins
or grants; it is the audit. Grants made later without a rerun are not detected. Bootstrap revokes PUBLIC access to the schema, tables and
default function execution. Operators own database/role provisioning; the migration
never creates logins or passwords, and never grants superuser.
## Fenced region ownership (schema v2, ADR-0060)
In schema v3 the runtime role has schema USAGE and EXECUTE on six SECURITY DEFINER
functions: `acquire_region_lease`, `renew_region_lease`, `release_region_lease`,
`load_region_head`, `publish_region_checkpoint` and `load_region_checkpoint`. It
cannot read or write tables, create objects, call internal check/parse functions or
provision regions. (Schema v2's `advance_region_head` is dropped in v3.) Each mutation locks the lease row
and checks holder, epoch and expiry with `clock_timestamp()` after the lock wait.
Clients never send times. Rejections: EVL01 busy, EVL02 fenced, EVL03 head mismatch,
EVL04 not provisioned, EVL05 epoch/sequence exhausted, EVL06 unsupported
profile/version/build or a changed rules/quantum, EVL07 reused commit ID, and 22023
malformed arguments or manifest. `evo::postgres` maps them to `LeaseStoreError`; a
timeout or broken connection is `unknown`.
## Checkpoint publication (schema v3, ADR-0061)
Each commit row stores the exact 280-byte manifest, and a constraint ties the
commit digest to `sha256(manifest)`. The head has a composite foreign key to its
commit, so even the owner cannot silently repoint or edit it.
Payloads are immutable files under a trusted object root:
`<root>/w<world>/r<region>/<sha256 hex>`, mode 0700 directories and 0600 files.
Mount a dedicated volume that only the region host's user can write. Nothing is
ever deleted; orphans from failed transactions are expected.
Operator steps:
1. Record each promoted build (image digest, 32 bytes) before a host uses it:
`INSERT INTO evolution.retained_builds(build_identity) VALUES ('\x<64 hex>');`
2. Provision genesis only through `provision_environmental_region` on an owner
connection. It uploads and verifies the tick-0 payload, then calls the
owner-only `provision_region_checkpoint(manifest)`.
Schema v4 (ADR-0062) additionally admits profile 2 (`environmental-lineage-v1`,
payload ≤ 4,268,144 bytes) and requires every commit to keep its predecessor's
profile, history, RNG and durability declarations. Provision history-capable
regions with `provision_lineage_region`; hosts use `publish/recover_lineage_checkpoint`.
Schema v5 (ADR-0063) requires manifest v2 with a decision block and adds:
- `region_decisions` and `region_outbox`, with one terminal outcome per actor;
- `resolve_region_commit`, `load_region_decisions`, `pending_region_outbox` and
`acknowledge_region_outbox`, bringing the runtime function set to ten.
Triggers make commits and decisions immutable (EVL09); outbox rows may only gain
a first delivery time. Owners can disable triggers, so treat that as an audited
break-glass action. EVL08 means an outcome was already final. After an
ambiguous publication, resolve the exact prepared bytes before doing anything
else.
There is no operator CLI for provisioning yet, and no real world has been
provisioned. Migrating v2→v3 fails closed if any v2 head exists, because v2 heads
have no payload.
## Host lease discipline
Hosts use `LeaseAuthority` with `LeasePolicy` defaults: 20 s lease, 5 s renewal
interval, 2 s safety margin, 2 s renewal deadline and 5 s commit deadline. The
local deadline runs from request send time on CLOCK_BOOTTIME. If a renewal outcome
is unknown, keep the earlier deadline and reconnect. Once fenced, stop admission,
simulation and output, then acquire a new epoch and recover. Never resume the old
epoch.
## Kubernetes Secrets
Create Secret **evolution-postgres-migrator** in evolution-dev outside Git. Keys:
`pg_service.conf`, `pgpass`, `ca.crt`. The service file has section
`[evolution-migrator]` with host, numeric hostaddr, port, dbname, user,
sslmode=verify-full, sslrootcert=/run/postgres/ca.crt and connect_timeout=5.
pgpass uses libpq's escaped host:port:database:user:password format. Do not paste
values into chat, manifests or image arguments. Configure cluster Secret RBAC and
encryption at rest through the existing platform policy.
The example Job mounts the Secret read-only; the entrypoint copies pgpass into a
private 0600 temporary file because libpq rejects group-readable password files.
The copy is removed on exit. Use a distinct **evolution-postgres-runtime** Secret
with a restricted login for the later region pod; never mount migrator credentials
into it. Rotation requires a new connection/Job; no hot reload is claimed.
`deploy/kubernetes/postgres-migration/job.yaml.example` is intentionally outside the
active Argo overlay: select a built immutable postgres-tools digest and verify DB
target/role before adding it. Existing default-deny egress needs a specific DB and
DNS allowance chosen for the actual endpoint. No live database/Secret was created
and no World0 resources are touched by this bootstrap.
CI builds the `postgres-tools` Docker target and publishes it when Gitea variable
`HARBOR_POSTGRES_REPOSITORY` is set (for example `aevolution-postgres-tools`). It uses
the existing HARBOR_PROJECT and Harbor robot secrets. Pin the reported registry
digest in the example Job; do not use the simulation image. See deploy/README.md.
Local Unix-socket integration is qualified; remote TLS/authentication, actual Secret
mounts and container execution still require environment-specific verification.