Files
aevolution/docs/infrastructure/POSTGRESQL.md
T
dingoandClaude Opus 5.5 5d4f59f901 Implement M7.4a checkpoint publication and verified recovery
ADR-0061 publishes environmental-v1 full checkpoints. Manifest v1 is a
fixed 280-byte record (identity, chain, lease, versions, build, zero
RNG/history declarations, durability policy, payload length/SHA-256)
that PostgreSQL parses and digests itself. Schema v3 adds region_commits
with a digest CHECK and a composite head FK, owner-managed retained
builds, fenced publish/load functions and owner-only genesis; it drops
the bare v2 head kernel and fails closed while v2 heads exist.

Linux evo::region_store adds OpenSSL SHA-256, an immutable
content-addressed filesystem object store (temp+fsync+link, no
overwrite/delete, O_NOFOLLOW exact-length verified reads), and
publication (validate -> upload -> read back -> fenced transaction ->
receipt check) and recovery (re-digest, allowlist, verify, full restore).
Portable evo::region_checkpoint holds the codec and payload validation.

Validated: Windows CTest 56/56; WSL Release, clang-tidy and ASan/UBSan
private-PostgreSQL suites; four SQL and one host mutation each caught.
M7.4 stays open for 7.4b (lineage archive codec and history binding).
No Docker/Gitea CI, remote store, process kill or deployment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 15:25:27 -04:00

8.8 KiB

PostgreSQL bootstrap (M7.2) and region fencing (M7.3)

ADR-0059 pins server/libpq 18.6. evo::postgres is optional, Linux-only and separate from all simulation targets. Preparation explicitly downloads a verified source archive; ordinary CMake configure does not fetch anything. PostgreSQL license is permissive; source COPYRIGHT and system TLS license information must accompany distributed tooling. No custom hashing/cryptography is introduced.

Local isolated verification

Linux prerequisites: GCC14, CMake>=3.28, Ninja, make, bison, flex, curl, bzip2, OpenSSL development headers and zlib development headers. Optional clang19 tools support analysis/ASan. Run as a normal user, not root:

bash tools/prepare-postgres.sh
cmake --preset release -B build/postgres -DCMAKE_CXX_COMPILER=g++-14 \
  -DEVO_BUILD_POSTGRES=ON -DPostgreSQL_ROOT="$PWD/.cache/postgres/install"
cmake --build build/postgres --target evo-postgres-tests --parallel 2
bash tools/ci/postgres-check.sh "$PWD/build/postgres/evo-postgres-tests"

The test script overrides all connection coordinates and creates a new private 0700 /tmp directory, private Unix socket and disposable local cluster. It creates the evo_runtime_test group and the evo_runtime_login member, then runs fencing tests as that real runtime login. TCP is off; local trust applies only inside that private directory. Cluster stops on exit; logs/data are retained there for diagnosis. It never uses production PG credentials or deletes a pre-existing database. Do not point this script at a shared server.

Adapter contract

One connection belongs to one I/O owner; one outstanding parameterized statement. No hidden threads, pool, transaction replay or automatic reconnect. Connections and queries take 1..30000ms deadlines on a steady clock. Remote hostaddr must be numeric, host supplies TLS identity, CA file is explicit, TLS verify-full required. Local sockets require no PGHOSTADDR/PGSERVICE override. GSS negotiation is disabled. Query SQL <=16KiB, <=16 parameters of <=4KiB, response <=128 rows/16 columns/64KiB decoded values; single-row mode prevents accumulating a whole result set. A single server field is allocated by libpq before its length can be checked: this is not an arbitrary-query memory sandbox. Only trusted bounded persistence SQL is admitted. Server statement/lock timeouts default to 10s/5s; callers choose shorter I/O deadlines.

Admission config/bounds errors send nothing. SQL errors include only SQLSTATE and require caller rollback when inside a transaction. Receive bounds, network errors and deadlines close the connection and report unknown: a mutation may have committed. Resolve durable request IDs at M7.5; never treat disconnect as rollback. Allocation exceptions propagate to the host boundary; do not catch and reuse an incomplete session. No PostgreSQL diagnostic strings or connection values are logged.

Migrations and privilege boundary

tools/migrate-postgres.sh uses psql environment/service files, no URI arguments. It requires exactly server 18.6 and serializes migrations with a transaction advisory lock. It hashes each checked-in SQL file and applies everything in one transaction. The recorded history must be an exact prefix: clean->v1->v2, v1->v2, or a v2 no-op. Unknown versions, checksum changes and partial foreign schemas fail closed. There is no down migration or destructive repair. The u64 domain rejects fractional, negative and above-u64 values without signed truncation.

Use a dedicated owner/migrator login with CREATE on the database. Before migrating, operators create a NOLOGIN runtime group role and pass its name as EVO_PG_RUNTIME_ROLE. The runtime login is a separate member of that group only. Every migrate run, including a no-op, audits effective privileges of the group and every role transitively in it, and rolls back on any violation:

  • privileged attributes or owner identity;
  • membership in any other role, whether inherited or SET-able (for example pg_write_all_data);
  • any table privilege in evolution, including via PUBLIC;
  • CREATE on the schema, the database or public;
  • EXECUTE outside the six fenced functions (schema v3). Attribute checks alone are not enough: a login with pg_write_all_data can update lease rows directly. Rerun the migrate Job after adding or changing runtime logins or grants; it is the audit. Grants made later without a rerun are not detected. Bootstrap revokes PUBLIC access to the schema, tables and default function execution. Operators own database/role provisioning; the migration never creates logins or passwords, and never grants superuser.

Fenced region ownership (schema v2, ADR-0060)

In schema v3 the runtime role has schema USAGE and EXECUTE on six SECURITY DEFINER functions: acquire_region_lease, renew_region_lease, release_region_lease, load_region_head, publish_region_checkpoint and load_region_checkpoint. It cannot read or write tables, create objects, call internal check/parse functions or provision regions. (Schema v2's advance_region_head is dropped in v3.) Each mutation locks the lease row and checks holder, epoch and expiry with clock_timestamp() after the lock wait. Clients never send times. Rejections: EVL01 busy, EVL02 fenced, EVL03 head mismatch, EVL04 not provisioned, EVL05 epoch/sequence exhausted, EVL06 unsupported profile/version/build or a changed rules/quantum, EVL07 reused commit ID, and 22023 malformed arguments or manifest. evo::postgres maps them to LeaseStoreError; a timeout or broken connection is unknown.

Checkpoint publication (schema v3, ADR-0061)

Each commit row stores the exact 280-byte manifest, and a constraint ties the commit digest to sha256(manifest). The head has a composite foreign key to its commit, so even the owner cannot silently repoint or edit it.

Payloads are immutable files under a trusted object root: <root>/w<world>/r<region>/<sha256 hex>, mode 0700 directories and 0600 files. Mount a dedicated volume that only the region host's user can write. Nothing is ever deleted; orphans from failed transactions are expected.

Operator steps:

  1. Record each promoted build (image digest, 32 bytes) before a host uses it: INSERT INTO evolution.retained_builds(build_identity) VALUES ('\x<64 hex>');
  2. Provision genesis only through provision_environmental_region on an owner connection. It uploads and verifies the tick-0 payload, then calls the owner-only provision_region_checkpoint(manifest).

There is no operator CLI for provisioning yet, and no real world has been provisioned. Migrating v2→v3 fails closed if any v2 head exists, because v2 heads have no payload.

Host lease discipline

Hosts use LeaseAuthority with LeasePolicy defaults: 20 s lease, 5 s renewal interval, 2 s safety margin, 2 s renewal deadline and 5 s commit deadline. The local deadline runs from request send time on CLOCK_BOOTTIME. If a renewal outcome is unknown, keep the earlier deadline and reconnect. Once fenced, stop admission, simulation and output, then acquire a new epoch and recover. Never resume the old epoch.

Kubernetes Secrets

Create Secret evolution-postgres-migrator in evolution-dev outside Git. Keys: pg_service.conf, pgpass, ca.crt. The service file has section [evolution-migrator] with host, numeric hostaddr, port, dbname, user, sslmode=verify-full, sslrootcert=/run/postgres/ca.crt and connect_timeout=5. pgpass uses libpq's escaped host:port:database:user:password format. Do not paste values into chat, manifests or image arguments. Configure cluster Secret RBAC and encryption at rest through the existing platform policy.

The example Job mounts the Secret read-only; the entrypoint copies pgpass into a private 0600 temporary file because libpq rejects group-readable password files. The copy is removed on exit. Use a distinct evolution-postgres-runtime Secret with a restricted login for the later region pod; never mount migrator credentials into it. Rotation requires a new connection/Job; no hot reload is claimed.

deploy/kubernetes/postgres-migration/job.yaml.example is intentionally outside the active Argo overlay: select a built immutable postgres-tools digest and verify DB target/role before adding it. Existing default-deny egress needs a specific DB and DNS allowance chosen for the actual endpoint. No live database/Secret was created and no World0 resources are touched by this bootstrap.

CI builds the postgres-tools Docker target and publishes it when Gitea variable HARBOR_POSTGRES_REPOSITORY is set (for example aevolution-postgres-tools). It uses the existing HARBOR_PROJECT and Harbor robot secrets. Pin the reported registry digest in the example Job; do not use the simulation image. See deploy/README.md. Local Unix-socket integration is qualified; remote TLS/authentication, actual Secret mounts and container execution still require environment-specific verification.