feat(backup): M8.7 world backup bundles and verified restore (ADR-0074)
Online, snapshot-consistent bundles (pg_dump at an exported snapshot, immutable objects verified by digest, MANIFEST of versions/builds/heads, SHA256SUMS); a fail-closed restore that audits every region's full history on independent storage; an isolated rebuild through clones that serves under load; an operator-run in-cluster backup streamed off the cluster. backup-check.sh runs in every PostgreSQL build, with corruption, missing-object, tampered-manifest and non-empty-target refusals. Destination, encryption, schedule and WAL archiving remain on the backlog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -192,7 +192,7 @@ To delete World0 data or identity on purpose:
|
||||
1. Record the operation and its approval.
|
||||
2. Use a credential whose groups include `aevolution:world0-break-glass`, for
|
||||
example a short-lived client certificate with that organization.
|
||||
3. Take and verify a backup first (M8.7).
|
||||
3. Take and verify a backup first (M8.7; see "Backups" below).
|
||||
|
||||
Removing the admission policy itself is the break-glass of last resort. It needs
|
||||
cluster-admin and must be recorded the same way. Never test by deleting World0: the
|
||||
@@ -206,3 +206,29 @@ verify script uses server-side dry runs.
|
||||
- **World0 volumes.** `local-path` provisions volumes with Delete. Patch each World0
|
||||
PV to `Retain` when it is bound, and the policy then keeps it there. Volumes stay
|
||||
node-local until M8.3.
|
||||
|
||||
## Backups (M8.7, ADR-0074)
|
||||
|
||||
Dev and stage only; the tools refuse World0. From a machine with kubectl access
|
||||
that is not the hypervisor host:
|
||||
|
||||
```bash
|
||||
# 1. Sync the environment's application once so the egress policy admits the
|
||||
# backup component and the store overlay pins a postgres-tools image that
|
||||
# contains pg_dump (any image built after ADR-0074).
|
||||
# 2. Back up to a local directory (the destination until one is chosen):
|
||||
tools/backup/cluster-backup.sh dev --to /path/to/backups
|
||||
# 3. Restore-test it on independent storage and rebuild it as isolated world 900
|
||||
# (WSL: the pinned PostgreSQL build and the Linux PostgreSQL build of the tools).
|
||||
PATH="$PWD/.cache/postgres/install/bin:$PATH" \
|
||||
LD_LIBRARY_PATH="$PWD/.cache/postgres/install/lib" \
|
||||
EVO_WORLD_CLONE_BIN="$PWD/build/postgres-linux/evo-world-clone" \
|
||||
EVO_SERVER_BIN="$PWD/build/postgres-linux/evo-server" \
|
||||
tools/backup/verify-backup.sh /path/to/backups/dev-<time> --isolate 900
|
||||
```
|
||||
|
||||
Only `verify-backup.sh` output that ends in `PASS restore verified and world ...
|
||||
rebuilt as isolated world ...` counts as restore evidence. The restore uses fresh
|
||||
roles on a throwaway cluster; nothing reaches the source database. A restored copy
|
||||
keeps the source world identity and is never served; restoring in place after host
|
||||
loss is M8.8.
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
# Operations and disaster recovery
|
||||
|
||||
Status: proposal requiring measured acceptance before World0. No deployment,
|
||||
backup or distributed restore has been exercised in this foundation.
|
||||
Status: proposal requiring measured acceptance before World0. M8.7 (ADR-0074)
|
||||
implements snapshot-consistent world backup bundles and a verified restore on
|
||||
independent storage with an isolated rebuild; the destination, encryption,
|
||||
schedule and WAL archiving are pending, and no host-loss restore in place has been
|
||||
exercised (M8.8).
|
||||
|
||||
| Failure | Proposed objective / gate |
|
||||
|---|---|
|
||||
@@ -36,8 +39,10 @@ pre-upgrade checkpoints, subject to capacity evidence. Genome/ancestry/lifecycle
|
||||
history does not expire with backup retention. GC respects all clone/backup pins
|
||||
and complete replay chains.
|
||||
|
||||
1. First implementation uses a coordinated admission/transfer barrier, avoiding
|
||||
an unfinished distributed-snapshot algorithm.
|
||||
1. First implementation (ADR-0074) needs no admission barrier: one exported
|
||||
database snapshot is the cut, and objects are immutable and verified before
|
||||
commit, so copying them after the snapshot is consistent. A barrier returns
|
||||
only if state ever spans more than one database.
|
||||
2. Commit region boundaries; seal DB recovery point, referenced objects, ongoing
|
||||
transfer state, region watermarks and versions into the manifest.
|
||||
3. Verify/copy objects, restore isolated PostgreSQL at the compatible point.
|
||||
|
||||
Reference in New Issue
Block a user