Fix M5.7 compaction and failure recovery
This commit is contained in:
@@ -27,20 +27,27 @@ state silently. No automatic orphan deletion in this gate.
|
||||
|
||||
Snapshot checkpoint validates bytes by restoring them under budget before publication.
|
||||
A journal commit must match the current checkpoint's exact bytes and high-water, and
|
||||
pass detached replay. Only one journal segment is active per checkpoint; checkpoint
|
||||
after replay to compact it. Recovery validates the selected manifest and referenced
|
||||
payloads using M5.6, then replays the optional segment before returning a candidate.
|
||||
Existing valid manifest stays the source until new publication completes.
|
||||
pass detached replay. Only one journal segment is active per checkpoint. While one is
|
||||
published, compaction accepts only the byte-identical canonical snapshot of recovery's
|
||||
replayed result; command high-water alone cannot prove continuation or prevent a fork.
|
||||
Recovery validates the selected manifest and referenced payloads using M5.6, then
|
||||
replays the optional segment before returning a candidate. Existing valid manifest
|
||||
stays the source until new publication completes. A poisoned handle refuses recovery,
|
||||
reports no current generation, and requires reopen to resolve an ambiguous publication
|
||||
error.
|
||||
|
||||
Implement the minimal native OS boundary directly: POSIX lock/write/fsync/link/rename/
|
||||
directory fsync; Windows exclusive CreateFile and write-through replacement. No third
|
||||
party library. Root directory is trusted and locally administered. Files are not
|
||||
network-safe/authenticated; FNV detects accidental corruption only.
|
||||
|
||||
Provide one-shot failure injection at payload sync, manifest sync, and manifest publish
|
||||
for deterministic crash-boundary tests. After injected interruption, discard the handle
|
||||
and reopen. A failure before manifest replacement yields old state; after replacement
|
||||
yields new state. Allocation/OS errors do not clear dirty state.
|
||||
Provide one-shot failure injection at payload sync, manifest sync, manifest publish,
|
||||
partial payload/manifest write (out of space), and post-replacement directory sync for
|
||||
deterministic crash-boundary tests. Partial-write failures leave only an ignored temp
|
||||
file and allow retry; injected interruptions poison the handle. After an interruption
|
||||
or ambiguous sync failure, discard the handle and reopen. A failure before manifest
|
||||
replacement yields old state; after replacement yields new state. Allocation/OS errors
|
||||
do not clear dirty state.
|
||||
|
||||
## Alternatives Considered
|
||||
In-place overwrite can destroy the old checkpoint on short write. A single manifest can
|
||||
@@ -60,6 +67,7 @@ differ. Same-host files do not survive host loss. Fail-closed corruption may req
|
||||
operator recovery. No production DR/RPO claim.
|
||||
|
||||
## Consequences
|
||||
Test first checkpoint, journal, compaction, restart continuation, competing lock, corrupt
|
||||
payload/manifest, every injected boundary, missing sequence and full-disk/OS errors.
|
||||
Test first checkpoint, journal, exact replay compaction, divergent-state rejection,
|
||||
restart continuation, competing lock, corrupt payload/manifest, every injected boundary,
|
||||
partial-write recovery, missing sequence and full-disk/OS errors.
|
||||
Record Linux checks separately. M5.8 performs the whole create/edit/compact/restart proof.
|
||||
|
||||
@@ -28,14 +28,33 @@ Executed on Windows x64 / MSVC 19.44.35207:
|
||||
- `tools/windows-check.ps1 -Preset release -Action format`: passed.
|
||||
- `git diff --check`: passed before commit.
|
||||
|
||||
New coverage validates checkpoint/journal/recovery, compaction over a published
|
||||
journal, exclusive writer lock, all three injected publication boundaries, corrupt
|
||||
manifest and missing published payload failure. Before-manifest failures recover
|
||||
the previous generation; after-manifest replacement recovers the new generation.
|
||||
Actual power-cut/full-disk behavior, POSIX syscall path, Linux CI/libFuzzer, and
|
||||
filesystem-controller flush guarantees remain unverified locally. There is no
|
||||
automatic orphan cleanup, production storage adapter, Kubernetes deployment or
|
||||
World0 change.
|
||||
New coverage validates checkpoint/journal/recovery, exact compaction over a published
|
||||
journal, divergent-state rejection, exclusive writer lock, injected publication
|
||||
boundaries, partial-write/out-of-space outcomes, corrupt manifest and missing payload.
|
||||
Before-manifest failures recover the previous state; after replacement, reopen
|
||||
recovers the new state. Actual power-cut/disk-full behavior, POSIX syscall path,
|
||||
Linux CI/libFuzzer, and filesystem-controller flush guarantees remain unverified
|
||||
locally. There is no automatic orphan cleanup, production storage adapter,
|
||||
Kubernetes deployment or World0 change.
|
||||
|
||||
### M5.7 review corrections
|
||||
|
||||
The follow-up review found that command high-water alone allowed a divergent
|
||||
checkpoint to replace an active journal. Checkpoint now compares its canonical bytes
|
||||
with the fully recovered journal result before compaction. Poisoned handles reject
|
||||
recovery; reopen resolves ambiguous post-replacement directory-sync failures.
|
||||
Interruption tests now use a different edited payload and compare recovered canonical
|
||||
state and debris across the commit point. Deterministic partial-write/out-of-space
|
||||
injection covers payload and manifest temporaries, including retry, and a directory
|
||||
sync failure hook verifies poison/reopen behavior. The injection models a partial
|
||||
write outcome; it is not a physical disk-full or power-cut test.
|
||||
|
||||
Revalidated after these corrections on Windows x64 / MSVC 19.44.35207:
|
||||
|
||||
- Release configure/build and 47/47 CTests passed.
|
||||
- clang-tidy analysis build and 47/47 CTests passed.
|
||||
- ASan build and 47/47 CTests passed.
|
||||
- clang-format and `git diff --check` passed.
|
||||
|
||||
## Tools and host
|
||||
|
||||
|
||||
@@ -82,6 +82,9 @@ and [the manifest schema](../../schemas/local-world-manifest.md). Two alternatin
|
||||
manifest slots select immutable M5.6 snapshot/journal payloads. Recovery validates
|
||||
the selected snapshot and optional journal into a detached store. The manifest
|
||||
replacement is the local commit point; this provides no multi-host durability,
|
||||
production lease, authentication or disaster recovery guarantee.
|
||||
production lease, authentication or disaster recovery guarantee. Compaction while a
|
||||
journal is active accepts only its byte-identical replay result, and poisoned handles
|
||||
must reopen before recovery. Partial-write and post-replacement sync failures have
|
||||
deterministic test injection.
|
||||
M5.6 does not write files, fsync, delete journals, clear dirty state or recover a live
|
||||
server process. M7 supplies the production durability/ownership path.
|
||||
|
||||
Reference in New Issue
Block a user