Fix object reuse durability and refuse unsafe pre-v5 upgrades
Review P1 (object durability): a name installed by a put whose directory fsync failed was later reused as success without syncing again, so a publication could acknowledge an object that might vanish on power loss. Reuse (existing and lost-link-race paths) now fsyncs the verified file and the directory before succeeding, and namespace opens always sync their parent. A test-only ObjectStoreFaults hook injects directory sync failures; tests cover failed sync then retries, namespace creation syncs, and publication refusing to proceed until durability holds. Review P1 (pre-v5 history): v5 recovery requires one durable decision per terminal actor, but v3/v4 published checkpoints carry none, so an accepted upgrade could make regions unrecoverable. pre_v5_guard.sql now runs before 0005 in every upgrade path and refuses (EVL10) when any published pre-v5 checkpoint exists; genesis-only regions pass. 0005 is unchanged. CI publishes a real v4 checkpoint and proves the upgrade is refused with nothing changed; a committed death without a decision is shown to recover as quarantined history. Validated: PostgreSQL suites Release x3, clang-tidy, ASan/UBSan x2; reverting each fix is caught by its test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -20,6 +20,12 @@ enum class ObjectError : std::uint8_t {
|
||||
bounds, // Larger than the configured maximum; rejected before allocation.
|
||||
io, // Write/sync/link failure; the object is absent or unverified.
|
||||
};
|
||||
// Test-only fault injection and accounting for directory fsyncs. When passed to
|
||||
// open() it must outlive the store; production callers pass nothing.
|
||||
struct ObjectStoreFaults {
|
||||
std::uint32_t fail_directory_syncs{}; // The next N directory syncs report failure.
|
||||
std::uint64_t directory_syncs{}; // Directory syncs that succeeded.
|
||||
};
|
||||
struct VerifiedObject {
|
||||
Sha256Digest sha256{};
|
||||
std::uint64_t bytes{};
|
||||
@@ -28,9 +34,11 @@ struct VerifiedObject {
|
||||
class FilesystemObjectStore {
|
||||
public:
|
||||
// trusted_root must be an existing real directory (not a symlink). Namespace
|
||||
// directories w<world>/r<region> are created 0700 and their parents synced.
|
||||
// directories w<world>/r<region> are created 0700; their parents are synced on
|
||||
// every open, so a creation whose sync failed earlier is made durable now.
|
||||
[[nodiscard]] static std::expected<std::unique_ptr<FilesystemObjectStore>, ObjectError>
|
||||
open(const std::filesystem::path& trusted_root, RegionKey key, std::size_t max_object_bytes);
|
||||
open(const std::filesystem::path& trusted_root, RegionKey key, std::size_t max_object_bytes,
|
||||
ObjectStoreFaults* faults = nullptr);
|
||||
FilesystemObjectStore(const FilesystemObjectStore&) = delete;
|
||||
FilesystemObjectStore& operator=(const FilesystemObjectStore&) = delete;
|
||||
FilesystemObjectStore(FilesystemObjectStore&&) = delete;
|
||||
@@ -38,7 +46,9 @@ class FilesystemObjectStore {
|
||||
~FilesystemObjectStore();
|
||||
// Writes a private temporary file, fsyncs it, links it to its key without
|
||||
// replacing anything, then fsyncs the directory. Identical bytes already stored
|
||||
// are verified and reused; a differing existing object is corrupt, never replaced.
|
||||
// (an earlier attempt whose directory sync failed, or a concurrent installer)
|
||||
// are verified and fsynced, file and directory, before success is reported. A
|
||||
// differing existing object is corrupt, never replaced.
|
||||
[[nodiscard]] std::expected<VerifiedObject, ObjectError>
|
||||
put(std::span<const std::uint8_t> bytes) const;
|
||||
// Reads only an exact-length regular file and verifies its SHA-256.
|
||||
@@ -49,9 +59,13 @@ class FilesystemObjectStore {
|
||||
}
|
||||
|
||||
private:
|
||||
FilesystemObjectStore(std::filesystem::path directory, int descriptor, std::size_t maximum);
|
||||
FilesystemObjectStore(std::filesystem::path directory, int descriptor, std::size_t maximum,
|
||||
ObjectStoreFaults* faults);
|
||||
[[nodiscard]] std::expected<VerifiedObject, ObjectError>
|
||||
durable_existing(const VerifiedObject& object) const;
|
||||
std::filesystem::path directory_;
|
||||
int descriptor_;
|
||||
std::size_t maximum_;
|
||||
ObjectStoreFaults* faults_;
|
||||
};
|
||||
} // namespace evo::persistence
|
||||
|
||||
Reference in New Issue
Block a user