Files
dingoandClaude Opus 5.5 2d950060db Add M8.3 dev/stage stores: Retain volumes, free-space reserve, store check
Keep the filesystem object store on a node-local Retain-class PVC per
environment (ADR-0070), with the database on the external PostgreSQL VM.
Because local-path does not enforce sizes, the object store now refuses
new objects below a free-space reserve with a distinct full error, so a
full volume publishes and acknowledges nothing. evo-world-clone gains
check-store to qualify volume retention, reserve, database version/TLS
and the runtime privilege boundary from inside the cluster; it now ships
in the server image. Add the storage class, object claims, hand-applied
store Jobs, policy rules forbidding password env vars, and the runbook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:51:15 -04:00

85 lines
4.2 KiB
C++

#pragma once
#include "evo/persistence/region_lease.hpp"
#include <cstddef>
#include <cstdint>
#include <expected>
#include <filesystem>
#include <memory>
#include <optional>
#include <span>
#include <vector>
// Immutable content-addressed payload store on a trusted local/mounted volume
// (ADR-0061). Linux only. One namespace directory per world/region; the object
// key is the lowercase hex SHA-256 of its bytes. There is no delete or overwrite:
// unreferenced uploads are orphans and remain until a reachability-proven GC exists.
namespace evo::persistence {
enum class ObjectError : std::uint8_t {
invalid, // Bad root, namespace or object reference; nothing done.
missing, // No object with that key.
corrupt, // Length or digest mismatch, or not a regular file: quarantine.
bounds, // Larger than the configured maximum; rejected before allocation.
io, // Write/sync/link failure; the object is absent or unverified.
full, // Writing would leave less free space than the reserve; nothing written.
};
// Test-only fault injection and accounting for directory fsyncs. When passed to
// open() it must outlive the store; production callers pass nothing.
struct ObjectStoreFaults {
std::uint32_t fail_directory_syncs{}; // The next N directory syncs report failure.
std::uint64_t directory_syncs{}; // Directory syncs that succeeded.
};
struct VerifiedObject {
Sha256Digest sha256{};
std::uint64_t bytes{};
friend bool operator==(const VerifiedObject&, const VerifiedObject&) = default;
};
class FilesystemObjectStore {
public:
// trusted_root must be an existing real directory (not a symlink): a missing or
// unmounted volume is refused, never created. Namespace directories
// w<world>/r<region> are created 0700; their parents are synced on every open, so
// a creation whose sync failed earlier is made durable now. reserve_bytes (M8.3):
// put() refuses new objects that would leave less free space than this on the
// volume, because node-local volumes do not enforce their requested size.
[[nodiscard]] static std::expected<std::unique_ptr<FilesystemObjectStore>, ObjectError>
open(const std::filesystem::path& trusted_root, RegionKey key, std::size_t max_object_bytes,
ObjectStoreFaults* faults = nullptr, std::uint64_t reserve_bytes = 0);
FilesystemObjectStore(const FilesystemObjectStore&) = delete;
FilesystemObjectStore& operator=(const FilesystemObjectStore&) = delete;
FilesystemObjectStore(FilesystemObjectStore&&) = delete;
FilesystemObjectStore& operator=(FilesystemObjectStore&&) = delete;
~FilesystemObjectStore();
// Refuses with `full`, before writing anything, when the volume's available
// space minus the object is below the reserve.
// Writes a private temporary file, fsyncs it, links it to its key without
// replacing anything, then fsyncs the directory. Identical bytes already stored
// (an earlier attempt whose directory sync failed, or a concurrent installer)
// are verified and fsynced, file and directory, before success is reported. A
// differing existing object is corrupt, never replaced.
[[nodiscard]] std::expected<VerifiedObject, ObjectError>
put(std::span<const std::uint8_t> bytes) const;
// Reads only an exact-length regular file and verifies its SHA-256.
[[nodiscard]] std::expected<std::vector<std::uint8_t>, ObjectError>
get(const VerifiedObject& object) const;
[[nodiscard]] const std::filesystem::path& directory() const noexcept {
return directory_;
}
// Bytes available to this process on the volume; nullopt if unknown.
[[nodiscard]] std::optional<std::uint64_t> available_bytes() const noexcept;
[[nodiscard]] std::uint64_t reserve_bytes() const noexcept {
return reserve_;
}
private:
FilesystemObjectStore(std::filesystem::path directory, int descriptor, std::size_t maximum,
ObjectStoreFaults* faults, std::uint64_t reserve);
[[nodiscard]] std::expected<VerifiedObject, ObjectError>
durable_existing(const VerifiedObject& object) const;
std::filesystem::path directory_;
int descriptor_;
std::size_t maximum_;
ObjectStoreFaults* faults_;
std::uint64_t reserve_;
};
} // namespace evo::persistence