Add M8.3 dev/stage stores: Retain volumes, free-space reserve, store check
Keep the filesystem object store on a node-local Retain-class PVC per environment (ADR-0070), with the database on the external PostgreSQL VM. Because local-path does not enforce sizes, the object store now refuses new objects below a free-space reserve with a distinct full error, so a full volume publishes and acknowledges nothing. evo-world-clone gains check-store to qualify volume retention, reserve, database version/TLS and the runtime privilege boundary from inside the cluster; it now ships in the server image. Add the storage class, object claims, hand-applied store Jobs, policy rules forbidding password env vars, and the runbook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,7 @@
|
||||
#include <expected>
|
||||
#include <filesystem>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <span>
|
||||
#include <vector>
|
||||
|
||||
@@ -19,6 +20,7 @@ enum class ObjectError : std::uint8_t {
|
||||
corrupt, // Length or digest mismatch, or not a regular file: quarantine.
|
||||
bounds, // Larger than the configured maximum; rejected before allocation.
|
||||
io, // Write/sync/link failure; the object is absent or unverified.
|
||||
full, // Writing would leave less free space than the reserve; nothing written.
|
||||
};
|
||||
// Test-only fault injection and accounting for directory fsyncs. When passed to
|
||||
// open() it must outlive the store; production callers pass nothing.
|
||||
@@ -33,17 +35,22 @@ struct VerifiedObject {
|
||||
};
|
||||
class FilesystemObjectStore {
|
||||
public:
|
||||
// trusted_root must be an existing real directory (not a symlink). Namespace
|
||||
// directories w<world>/r<region> are created 0700; their parents are synced on
|
||||
// every open, so a creation whose sync failed earlier is made durable now.
|
||||
// trusted_root must be an existing real directory (not a symlink): a missing or
|
||||
// unmounted volume is refused, never created. Namespace directories
|
||||
// w<world>/r<region> are created 0700; their parents are synced on every open, so
|
||||
// a creation whose sync failed earlier is made durable now. reserve_bytes (M8.3):
|
||||
// put() refuses new objects that would leave less free space than this on the
|
||||
// volume, because node-local volumes do not enforce their requested size.
|
||||
[[nodiscard]] static std::expected<std::unique_ptr<FilesystemObjectStore>, ObjectError>
|
||||
open(const std::filesystem::path& trusted_root, RegionKey key, std::size_t max_object_bytes,
|
||||
ObjectStoreFaults* faults = nullptr);
|
||||
ObjectStoreFaults* faults = nullptr, std::uint64_t reserve_bytes = 0);
|
||||
FilesystemObjectStore(const FilesystemObjectStore&) = delete;
|
||||
FilesystemObjectStore& operator=(const FilesystemObjectStore&) = delete;
|
||||
FilesystemObjectStore(FilesystemObjectStore&&) = delete;
|
||||
FilesystemObjectStore& operator=(FilesystemObjectStore&&) = delete;
|
||||
~FilesystemObjectStore();
|
||||
// Refuses with `full`, before writing anything, when the volume's available
|
||||
// space minus the object is below the reserve.
|
||||
// Writes a private temporary file, fsyncs it, links it to its key without
|
||||
// replacing anything, then fsyncs the directory. Identical bytes already stored
|
||||
// (an earlier attempt whose directory sync failed, or a concurrent installer)
|
||||
@@ -57,15 +64,21 @@ class FilesystemObjectStore {
|
||||
[[nodiscard]] const std::filesystem::path& directory() const noexcept {
|
||||
return directory_;
|
||||
}
|
||||
// Bytes available to this process on the volume; nullopt if unknown.
|
||||
[[nodiscard]] std::optional<std::uint64_t> available_bytes() const noexcept;
|
||||
[[nodiscard]] std::uint64_t reserve_bytes() const noexcept {
|
||||
return reserve_;
|
||||
}
|
||||
|
||||
private:
|
||||
FilesystemObjectStore(std::filesystem::path directory, int descriptor, std::size_t maximum,
|
||||
ObjectStoreFaults* faults);
|
||||
ObjectStoreFaults* faults, std::uint64_t reserve);
|
||||
[[nodiscard]] std::expected<VerifiedObject, ObjectError>
|
||||
durable_existing(const VerifiedObject& object) const;
|
||||
std::filesystem::path directory_;
|
||||
int descriptor_;
|
||||
std::size_t maximum_;
|
||||
ObjectStoreFaults* faults_;
|
||||
std::uint64_t reserve_;
|
||||
};
|
||||
} // namespace evo::persistence
|
||||
|
||||
Reference in New Issue
Block a user