Audit effective runtime privileges and bound lease policy arithmetic
Review P1: the v2 attribute-only runtime role check accepted a login that inherits pg_write_all_data, which can UPDATE lease rows directly and bypass fencing. apply.sql now audits, on every migrate run, the runtime group and all transitive members: privileged attributes, any membership outside the group (inherited or SET-able), table privileges in evolution (including PUBLIC), CREATE rights and EXECUTE beyond the five fenced functions. 0002 and its checksum are unchanged. CI reproduces the bypass and proves each configuration is rejected and rolled back. Review P2: valid(LeasePolicy) bounds every term by the duration before arithmetic; LeaseAuthority validates durations before deadline math and renewal_unknown ignores impossible (>60s) and fences non-positive requests. Extreme min/max tests reproduce the old UBSan overflow and now pass. Validated: Windows CTest 55/55; WSL Release, clang-tidy and ASan/UBSan private-PostgreSQL suites. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -32,8 +32,12 @@ table privileges. Provisioning (genesis) and the internal lease check stay owner
|
||||
- Rejections use custom SQLSTATEs EVL01..EVL05 (busy, fenced, head mismatch, not
|
||||
provisioned, exhausted) and 22023 for invalid arguments. A rejection aborts any
|
||||
enclosing transaction. Clients never send timestamps.
|
||||
- The migration requires an operator-created runtime role that is unprivileged and
|
||||
does not inherit the owner. History admission is an exact prefix: clean→v1→v2,
|
||||
- The migration requires an operator-created runtime role. Every migrate run audits
|
||||
the effective privileges of that role and all its members. It rejects privileged
|
||||
attributes, any membership outside the group (including predefined roles such as
|
||||
pg_write_all_data), table or CREATE rights, and EXECUTE beyond the five
|
||||
functions. This addresses a review finding: an attribute-only check let an
|
||||
inherited pg_write_all_data bypass fencing. History admission is an exact prefix: clean→v1→v2,
|
||||
v1→v2, or a v2 no-op. Anything else fails closed.
|
||||
|
||||
Host side: the portable `LeaseAuthority` sets its local deadline to request-send time
|
||||
@@ -84,7 +88,9 @@ the region stays unavailable for up to one lease period. The runtime role must b
|
||||
pre-created by an operator. This adds a second migration and a chain of prefix checksums.
|
||||
|
||||
## Risks
|
||||
A frozen VM/hypervisor can stop all guest clocks. Database commits stay fenced, but
|
||||
Grants made after the last migrate run are not audited until it runs again, so
|
||||
operators must rerun it after role changes. A frozen VM/hypervisor can stop all
|
||||
guest clocks. Database commits stay fenced, but
|
||||
provisional output could then be late. Gateways still need bounded-validity
|
||||
registration (ADR-0058). The bare head kernel accepts any 32-byte digest until M7.4
|
||||
binds it to verified payloads, so v2 must not host real world state. Custom
|
||||
|
||||
@@ -2081,3 +2081,25 @@ Only clean-to-v1 and exact-v1 restart are supported migration paths at this gate
|
||||
kill, real network partition (a lock wait plus client deadline stands in for
|
||||
it), VM freeze and TSan. The contention threads each own a separate
|
||||
connection; libpq is uninstrumented.
|
||||
|
||||
### M7.3 review fixes - 2026-09-28
|
||||
|
||||
- P1: an attribute-only runtime role check let a login inheriting
|
||||
`pg_write_all_data` update `evolution.region_leases` directly. The CI script
|
||||
reproduces that UPDATE first. `apply.sql` now audits effective privileges on
|
||||
every run (the v2 migration file and its checksum are unchanged). Rejected, each
|
||||
with a rollback:
|
||||
- a runtime group inheriting `pg_write_all_data` on a fresh database (no
|
||||
schema left);
|
||||
- that grant on the login, and on the group WITH INHERIT FALSE;
|
||||
- a PUBLIC table grant, EXECUTE on the internal check function, and a
|
||||
CREATEDB login.
|
||||
The restored configuration passes again.
|
||||
- P2: `valid(LeasePolicy)` now bounds each term by the duration before any
|
||||
arithmetic. `acquired` validates the duration before computing a deadline.
|
||||
`renewal_unknown` ignores durations above 60 s and fences on non-positive ones.
|
||||
New tests use `milliseconds::max()` and `min()` for every field. Built against
|
||||
the previous source under clang19 UBSan, they reproduce the reported signed
|
||||
overflow (`9223372036854775807 + 2000`).
|
||||
- Reran Windows CTest 55/55, the repository format check, and GCC14 Release,
|
||||
clang-tidy analysis and ASan/UBSan private-PostgreSQL suites: all passed.
|
||||
|
||||
@@ -60,10 +60,18 @@ fractional, negative and above-u64 values without signed truncation.
|
||||
|
||||
Use a dedicated owner/migrator login with CREATE on the database. Before
|
||||
migrating, operators create a NOLOGIN runtime group role and pass its name as
|
||||
`EVO_PG_RUNTIME_ROLE`. The migration refuses a role that is missing, is the owner,
|
||||
inherits the owner, or has superuser/CREATEROLE/CREATEDB/replication/BYPASSRLS. The
|
||||
runtime login is a separate member of that group only, with no database CREATE or
|
||||
public schema CREATE. Bootstrap revokes PUBLIC access to the schema, tables and
|
||||
`EVO_PG_RUNTIME_ROLE`. The runtime login is a separate member of that group only.
|
||||
Every migrate run, including a no-op, audits effective privileges of the group and
|
||||
every role transitively in it, and rolls back on any violation:
|
||||
- privileged attributes or owner identity;
|
||||
- membership in any other role, whether inherited or SET-able (for example
|
||||
`pg_write_all_data`);
|
||||
- any table privilege in `evolution`, including via PUBLIC;
|
||||
- CREATE on the schema, the database or `public`;
|
||||
- EXECUTE outside the five fenced functions.
|
||||
Attribute checks alone are not enough: a login with `pg_write_all_data` can update
|
||||
lease rows directly. Rerun the migrate Job after adding or changing runtime logins
|
||||
or grants; it is the audit. Grants made later without a rerun are not detected. Bootstrap revokes PUBLIC access to the schema, tables and
|
||||
default function execution. Operators own database/role provisioning; the migration
|
||||
never creates logins or passwords, and never grants superuser.
|
||||
|
||||
|
||||
@@ -30,7 +30,8 @@ struct LeaseGrant {
|
||||
std::chrono::milliseconds duration{};
|
||||
};
|
||||
|
||||
// Selected M7.3 timings. Validity: duration 1..60s (database bound); margin at
|
||||
// Selected M7.3 timings. Validity: duration 1..60s (database bound); every other
|
||||
// term positive and below the duration (checked before any arithmetic); margin at
|
||||
// least 10% of duration (clock-rate and check-to-effect allowance); two complete
|
||||
// renewal attempts fit before local fencing; I/O deadlines within adapter limits.
|
||||
struct LeasePolicy {
|
||||
@@ -76,7 +77,8 @@ class LeaseAuthority {
|
||||
LeaseClock::time_point sent,
|
||||
LeaseClock::time_point received) noexcept;
|
||||
// Ambiguous renewal: the database may have applied a shorter expiry, so keep
|
||||
// the earlier of the old deadline and the one the attempt could have set.
|
||||
// the earlier of the old deadline and the one the attempt could have set. A
|
||||
// request above 60s cannot have applied; a non-positive request fences.
|
||||
void renewal_unknown(std::chrono::milliseconds requested, LeaseClock::time_point sent) noexcept;
|
||||
void fence() noexcept;
|
||||
[[nodiscard]] LeaseState state(LeaseClock::time_point now) const noexcept;
|
||||
|
||||
@@ -29,4 +29,67 @@ SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname='evolution')
|
||||
\ir 0001_region_bootstrap.sql
|
||||
\ir 0002_region_fencing.sql
|
||||
\endif
|
||||
-- Effective-privilege audit on every run, including no-ops (ADR-0060). Attributes
|
||||
-- alone are insufficient: predefined roles such as pg_write_all_data, PUBLIC
|
||||
-- grants or SET-able memberships would bypass the fenced functions. Checks the
|
||||
-- runtime group and every role that is (transitively) a member of it.
|
||||
SELECT pg_catalog.set_config('evolution.runtime_role', :'runtime_role', true);
|
||||
DO $$
|
||||
DECLARE
|
||||
runtime oid := (SELECT r.oid FROM pg_catalog.pg_roles r
|
||||
WHERE r.rolname = pg_catalog.current_setting('evolution.runtime_role'));
|
||||
bad text;
|
||||
BEGIN
|
||||
IF runtime IS NULL THEN
|
||||
RAISE EXCEPTION 'runtime role does not exist';
|
||||
END IF;
|
||||
WITH RECURSIVE closure(oid) AS (
|
||||
SELECT runtime
|
||||
UNION
|
||||
SELECT m.member FROM pg_catalog.pg_auth_members m JOIN closure c ON m.roleid = c.oid
|
||||
), fenced(oid) AS (VALUES
|
||||
('evolution.acquire_region_lease(evolution.u64,evolution.u64,integer)'::regprocedure::oid),
|
||||
('evolution.renew_region_lease(evolution.u64,evolution.u64,uuid,evolution.u64,integer)'::regprocedure::oid),
|
||||
('evolution.release_region_lease(evolution.u64,evolution.u64,uuid,evolution.u64)'::regprocedure::oid),
|
||||
('evolution.load_region_head(evolution.u64,evolution.u64)'::regprocedure::oid),
|
||||
('evolution.advance_region_head(evolution.u64,evolution.u64,uuid,evolution.u64,evolution.u64,uuid,bytea,uuid,bytea,evolution.u64)'::regprocedure::oid)
|
||||
), violations(role_oid, reason) AS (
|
||||
SELECT r.oid, 'privileged attribute or owner identity' FROM pg_catalog.pg_roles r
|
||||
JOIN closure c ON c.oid = r.oid
|
||||
WHERE r.rolsuper OR r.rolcreaterole OR r.rolcreatedb OR r.rolreplication
|
||||
OR r.rolbypassrls OR r.rolname = current_user
|
||||
UNION ALL
|
||||
-- Any membership outside the runtime closure (inherited or SET-able).
|
||||
SELECT m.member, 'membership outside the runtime group' FROM pg_catalog.pg_auth_members m
|
||||
JOIN closure c ON c.oid = m.member
|
||||
WHERE m.roleid NOT IN (SELECT oid FROM closure)
|
||||
UNION ALL
|
||||
SELECT c.oid, 'table privilege in evolution schema' FROM closure c
|
||||
CROSS JOIN pg_catalog.pg_class t
|
||||
WHERE t.relnamespace = 'evolution'::regnamespace AND t.relkind IN ('r', 'p', 'v', 'm', 'S')
|
||||
AND pg_catalog.has_table_privilege(c.oid, t.oid,
|
||||
'SELECT,INSERT,UPDATE,DELETE,TRUNCATE,REFERENCES,TRIGGER,MAINTAIN')
|
||||
UNION ALL
|
||||
SELECT c.oid, 'CREATE on schema or database' FROM closure c
|
||||
WHERE pg_catalog.has_schema_privilege(c.oid, 'evolution', 'CREATE')
|
||||
OR pg_catalog.has_database_privilege(c.oid, pg_catalog.current_database(), 'CREATE')
|
||||
OR EXISTS (SELECT 1 FROM pg_catalog.pg_namespace n WHERE n.nspname = 'public'
|
||||
AND pg_catalog.has_schema_privilege(c.oid, n.oid, 'CREATE'))
|
||||
UNION ALL
|
||||
-- Only the five guarded functions are executable; provisioning/internal are not.
|
||||
SELECT c.oid, 'EXECUTE outside the fenced function set' FROM closure c
|
||||
CROSS JOIN pg_catalog.pg_proc p
|
||||
WHERE p.pronamespace = 'evolution'::regnamespace
|
||||
AND p.oid NOT IN (SELECT oid FROM fenced)
|
||||
AND pg_catalog.has_function_privilege(c.oid, p.oid, 'EXECUTE')
|
||||
UNION ALL
|
||||
SELECT runtime, 'missing EXECUTE on a fenced function' FROM fenced f
|
||||
WHERE NOT pg_catalog.has_function_privilege(runtime, f.oid, 'EXECUTE')
|
||||
)
|
||||
SELECT pg_catalog.string_agg(DISTINCT v.reason, '; ') INTO bad FROM violations v;
|
||||
IF bad IS NOT NULL THEN
|
||||
-- Reasons only; no role names or connection details are printed.
|
||||
RAISE EXCEPTION 'runtime role audit failed: %', bad;
|
||||
END IF;
|
||||
END $$;
|
||||
COMMIT;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
#include "evo/persistence/region_lease.hpp"
|
||||
#include <algorithm>
|
||||
#include <initializer_list>
|
||||
#if defined(__linux__)
|
||||
#include <time.h>
|
||||
#endif
|
||||
@@ -13,11 +14,13 @@ constexpr milliseconds maximum_io{30000}; // Postgres adapter deadline bound.
|
||||
} // namespace
|
||||
|
||||
bool valid(const LeasePolicy& p) noexcept {
|
||||
const milliseconds zero{};
|
||||
if (p.duration < minimum_duration || p.duration > maximum_duration ||
|
||||
p.renew_interval <= zero || p.safety_margin <= zero || p.renew_timeout <= zero ||
|
||||
p.commit_timeout <= zero || p.renew_timeout > maximum_io || p.commit_timeout > maximum_io ||
|
||||
p.safety_margin >= p.duration)
|
||||
if (p.duration < minimum_duration || p.duration > maximum_duration)
|
||||
return false;
|
||||
// Bound every term by the (<=60s) duration before any arithmetic can overflow.
|
||||
for (const auto term : {p.renew_interval, p.safety_margin, p.renew_timeout, p.commit_timeout})
|
||||
if (term <= milliseconds{} || term >= p.duration)
|
||||
return false;
|
||||
if (p.renew_timeout > maximum_io || p.commit_timeout > maximum_io)
|
||||
return false;
|
||||
const auto usable = p.duration - p.safety_margin;
|
||||
return p.safety_margin * 10 >= p.duration &&
|
||||
@@ -39,9 +42,14 @@ LeaseClock::time_point LeaseClock::now() noexcept {
|
||||
|
||||
bool LeaseAuthority::acquired(const LeaseGrant& grant, LeaseClock::time_point sent,
|
||||
LeaseClock::time_point received) noexcept {
|
||||
const auto deadline = sent + grant.duration - policy_.safety_margin;
|
||||
// The duration equals the validated policy before it enters time arithmetic.
|
||||
if (!valid(policy_) || grant.epoch == 0 || grant.epoch <= highest_epoch_ || grant.frame == 0 ||
|
||||
grant.duration != policy_.duration || received < sent || received >= deadline) {
|
||||
grant.duration != policy_.duration || received < sent) {
|
||||
fence();
|
||||
return false;
|
||||
}
|
||||
const auto deadline = sent + grant.duration - policy_.safety_margin;
|
||||
if (received >= deadline) {
|
||||
fence();
|
||||
return false;
|
||||
}
|
||||
@@ -69,7 +77,14 @@ bool LeaseAuthority::renewed(std::uint64_t epoch, milliseconds granted, LeaseClo
|
||||
void LeaseAuthority::renewal_unknown(milliseconds requested, LeaseClock::time_point sent) noexcept {
|
||||
if (!grant_ || fenced_)
|
||||
return;
|
||||
deadline_ = std::min(deadline_, sent + requested - policy_.safety_margin);
|
||||
// The database rejects durations outside 1..60s, so such a request cannot have
|
||||
// changed expiry. A non-positive request is a caller error: fence.
|
||||
if (requested <= milliseconds{}) {
|
||||
fence();
|
||||
return;
|
||||
}
|
||||
if (requested <= maximum_duration)
|
||||
deadline_ = std::min(deadline_, sent + requested - policy_.safety_margin);
|
||||
last_sent_ = std::max(last_sent_, sent);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
#include "evo/persistence/region_lease.hpp"
|
||||
|
||||
#include <array>
|
||||
#include <cstdio>
|
||||
#include <source_location>
|
||||
#include <stdexcept>
|
||||
@@ -110,6 +111,39 @@ void epochs_are_never_reused() {
|
||||
CHECK(current && current->epoch == 4);
|
||||
}
|
||||
|
||||
// Extreme values must be rejected without signed overflow (checked under UBSan).
|
||||
void extreme_durations_are_rejected_without_overflow() {
|
||||
constexpr auto most = std::chrono::milliseconds::max();
|
||||
constexpr auto least = std::chrono::milliseconds::min();
|
||||
for (const auto extreme : {most, least}) {
|
||||
for (int field = 0; field < 5; ++field) {
|
||||
LeasePolicy p;
|
||||
std::array<std::chrono::milliseconds*, 5> terms{&p.duration, &p.renew_interval,
|
||||
&p.safety_margin, &p.renew_timeout,
|
||||
&p.commit_timeout};
|
||||
*terms[static_cast<std::size_t>(field)] = extreme;
|
||||
CHECK(!valid(p));
|
||||
LeaseAuthority a{p};
|
||||
CHECK(!a.acquired(grant(1), at(0ms), at(1ms)));
|
||||
CHECK(a.state(at(1ms)) == LeaseState::absent && !a.may_begin_commit(at(1ms)));
|
||||
}
|
||||
}
|
||||
LeasePolicy interval;
|
||||
interval.renew_interval = 20000ms; // Equal to the duration: no room to renew.
|
||||
CHECK(!valid(interval));
|
||||
|
||||
LeaseAuthority a{LeasePolicy{}};
|
||||
CHECK(!a.acquired({{1}, 1, 7, most}, at(0ms), at(1ms)));
|
||||
CHECK(!a.acquired({{1}, 1, 7, least}, at(0ms), at(1ms)));
|
||||
CHECK(a.acquired(grant(2), at(0ms), at(1ms)));
|
||||
CHECK(!a.renewed(2, most, at(1000ms), at(1001ms)));
|
||||
CHECK(a.acquired(grant(3), at(2000ms), at(2001ms)));
|
||||
a.renewal_unknown(most, at(3000ms)); // Database would reject: deadline unchanged.
|
||||
CHECK(a.deadline() == at(20000ms) && a.may_originate(at(3000ms)));
|
||||
a.renewal_unknown(least, at(3000ms)); // Caller error: fence.
|
||||
CHECK(a.state(at(3000ms)) == LeaseState::fenced);
|
||||
}
|
||||
|
||||
void clock_is_monotonic() {
|
||||
const auto first = LeaseClock::now();
|
||||
const auto second = LeaseClock::now();
|
||||
@@ -123,6 +157,7 @@ int main() {
|
||||
deadlines_follow_send_time();
|
||||
renewals();
|
||||
epochs_are_never_reused();
|
||||
extreme_durations_are_rejected_without_overflow();
|
||||
clock_is_monotonic();
|
||||
std::puts("region lease contracts passed");
|
||||
return 0;
|
||||
|
||||
@@ -44,6 +44,35 @@ PGDATABASE=upgrade_path psql -X -v ON_ERROR_STOP=1 -1 -v "migration_sha=$v1_sha"
|
||||
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',') FROM evolution.schema_migrations")" = 1
|
||||
PGDATABASE=upgrade_path bash "$root/tools/migrate-postgres.sh"
|
||||
test "$(PGDATABASE=upgrade_path psql -XAtc "SELECT string_agg(version::text, ',' ORDER BY version) FROM evolution.schema_migrations")" = 1,2
|
||||
# Inherited privileges bypass fencing, so every migrate run audits effective rights.
|
||||
psql -X -v ON_ERROR_STOP=1 -c 'CREATE ROLE evo_bad_runtime NOLOGIN IN ROLE pg_write_all_data'
|
||||
if (export PGDATABASE=migration_rollback EVO_PG_RUNTIME_ROLE=evo_bad_runtime; \
|
||||
bash "$root/tools/migrate-postgres.sh"); then
|
||||
echo 'Expected inherited-privilege runtime rejection' >&2; exit 1
|
||||
fi
|
||||
test "$(PGDATABASE=migration_rollback psql -XAtc "SELECT count(*) FROM pg_namespace WHERE nspname='evolution'")" = 0
|
||||
expect_audit_failure() {
|
||||
psql -X -v ON_ERROR_STOP=1 -c "$1"
|
||||
if bash "$root/tools/migrate-postgres.sh"; then
|
||||
echo "Expected runtime audit failure after: $1" >&2; exit 1
|
||||
fi
|
||||
psql -X -v ON_ERROR_STOP=1 -c "$2"
|
||||
}
|
||||
# The reviewed scenario: the login inherits pg_write_all_data and can bypass the guard.
|
||||
psql -X -v ON_ERROR_STOP=1 -c 'GRANT pg_write_all_data TO evo_runtime_login'
|
||||
PGUSER=evo_runtime_login psql -X -v ON_ERROR_STOP=1 \
|
||||
-c 'UPDATE evolution.region_leases SET epoch = 99 WHERE false' > /dev/null
|
||||
psql -X -v ON_ERROR_STOP=1 -c 'REVOKE pg_write_all_data FROM evo_runtime_login'
|
||||
expect_audit_failure 'GRANT pg_write_all_data TO evo_runtime_login' \
|
||||
'REVOKE pg_write_all_data FROM evo_runtime_login'
|
||||
expect_audit_failure 'GRANT pg_write_all_data TO evo_runtime_test WITH INHERIT FALSE' \
|
||||
'REVOKE pg_write_all_data FROM evo_runtime_test'
|
||||
expect_audit_failure 'GRANT UPDATE ON evolution.region_heads TO PUBLIC' \
|
||||
'REVOKE UPDATE ON evolution.region_heads FROM PUBLIC'
|
||||
expect_audit_failure 'GRANT EXECUTE ON FUNCTION evolution.check_region_lease(evolution.u64, evolution.u64, uuid, evolution.u64) TO evo_runtime_test' \
|
||||
'REVOKE EXECUTE ON FUNCTION evolution.check_region_lease(evolution.u64, evolution.u64, uuid, evolution.u64) FROM evo_runtime_test'
|
||||
expect_audit_failure 'ALTER ROLE evo_runtime_login CREATEDB' 'ALTER ROLE evo_runtime_login NOCREATEDB'
|
||||
bash "$root/tools/migrate-postgres.sh" # Restored configuration passes the audit again.
|
||||
# Unknown future versions and checksum drift must not be repaired silently.
|
||||
psql -X -v ON_ERROR_STOP=1 -c 'UPDATE evolution.schema_migrations SET version=3 WHERE version=2'
|
||||
if bash "$root/tools/migrate-postgres.sh"; then exit 1; fi
|
||||
|
||||
Reference in New Issue
Block a user