Check rendered kustomize output in the deploy policy (M9.8f review)

The policy check now renders every overlay with pinned, checksum-verified
kustomize v5.8.1 (ADR-0088) and validates effective namespaces, session/input/
listener environment and arguments, ports, Services, quotas, policies, Ingresses
and Secrets, so inline/JSON patches and namespace changes cannot bypass it.
Route enabling is gated on an ADR-0087 amendment with tested server windows, route
addresses must be RFC 1918 or explicitly approved, and the Local health-check node
port is documented with its enabling evidence. Self-test grows to 33 cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
dingo
2026-10-02 18:02:57 -04:00
co-authored by Claude Opus 5.5
parent 9f622cf6b3
commit 8a1e5ef53d
12 changed files with 386 additions and 32 deletions
+4 -1
View File
@@ -86,9 +86,12 @@ RUN cmake --preset analysis -B build/ecs-analysis -DCMAKE_CXX_COMPILER=clang++-1
FROM ${UBUNTU_IMAGE} AS ci-deploy FROM ${UBUNTU_IMAGE} AS ci-deploy
ENV DEBIAN_FRONTEND=noninteractive ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-yaml \ && apt-get install -y --no-install-recommends python3 python3-yaml curl ca-certificates \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /source WORKDIR /source
# Pinned, checksum-verified kustomize for the rendered-output checks (ADR-0088).
COPY tools/ci/prepare-kustomize.sh tools/ci/
RUN bash tools/ci/prepare-kustomize.sh /usr/local/bin
COPY deploy/ deploy/ COPY deploy/ deploy/
COPY tools/ci/deploy-policy-check.py tools/ci/deploy-policy-selftest.py tools/ci/ COPY tools/ci/deploy-policy-check.py tools/ci/deploy-policy-selftest.py tools/ci/
COPY tools/ops/ tools/ops/ COPY tools/ops/ tools/ops/
@@ -3,7 +3,10 @@
# name void.aeternalisgames.com resolves to (never the control-plane address). # name void.aeternalisgames.com resolves to (never the control-plane address).
# - externalTrafficPolicy Local keeps client source addresses, so the NetworkPolicy's # - externalTrafficPolicy Local keeps client source addresses, so the NetworkPolicy's
# ipBlock (same CIDRs) is enforceable; new connections reach only a ready pod. # ipBlock (same CIDRs) is enforceable; new connections reach only a ready pod.
# - allocateLoadBalancerNodePorts false: no node ports (the quota forbids them). # - allocateLoadBalancerNodePorts false: no node port for the session data. Local
# still makes Kubernetes allocate spec.healthCheckNodePort, an HTTP endpoint kube-proxy
# serves on every node; it carries no session traffic and must stay closed at the edge
# firewall. Its allocation and quota accounting are verified before enabling (ADR-0087).
# - loadBalancerSourceRanges duplicates the CIDR at the Service; the edge firewall is # - loadBalancerSourceRanges duplicates the CIDR at the Service; the edge firewall is
# the third layer. TLS 1.3, the dev CA and single-use grants remain mandatory. # the third layer. TLS 1.3, the dev CA and single-use grants remain mandatory.
apiVersion: v1 apiVersion: v1
+37 -8
View File
@@ -2,7 +2,8 @@
## Status ## Status
Accepted for M9.8f as design (2026-10-02), evolution-dev only. The route is committed Accepted for M9.8f as design (2026-10-02), evolution-dev only. The route is committed
**disabled**; nothing was applied to the cluster. Enabling it is blocked by policy until **disabled**; nothing was applied to the cluster. The M9.8f gate stays open until the
route is enabled and connectivity is recorded. Enabling it is blocked by policy until
the inputs and server-side window enforcement below exist. Narrowly supersedes the inputs and server-side window enforcement below exist. Narrowly supersedes
ADR-0083 (loopback-only listener, never a Service) and ADR-0069 (no LoadBalancer ADR-0083 (loopback-only listener, never a Service) and ADR-0069 (no LoadBalancer
Services) for evolution-dev, and only once enabled. Services) for evolution-dev, and only once enabled.
@@ -23,8 +24,13 @@ fronts one LoadBalancer Service `evo-region-session` in evolution-dev, which for
the region pod's port 7400. TLS 1.3 terminates in the region pod; its dev-CA server the region pod's port 7400. TLS 1.3 terminates in the region pod; its dev-CA server
certificate carries `void.aeternalisgames.com`, which the client verifies. No HTTP certificate carries `void.aeternalisgames.com`, which the client verifies. No HTTP
Ingress, no TLS termination at nginx, no PROXY-protocol prefix. The Service uses Ingress, no TLS termination at nginx, no PROXY-protocol prefix. The Service uses
`externalTrafficPolicy: Local` (client addresses preserved), no node ports, a pinned `externalTrafficPolicy: Local` (client addresses preserved), no node port for the
`metallb.io/loadBalancerIPs` annotation and `loadBalancerSourceRanges`. New connections session data (`allocateLoadBalancerNodePorts: false`), a pinned
`metallb.io/loadBalancerIPs` annotation and `loadBalancerSourceRanges`.
**Health-check exception:** with `Local`, Kubernetes still allocates
`spec.healthCheckNodePort`, an HTTP endpoint kube-proxy serves on every node reporting
local endpoint health. It carries no session traffic, must stay closed at the edge
firewall, and `services.nodeports: "0"` is not assumed to prevent or account for it. New connections
reach only a ready pod; after a replacement, clients reconnect with a fresh grant and reach only a ready pod; after a replacement, clients reconnect with a fresh grant and
baseline (ADR-0083). baseline (ADR-0083).
@@ -54,18 +60,37 @@ Expiry is enforced **inside evo-server**, not by a delayed cleanup job:
closed Git configuration. Grant expiry alone does not stop the generator. closed Git configuration. Grant expiry alone does not stop the generator.
## Enabling (all required; `tools/ci/deploy-policy-check.py` refuses otherwise) ## Enabling (all required; `tools/ci/deploy-policy-check.py` refuses otherwise)
1. The VIP (private, from the MetalLB pool) and permitted client CIDR(s) (private, no 0. `ROUTE_ENABLING_PERMITTED` in the policy check is `False` and is changed only together
wider than /16) replace the REPLACE markers; Service ranges equal the policy's. with an amendment here that records the implemented window argument/configuration
contract and its runtime expiry tests (expiry during an active session, restart after
expiry, delayed application, extend/close races, clock movement, synthetic inputs
stopping while pending commits drain). A flag string in a manifest is not evidence.
1. The VIP (from the MetalLB pool) and permitted client CIDR(s) (no wider than /16)
replace the REPLACE markers; Service ranges equal the policy's. Addresses must be
RFC 1918 or in a range approved here and listed in `APPROVED_INTERNAL` (none yet;
the recorded cluster LAN addresses, 172.10.1.x, are not RFC 1918).
2. `deploy/kubernetes/platform/dev` raises `services.loadbalancers` to exactly 1; 2. `deploy/kubernetes/platform/dev` raises `services.loadbalancers` to exactly 1;
node ports stay 0; stage and World0 stay 0. node ports stay 0; stage and World0 stay 0.
3. evo-server implements the session window (`--session-window`) as above, with tests 3. evo-server implements the session window (`--session-window`) as above, with tests
and an amendment to this ADR, and the base deployment passes it. and an amendment to this ADR, and the base deployment passes it.
4. Only `region/dev` includes the component. Then: internal DNS for void to the VIP, the 4. Only `region/dev` includes the component. Then: internal DNS for void to the VIP, the
edge firewall rule, and a server-side dry-run (quota accounting of the Local health edge firewall rule, a new dev server certificate for void.aeternalisgames.com, and live
check port is to be confirmed there), a new dev server certificate for connectivity recorded separately in VALIDATION.md.
void.aeternalisgames.com, and live connectivity recorded separately in VALIDATION.md. 5. Health-check port: a server-side dry-run and the applied object show the allocated
`healthCheckNodePort` and how the quota accounts for it; the firewall handling is
recorded. `services.nodeports` is not raised without that evidence.
6. The rendered-output checks (ADR-0088) pass for every overlay with the route included.
## Open points for the maintainer ## Open points for the maintainer
Review recommendations (M9.8f review, 2026-10-02), pending the maintainer's decision:
keep manual Argo sync for the first delivery and report a workflow-created window as
pending until applied, with an absolute expiry that delayed sync never shifts (Close
also needs propagation; the server deadline is the upper bound); keep the existing void
HTTP endpoint unless explicitly retired, possibly choosing a dedicated session hostname
instead (DNS cannot select by TCP port), before issuing the certificate; confirm the
actual MetalLB pool and client addresses; set the maximum window length and clock
handling before implementing windows.
- **Argo delivery.** The aevolution-dev Application has no automated sync, and the - **Argo delivery.** The aevolution-dev Application has no automated sync, and the
policy forbids auto-sync outside the platform app (ADR-0069). A workflow commit alone policy forbids auto-sync outside the platform app (ADR-0069). A workflow commit alone
will not deploy; either a manual sync per window or a scoped dev exception must be will not deploy; either a manual sync per window or a scoped dev exception must be
@@ -83,6 +108,10 @@ Expiry is enforced **inside evo-server**, not by a delayed cleanup job:
- NodePort: opens a port on every node; forbidden by ADR-0069. - NodePort: opens a port on every node; forbidden by ADR-0069.
- Always-on sessions in dev: continuous synthetic history; rejected for now. - Always-on sessions in dev: continuous synthetic history; rejected for now.
Policy evaluation (ADR-0088): the check renders every overlay with the pinned kustomize
and validates the effective objects, so inline or JSON patches and namespace changes
cannot enable listeners, sessions, inputs or session ports outside the route.
## Consequences ## Consequences
One reviewed exception to "no LoadBalancer" for evolution-dev. The policy check and its One reviewed exception to "no LoadBalancer" for evolution-dev. The policy check and its
fail-closed self-test (`tools/ci/deploy-policy-selftest.py`) encode the route's shape and fail-closed self-test (`tools/ci/deploy-policy-selftest.py`) encode the route's shape and
@@ -0,0 +1,41 @@
# ADR-0088: Rendered deployment-policy checks with pinned kustomize
## Status
Accepted 2026-10-02 (M9.8f review correction). Extends ADR-0069's offline policy check.
## Context
`tools/ci/deploy-policy-check.py` validated manifest files one by one. Kustomize
inline strategic-merge or JSON patches, components and namespace transformations
change the effective objects without appearing as checkable files; the M9.8f review
reproduced inline patches enabling the session listener, sessions, synthetic inputs
and a session port while the check passed.
## Decision
- The check renders every non-component kustomization under `deploy/` with a pinned
kustomize and validates the effective output: overlay namespaces; evo-region's
session/input/listener environment, the single listener argument and container ports;
Service types and the session port; route Service shape; NetworkPolicies admitting the
session port; quotas; Ingresses; and the absence of Secrets. Source-file checks stay.
- Kustomizations may use only the keys in use today (resources, components, patches,
images, labels, common labels/annotations, namespace, ConfigMap generators);
generators/transformers plugins, Helm inflation and Secret generators are refused.
- Dependency: kustomize **v5.8.1** (kubernetes-sigs, Apache-2.0), official Linux amd64
release archive, SHA-256
029a7f0f4e1932c52a0476cf02a0fd855c0bb85694b82c338fc648dcb53a819d, matching the
release's checksums.txt. Maintenance evidence (GitHub releases, 2026-10-02): v5.8.0
2025-11-09, v5.8.1 2026-02-09, v5.8.2 2026-09-30. v5.8.1 is pinned rather than the
two-day-old v5.8.2; updates change the version and checksum together.
- Acquisition is explicit: `tools/ci/prepare-kustomize.sh [dir]` downloads, verifies and
installs it (default `.cache/tools/kustomize-v5.8.1`); the CI deploy image runs it.
The check uses `$KUSTOMIZE`, that cache path or `kustomize` on PATH, and fails unless
the binary reports exactly the pinned version. No build step downloads it implicitly.
## Alternatives considered
- `kubectl kustomize`: its embedded kustomize version follows kubectl releases.
- Parsing patches in Python: re-implements kustomize semantics incompletely.
- kubeconform/schema validation: complements but does not evaluate policy.
## Consequences
Running the policy check locally needs the pinned binary (Linux x86_64 is pinned).
Rendering adds seconds to the deploy suite. The self-test covers inline and JSON patches,
namespace changes and unsupported features.
+1
View File
@@ -97,6 +97,7 @@ design. No unadopted candidate is a link dependency of the proof.
- [ADR-0083: Dev region sessions](0083-dev-region-sessions.md) - session server in evo-server, durable no-effect intentions on synthetic carriers, empty organism presentation, reconnect with fresh baselines, loopback-only drill-time dev deployment. - [ADR-0083: Dev region sessions](0083-dev-region-sessions.md) - session server in evo-server, durable no-effect intentions on synthetic carriers, empty organism presentation, reconnect with fresh baselines, loopback-only drill-time dev deployment.
- [ADR-0084: Engineering UI toolkit](0084-engineering-ui-toolkit.md) - Dear ImGui 1.92.9b pinned (M9.8a) and integrated as the engineering client overlay with toolkit-first input routing (M9.8c). - [ADR-0084: Engineering UI toolkit](0084-engineering-ui-toolkit.md) - Dear ImGui 1.92.9b pinned (M9.8a) and integrated as the engineering client overlay with toolkit-first input routing (M9.8c).
- [ADR-0088: Rendered deployment policy](0088-rendered-deployment-policy.md) - the deploy policy check renders every overlay with pinned kustomize v5.8.1 (checksum-verified, explicit preparation) and validates effective objects; unsupported kustomize features refused.
- [ADR-0087: Dev session route](0087-dev-session-route.md) - evolution-dev only: void.aeternalisgames.com:7400 via a dedicated internal MetalLB VIP to the region pod, TLS end to end, CIDR-limited, operator-opened windows enforced by evo-server; committed disabled with fail-closed policy tests. - [ADR-0087: Dev session route](0087-dev-session-route.md) - evolution-dev only: void.aeternalisgames.com:7400 via a dedicated internal MetalLB VIP to the region pod, TLS end to end, CIDR-limited, operator-opened windows enforced by evo-server; committed disabled with fail-closed policy tests.
- [ADR-0086: Windows client package](0086-windows-client-package.md) - static-runtime Windows x64 ZIP with explicit allowlist, licence texts, build identity, SHA256 manifest, credential guards and deterministic layout; local packaging until a Windows runner exists. - [ADR-0086: Windows client package](0086-windows-client-package.md) - static-runtime Windows x64 ZIP with explicit allowlist, licence texts, build identity, SHA256 manifest, credential guards and deterministic layout; local packaging until a Windows runner exists.
- [ADR-0085: Rebindable engineering controls](0085-rebindable-engineering-controls.md) - SDL-free action/HID-key bindings, conflict validation, versioned all-or-nothing controls file, binding capture before toolkit/world. - [ADR-0085: Rebindable engineering controls](0085-rebindable-engineering-controls.md) - SDL-free action/HID-key bindings, conflict validation, versioned all-or-nothing controls file, binding capture before toolkit/world.
+4 -3
View File
@@ -4,9 +4,10 @@ Status: M9.8a probe, M9.8b owned state/capture/event bounds, M9.8c launcher,
legend, labelled inspector and click selection, and M9.8d rebindable controls legend, labelled inspector and click selection, and M9.8d rebindable controls
complete, 2026-10-02. M9.8e local Windows ZIP complete (ADR-0086); its Gitea complete, 2026-10-02. M9.8e local Windows ZIP complete (ADR-0086); its Gitea
Actions publication waits for a Windows runner and its clean-VM run for M9.8g. Actions publication waits for a Windows runner and its clean-VM run for M9.8g.
M9.8f route design, disabled manifests and fail-closed policy tests complete M9.8f design, disabled manifests and rendered fail-closed policy checks delivered
(ADR-0087); enabling waits for the VIP, client CIDR, DNS, dev quota and server-enforced (ADR-0087, ADR-0088); the M9.8f gate stays open until the route is enabled (VIP, client
session windows. M9.8g remains. CIDR, DNS, dev quota, health-check port evidence and server-enforced session windows)
and connectivity is recorded. M9.8g remains.
No application UI or deployment completion is claimed. See ADR-0084 for the toolkit boundary. No application UI or deployment completion is claimed. See ADR-0084 for the toolkit boundary.
## Confirmed scope ## Confirmed scope
+32
View File
@@ -3930,3 +3930,35 @@ were run directly), any cluster/server dry-run, MetalLB allocation, DNS, firewal
connectivity. Server-enforced session windows and the Gitea window workflow are not connectivity. Server-enforced session windows and the Gitea window workflow are not
implemented; Argo delivery for windows and the existing void HTTP Ingress are open implemented; Argo delivery for windows and the existing void HTTP Ingress are open
maintainer decisions recorded in ADR-0087. maintainer decisions recorded in ADR-0087.
## 2026-10-02 — M9.8f review corrections (.cache/reviews/M9.8f-9f622cf-review.md)
P1 (rendered configuration): the policy check now renders all 17 non-component
kustomizations with pinned kustomize v5.8.1 (ADR-0088; official archive, SHA-256
029a7f0f…a819d checked against the release checksums) and validates the effective
objects: overlay namespaces, evo-region session/input/listener environment, a single
listener argument, container ports, Service types and session port, route shape, session
NetworkPolicies, quotas, Ingresses and no Secrets. Unsupported kustomize features
(plugin generators/transformers, Helm, Secret generators) are refused. Route enabling is
gated by `ROUTE_ENABLING_PERMITTED = False`, changed only with an ADR-0087 amendment
recording the implemented window contract and runtime expiry tests; the earlier
`--session-window` string test is gone. Route addresses must be RFC 1918 or listed in
`APPROVED_INTERNAL` (empty; 172.10.1.x is not RFC 1918).
P2 (health-check port): manifest comment and ADR-0087 now state that the Local policy
allocates `healthCheckNodePort`; allocation, quota accounting and firewall handling are
enabling preconditions. Review recommendations (manual Argo sync, preserving the void
endpoint, address confirmation, window limits and tests) are recorded in ADR-0087 as
pending maintainer decisions. The M9.8f gate stays open.
Executed (WSL Ubuntu, Python 3, PyYAML 6.0.1, kustomize v5.8.1 via the new prepare
script, idempotent on rerun): policy check 70 source objects / 126 rendered objects in
17 overlays, 0 failures; self-test 33/33 refused (20 earlier cases plus non-RFC 1918 VIP,
the reviewer's combined inline patches, inline listener/session/input patches, JSON
patches adding a session port, a LoadBalancer, a container port and a second listener
argument, namespace transformation, inline stage quota, Secret generator, Helm inflation);
the reviewer's m98f-policy-repro.py now exits 1 with the listener, session/input and
ops-port failures in both region/dev and dev; the check fails without kustomize and with
kustomize v5.4.3; ops-lib and ci-path fixtures pass.
Not run: the ci-deploy Docker stage (no Docker here; its commands were run directly,
including prepare-kustomize.sh), any cluster dry-run or live action.
+3 -2
View File
@@ -89,5 +89,6 @@ ADR-0087 designs `void.aeternalisgames.com:7400` through a dedicated internal Me
VIP to the region pod (TLS end to end, development/test CIDR only), opened only in VIP to the region pod (TLS end to end, development/test CIDR only), opened only in
operator windows that evo-server itself expires. The manifests are in operator windows that evo-server itself expires. The manifests are in
`deploy/kubernetes/region/dev-session-route` and are not included anywhere; the policy `deploy/kubernetes/region/dev-session-route` and are not included anywhere; the policy
check refuses to enable them until the VIP, client CIDR, dev quota and the server-side check refuses to enable them until the VIP, client CIDR, dev quota and server-enforced
`--session-window` exist. Until then use the port-forward procedures above. session windows exist (ADR-0087 enabling list). Until then use the port-forward
procedures above.
+1 -1
View File
@@ -46,7 +46,7 @@ case "$suite" in
pattern="$common|$native|tests/|$postgres_tools|third_party/mbedtls/|tools/prepare-mbedtls\.cmake|tools/mbedtls-pin\.cmake|deploy/docker/Dockerfile" pattern="$common|$native|tests/|$postgres_tools|third_party/mbedtls/|tools/prepare-mbedtls\.cmake|tools/mbedtls-pin\.cmake|deploy/docker/Dockerfile"
;; ;;
deploy) deploy)
pattern="$common|deploy/|tools/ops/|tools/backup/|tools/ci/deploy-policy-(check|selftest)\.py|tools/ci/monitoring-check\.sh|tools/ci/ops-lib-check\.sh" pattern="$common|deploy/|tools/ops/|tools/backup/|tools/ci/deploy-policy-(check|selftest)\.py|tools/ci/prepare-kustomize\.sh|tools/ci/monitoring-check\.sh|tools/ci/ops-lib-check\.sh"
;; ;;
server-image) server-image)
pattern="$common|$native|migrations/|tools/prepare-postgres\.sh|tools/ci/server-image-smoke\.sh|deploy/docker/Dockerfile" pattern="$common|$native|migrations/|tools/prepare-postgres\.sh|tools/ci/server-image-smoke\.sh|deploy/docker/Dockerfile"
+146 -9
View File
@@ -28,10 +28,31 @@ environment-isolation or World0-protection invariant is broken:
- the dev session route (ADR-0087): loopback listener everywhere except the - the dev session route (ADR-0087): loopback listener everywhere except the
evolution-dev route component; that component is the only LoadBalancer and the evolution-dev route component; that component is the only LoadBalancer and the
only Service on the session port, narrowly shaped, and it cannot be enabled while only Service on the session port, narrowly shaped, and it cannot be enabled while
markers are unresolved or before server-enforced session windows exist. markers are unresolved or before server-enforced session windows exist;
- rendered output (ADR-0088): every overlay is built with the pinned kustomize and
the effective objects are checked (namespaces, listener/session/input environment,
arguments, ports, Services, quotas, policies, Ingresses, Secrets), so inline or
JSON patches and namespace transformations cannot bypass the source checks.
""" """
import ipaddress import ipaddress
import re import re
# Enabling the dev session route needs ADR-0087 to be amended with the implemented
# session-window argument contract and runtime expiry tests (expiry during a session,
# restart after expiry, delayed application, extend/close races, clock movement,
# synthetic inputs stopping while commits drain). Until then this stays False; a
# string such as --session-window in a manifest is not proof of enforcement.
ROUTE_ENABLING_PERMITTED = False
# Addresses the route may use: RFC 1918 plus internal ranges the maintainer has
# explicitly approved in ADR-0087 (none yet; the cluster LAN 172.10.0.0/16 is not
# RFC 1918 and needs that approval before use).
APPROVED_INTERNAL = []
INTERNAL = [ipaddress.ip_network(n) for n in ('10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16')] + \
[ipaddress.ip_network(n) for n in APPROVED_INTERNAL]
def internal(network):
return any(network.version == r.version and network.subnet_of(r) for r in INTERNAL)
import os import os
import shutil import shutil
import subprocess import subprocess
@@ -315,7 +336,9 @@ for path, doc in objects:
check(len(route_ranges) >= 1, f'{path}: loadBalancerSourceRanges must name the permitted CIDRs') check(len(route_ranges) >= 1, f'{path}: loadBalancerSourceRanges must name the permitted CIDRs')
if not vip.startswith('REPLACE'): if not vip.startswith('REPLACE'):
try: try:
check(ipaddress.ip_address(vip).is_private, f'{path}: the VIP must be an internal address') address = ipaddress.ip_address(vip)
check(internal(ipaddress.ip_network(f'{address}/{address.max_prefixlen}')),
f'{path}: the VIP must be an internal address (RFC 1918 or approved in ADR-0087)')
except ValueError: except ValueError:
check(False, f'{path}: the VIP must be one IP address') check(False, f'{path}: the VIP must be one IP address')
for cidr in route_ranges: for cidr in route_ranges:
@@ -323,7 +346,7 @@ for path, doc in objects:
continue continue
try: try:
network = ipaddress.ip_network(cidr) network = ipaddress.ip_network(cidr)
check(network.is_private and network.prefixlen >= (16 if network.version == 4 else 48), check(internal(network) and network.prefixlen >= (16 if network.version == 4 else 48),
f'{path}: {cidr} must be a private range no wider than /16 (IPv6 /48)') f'{path}: {cidr} must be a private range no wider than /16 (IPv6 /48)')
except ValueError: except ValueError:
check(False, f'{path}: {cidr} is not a CIDR') check(False, f'{path}: {cidr} is not a CIDR')
@@ -354,11 +377,9 @@ if route_enabled:
for name in os.listdir(os.path.join(ROOT, ROUTE)): for name in os.listdir(os.path.join(ROOT, ROUTE)):
with open(os.path.join(ROOT, ROUTE, name), encoding='utf-8') as handle: with open(os.path.join(ROOT, ROUTE, name), encoding='utf-8') as handle:
check('REPLACE' not in handle.read(), f'{ROUTE}{name}: unresolved REPLACE marker with the route enabled') check('REPLACE' not in handle.read(), f'{ROUTE}{name}: unresolved REPLACE marker with the route enabled')
base_script = ' '.join(' '.join(c.get('args', [])) for p, d in objects check(ROUTE_ENABLING_PERMITTED,
if p == 'deploy/kubernetes/region/base/deployment.yaml' and d['kind'] == 'Deployment' 'the dev session route cannot be enabled until ADR-0087 records the implemented, tested '
for c in d['spec']['template']['spec']['containers']) 'server-enforced session windows')
check('--session-window' in base_script,
'the dev session route needs server-enforced session windows (--session-window, ADR-0087)')
dev_quota = [d.get('spec', {}).get('hard', {}).get('services.loadbalancers') for p, d in objects dev_quota = [d.get('spec', {}).get('hard', {}).get('services.loadbalancers') for p, d in objects
if d['kind'] == 'ResourceQuota' and p.startswith(f'{platform}/dev/')] if d['kind'] == 'ResourceQuota' and p.startswith(f'{platform}/dev/')]
check(dev_quota == ['1'], 'the dev quota must allow exactly one LoadBalancer for the route') check(dev_quota == ['1'], 'the dev quota must allow exactly one LoadBalancer for the route')
@@ -541,7 +562,123 @@ for tool, args, component in generators:
f'{tool}: component {component} must be admitted by the database egress policy') f'{tool}: component {component} must be admitted by the database egress policy')
check(pod['metadata'].get('namespace') == 'evolution-dev', f'{tool}: renders into its environment') check(pod['metadata'].get('namespace') == 'evolution-dev', f'{tool}: renders into its environment')
# ADR-0088: the effective configuration is what kustomize renders. Every overlay is
# built with the pinned binary and its output checked, independent of how patches,
# components or namespace transformations produced it.
def kustomize_binary():
with open(os.path.join(ROOT, 'tools', 'ci', 'prepare-kustomize.sh'), encoding='utf-8') as f:
version = re.search(r'^version=(v[0-9.]+)$', f.read(), re.M).group(1)
candidates = [os.environ.get('KUSTOMIZE', ''),
os.path.join(ROOT, '.cache', 'tools', f'kustomize-{version}', 'kustomize'),
shutil.which('kustomize') or '']
for candidate in candidates:
if candidate and os.path.isfile(candidate):
reported = subprocess.run([candidate, 'version'], capture_output=True, text=True).stdout.strip()
if reported == version:
return candidate
return None
SUPPORTED_KEYS = {'apiVersion', 'kind', 'namespace', 'resources', 'components', 'patches', 'images',
'labels', 'commonAnnotations', 'commonLabels', 'configMapGenerator', 'generatorOptions'}
for path, ds in manifests.items():
if path.endswith('kustomization.yaml') and ds:
unsupported = set(ds[0]) - SUPPORTED_KEYS
check(not unsupported, f'{path}: unsupported kustomize features {sorted(unsupported)} (ADR-0088)')
EXPECTED_NAMESPACE = {'deploy/kubernetes/dev': 'evolution-dev', 'deploy/kubernetes/stage': 'evolution-stage',
'deploy/kubernetes/monitoring': 'aevolution-monitoring'}
for env in ('dev', 'stage', 'world0'):
for component in ('platform', 'region', 'stores'):
EXPECTED_NAMESPACE[f'deploy/kubernetes/{component}/{env}'] = f'evolution-{env}'
EXPECTED_NAMESPACE['deploy/kubernetes/platform/monitoring'] = 'aevolution-monitoring'
CLUSTER_KINDS = {'Namespace', 'StorageClass', 'ValidatingAdmissionPolicy', 'ValidatingAdmissionPolicyBinding',
'ClusterRole', 'ClusterRoleBinding', 'PersistentVolume'}
binary = kustomize_binary()
check(binary is not None, 'the pinned kustomize is required: run tools/ci/prepare-kustomize.sh (ADR-0088)')
renders = {}
if binary:
for path, ds in sorted(manifests.items()):
if not path.endswith('kustomization.yaml') or not ds or ds[0].get('kind') == 'Component':
continue
overlay = os.path.dirname(path)
result = subprocess.run([binary, 'build', os.path.join(ROOT, overlay)], capture_output=True, text=True)
check(result.returncode == 0, f'{overlay}: kustomize build failed: {result.stderr.strip()[:300]}')
if result.returncode == 0:
renders[overlay] = [d for d in yaml.safe_load_all(result.stdout) if isinstance(d, dict)]
for overlay, docs in renders.items():
expected = EXPECTED_NAMESPACE.get(overlay)
route_services = [d for d in docs if d['kind'] == 'Service' and d['metadata']['name'] == 'evo-region-session']
route_active = bool(route_services)
for doc in docs:
kind, name = doc['kind'], doc['metadata'].get('name', '')
namespace = doc['metadata'].get('namespace')
where = f'{overlay} (rendered) {kind}/{name}'
if expected and kind not in CLUSTER_KINDS:
check(namespace == expected, f'{where}: renders into {namespace}, expected {expected}')
check(kind != 'Secret', f'{where}: rendered Secrets never come from Git')
if kind == 'Deployment' and name == 'evo-region':
pod = doc['spec']['template']['spec']
for container in pod.get('containers', []) + pod.get('initContainers', []):
env = {v.get('name'): v.get('value') for v in container.get('env', [])}
text = ' '.join(container.get('command', []) + container.get('args', []))
if container.get('name') == 'evo-server':
check(env.get('REGION_SESSION') == '' and env.get('REGION_INPUTS') == '',
f'{where}: sessions and synthetic inputs stay off in Git')
listen = '0.0.0.0:7400' if route_active else '127.0.0.1:7400'
check(env.get('REGION_SESSION_LISTEN') == listen,
f'{where}: REGION_SESSION_LISTEN must be {listen} (route {"on" if route_active else "off"})')
check(text.count('--session-listen') == 1 and '--session-listen "$REGION_SESSION_LISTEN"' in text,
f'{where}: one session listener, addressed only by REGION_SESSION_LISTEN')
else:
check('--session-listen' not in text and '--synthetic-inputs' not in text,
f'{where}: only evo-server may listen for sessions')
ports = sorted((p.get('name'), p.get('containerPort')) for p in container.get('ports', []))
allowed = [('ops', 9100), ('session', 7400)] if route_active else [('ops', 9100)]
if container.get('name') == 'evo-server':
check(ports == sorted(allowed), f'{where}: container ports must be {allowed}')
if kind == 'Service':
spec = doc['spec']
session = any(p.get('port') == 7400 or p.get('targetPort') in (7400, 'session')
for p in spec.get('ports', []))
if name == 'evo-region-session':
check(namespace == 'evolution-dev' and spec.get('type') == 'LoadBalancer'
and spec.get('externalTrafficPolicy') == 'Local'
and spec.get('allocateLoadBalancerNodePorts') is False
and spec.get('ports') == [{'name': 'session', 'port': 7400, 'targetPort': 'session',
'protocol': 'TCP'}],
f'{where}: the route keeps its reviewed shape in evolution-dev')
else:
check(spec.get('type', 'ClusterIP') == 'ClusterIP' and not session,
f'{where}: only the route may be a LoadBalancer or expose the session port')
if kind == 'Ingress':
check('7400' not in yaml.safe_dump(doc) and 'session' not in yaml.safe_dump(doc),
f'{where}: the dev session is never an Ingress')
if kind == 'NetworkPolicy':
for rule in doc['spec'].get('ingress', []):
if any(p.get('port') == 7400 for p in rule.get('ports', [])):
ranges = route_services[0]['spec'].get('loadBalancerSourceRanges', []) if route_services else []
blocks = [peer.get('ipBlock', {}) for peer in rule.get('from', [])]
check(name == 'evo-region-session-ingress' and route_active and blocks
and all(set(b) == {'cidr'} for b in blocks) and len(blocks) == len(rule.get('from', []))
and sorted(b['cidr'] for b in blocks) == sorted(ranges),
f'{where}: only the route policy admits TCP 7400, from exactly its source ranges')
if kind == 'ResourceQuota':
hard = doc['spec'].get('hard', {})
check(hard.get('services.nodeports') == '0', f'{where}: node ports stay forbidden')
limit = ('0', '1') if namespace == 'evolution-dev' and route_enabled else ('0',)
check(hard.get('services.loadbalancers') in limit,
f'{where}: LoadBalancers only in evolution-dev with the enabled route')
if route_active:
check(overlay in ('deploy/kubernetes/region/dev', 'deploy/kubernetes/dev'),
f'{overlay}: renders the dev session route outside evolution-dev')
check(ROUTE_ENABLING_PERMITTED,
f'{overlay}: renders the dev session route, which cannot be enabled until ADR-0087 records '
'the implemented, tested server-enforced session windows')
check('REPLACE' not in yaml.safe_dump_all(docs), f'{overlay}: unresolved REPLACE marker in the rendered route')
for message in failures: for message in failures:
print('FAIL', message) print('FAIL', message)
print(f'deploy policy: {len(objects)} objects in {len(manifests)} files, {len(failures)} failures') print(f'deploy policy: {len(objects)} objects in {len(manifests)} files, '
f'{sum(len(d) for d in renders.values())} rendered objects in {len(renders)} overlays, {len(failures)} failures')
sys.exit(1 if failures else 0) sys.exit(1 if failures else 0)
+84 -7
View File
@@ -1,8 +1,10 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""M9.8f fail-closed tests for deploy-policy-check.py (ADR-0087). Needs Python 3 and """M9.8f fail-closed tests for deploy-policy-check.py (ADR-0087, ADR-0088). Needs
PyYAML only. Copies the manifests and the tools the check runs into a temporary tree, Python 3, PyYAML and the pinned kustomize (tools/ci/prepare-kustomize.sh). Copies the
applies one unsafe change at a time and requires the check to fail with the expected manifests and the tools the check runs into a temporary tree, applies one unsafe change
message; the unmodified copy must pass. at a time and requires the check to fail with the expected message; the unmodified copy
must pass. Includes inline strategic-merge and JSON patches and namespace changes, which
only the rendered-output checks can see.
""" """
import os import os
import re import re
@@ -41,6 +43,26 @@ def write(tree, path, text):
f.write(text) f.write(text)
def inline_patch(tree, path, patch, target=None):
"""Appends an inline patch (strategic merge, or JSON 6902 with a target)."""
import yaml
full = os.path.join(tree, path)
with open(full, encoding='utf-8') as f:
data = yaml.safe_load(f)
entry = {'patch': patch}
if target:
entry['target'] = target
data.setdefault('patches', []).append(entry)
with open(full, 'w', encoding='utf-8') as f:
yaml.safe_dump(data, f, sort_keys=False)
def env_patch(name, value):
return ('apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: evo-region\nspec:\n template:\n'
' spec:\n containers:\n - name: evo-server\n env:\n'
f' - {{name: {name}, value: "{value}"}}\n')
def enable(tree): def enable(tree):
append(tree, DEV, 'components:\n - ../dev-session-route\n') append(tree, DEV, 'components:\n - ../dev-session-route\n')
@@ -59,7 +81,47 @@ def quota_one(tree):
CASES = [ CASES = [
('enabling with unresolved markers', lambda t: enable(t), 'unresolved REPLACE marker'), ('enabling with unresolved markers', lambda t: enable(t), 'unresolved REPLACE marker'),
('enabling before server-enforced windows', lambda t: (enable(t), resolve(t), quota_one(t)), ('enabling before server-enforced windows', lambda t: (enable(t), resolve(t), quota_one(t)),
'server-enforced session windows'), 'cannot be enabled until ADR-0087'),
('non-RFC 1918 VIP without approval', lambda t: edit(t, ROUTE + 'service.yaml', 'REPLACE-dev-session-vip',
'172.10.1.210'), 'internal address'),
# Review M9.8f P1: inline/JSON patches and namespace changes, seen only when rendered.
('reviewer reproduction (inline listener, sessions, inputs, ops port)', lambda t: (
inline_patch(t, DEV, env_patch('REGION_SESSION_LISTEN', '0.0.0.0:7400')),
inline_patch(t, DEV, env_patch('REGION_SESSION', 'dev')),
inline_patch(t, DEV, env_patch('REGION_INPUTS', 'development-v1')),
inline_patch(t, DEV, 'apiVersion: v1\nkind: Service\nmetadata:\n name: evo-region-ops\nspec:\n ports:\n'
' - {name: session, port: 7400, targetPort: 7400, protocol: TCP}\n')),
'REGION_SESSION_LISTEN must be 127.0.0.1:7400'),
('inline patch: listener on the pod interface', lambda t: inline_patch(
t, DEV, env_patch('REGION_SESSION_LISTEN', '0.0.0.0:7400')), 'REGION_SESSION_LISTEN must be 127.0.0.1:7400'),
('inline patch: sessions on', lambda t: inline_patch(t, DEV, env_patch('REGION_SESSION', 'dev')),
'sessions and synthetic inputs stay off'),
('inline patch: synthetic inputs on', lambda t: inline_patch(
t, 'deploy/kubernetes/dev/kustomization.yaml', env_patch('REGION_INPUTS', 'development-v1')),
'sessions and synthetic inputs stay off'),
('JSON patch: session port on the ops Service', lambda t: inline_patch(
t, DEV, '- op: add\n path: /spec/ports/-\n value: {name: session, port: 7400, targetPort: 7400, protocol: TCP}\n',
{'kind': 'Service', 'name': 'evo-region-ops'}), 'only the route may be a LoadBalancer'),
('JSON patch: LoadBalancer ops Service', lambda t: inline_patch(
t, 'deploy/kubernetes/stage/kustomization.yaml', '- op: replace\n path: /spec/type\n value: LoadBalancer\n',
{'kind': 'Service', 'name': 'evo-region-ops'}), 'only the route may be a LoadBalancer'),
('JSON patch: container port 7400', lambda t: inline_patch(
t, DEV, '- op: add\n path: /spec/template/spec/containers/0/ports/-\n'
' value: {name: session, containerPort: 7400, protocol: TCP}\n',
{'kind': 'Deployment', 'name': 'evo-region'}), 'container ports must be'),
('JSON patch: second listener argument', lambda t: inline_patch(
t, DEV, '- op: add\n path: /spec/template/spec/containers/0/args/-\n value: --session-listen 0.0.0.0:7401\n',
{'kind': 'Deployment', 'name': 'evo-region'}), 'one session listener'),
('namespace transformation', lambda t: edit(t, DEV, 'namespace: evolution-dev', 'namespace: evolution-stage'),
'expected evolution-dev'),
('inline quota patch in stage', lambda t: inline_patch(
t, 'deploy/kubernetes/platform/stage/kustomization.yaml',
'apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: aevolution-quota\nspec:\n hard:\n'
' services.loadbalancers: "1"\n'), 'LoadBalancers only in evolution-dev'),
('secret generator', lambda t: append(t, DEV, 'secretGenerator:\n - name: leaked\n literals: [a=b]\n'),
'unsupported kustomize features'),
('helm chart inflation', lambda t: append(t, DEV, 'helmCharts:\n - name: x\n repo: https://example.invalid\n'),
'unsupported kustomize features'),
('enabling without the dev quota', lambda t: (enable(t), resolve(t)), 'exactly one LoadBalancer'), ('enabling without the dev quota', lambda t: (enable(t), resolve(t)), 'exactly one LoadBalancer'),
('route included by stage', lambda t: append(t, 'deploy/kubernetes/region/stage/kustomization.yaml', ('route included by stage', lambda t: append(t, 'deploy/kubernetes/region/stage/kustomization.yaml',
'components:\n - ../dev-session-route\n'), 'components:\n - ../dev-session-route\n'),
@@ -118,12 +180,27 @@ def copy_tree(target):
for directory in ('deploy', 'tools/ops', 'tools/backup'): for directory in ('deploy', 'tools/ops', 'tools/backup'):
shutil.copytree(os.path.join(ROOT, directory), os.path.join(target, directory)) shutil.copytree(os.path.join(ROOT, directory), os.path.join(target, directory))
os.makedirs(os.path.join(target, 'tools', 'ci')) os.makedirs(os.path.join(target, 'tools', 'ci'))
shutil.copy(os.path.join(ROOT, 'tools', 'ci', 'deploy-policy-check.py'), os.path.join(target, 'tools', 'ci')) for name in ('deploy-policy-check.py', 'prepare-kustomize.sh'):
shutil.copy(os.path.join(ROOT, 'tools', 'ci', name), os.path.join(target, 'tools', 'ci'))
def pinned_kustomize():
with open(os.path.join(ROOT, 'tools', 'ci', 'prepare-kustomize.sh'), encoding='utf-8') as f:
version = re.search(r'^version=(v[0-9.]+)$', f.read(), re.M).group(1)
for candidate in (os.environ.get('KUSTOMIZE', ''),
os.path.join(ROOT, '.cache', 'tools', f'kustomize-{version}', 'kustomize'),
shutil.which('kustomize') or ''):
if candidate and os.path.isfile(candidate):
return os.path.abspath(candidate)
raise SystemExit('selftest: the pinned kustomize is required (tools/ci/prepare-kustomize.sh)')
KUSTOMIZE = pinned_kustomize()
def run(tree): def run(tree):
result = subprocess.run([sys.executable, os.path.join(tree, 'tools', 'ci', 'deploy-policy-check.py')], result = subprocess.run([sys.executable, os.path.join(tree, 'tools', 'ci', 'deploy-policy-check.py')],
capture_output=True, text=True) capture_output=True, text=True, env=dict(os.environ, KUSTOMIZE=KUSTOMIZE))
return result.returncode, result.stdout + result.stderr return result.returncode, result.stdout + result.stderr
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# M9.8f review: pinned kustomize for rendered deployment-policy checks (ADR-0088).
# tools/ci/prepare-kustomize.sh [install-dir] (default .cache/tools/kustomize-<version>)
# Explicit preparation only: downloads the official Linux amd64 release archive,
# verifies the pinned SHA-256 and installs the single binary. Idempotent.
set -euo pipefail
version=v5.8.1
sha256=029a7f0f4e1932c52a0476cf02a0fd855c0bb85694b82c338fc648dcb53a819d
root="$(cd "$(dirname "$0")/../.." && pwd)"
target="${1:-$root/.cache/tools/kustomize-$version}"
if [ -x "$target/kustomize" ] && "$target/kustomize" version | grep -qx "$version"; then
echo "kustomize $version: $target/kustomize"
exit 0
fi
case "$(uname -s)/$(uname -m)" in
Linux/x86_64) ;;
*) echo "prepare-kustomize: only Linux x86_64 is pinned (got $(uname -s)/$(uname -m))" >&2; exit 2 ;;
esac
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
archive="kustomize_${version}_linux_amd64.tar.gz"
curl -fsSL --retry 3 -o "$work/$archive" \
"https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F$version/$archive"
echo "$sha256 $work/$archive" | sha256sum -c --quiet -
tar -xzf "$work/$archive" -C "$work" kustomize
mkdir -p "$target"
install -m 0755 "$work/kustomize" "$target/kustomize"
"$target/kustomize" version | grep -qx "$version"
echo "kustomize $version: $target/kustomize"