Check rendered kustomize output in the deploy policy (M9.8f review)
The policy check now renders every overlay with pinned, checksum-verified kustomize v5.8.1 (ADR-0088) and validates effective namespaces, session/input/ listener environment and arguments, ports, Services, quotas, policies, Ingresses and Secrets, so inline/JSON patches and namespace changes cannot bypass it. Route enabling is gated on an ADR-0087 amendment with tested server windows, route addresses must be RFC 1918 or explicitly approved, and the Local health-check node port is documented with its enabling evidence. Self-test grows to 33 cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -86,9 +86,12 @@ RUN cmake --preset analysis -B build/ecs-analysis -DCMAKE_CXX_COMPILER=clang++-1
|
||||
FROM ${UBUNTU_IMAGE} AS ci-deploy
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends python3 python3-yaml \
|
||||
&& apt-get install -y --no-install-recommends python3 python3-yaml curl ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
WORKDIR /source
|
||||
# Pinned, checksum-verified kustomize for the rendered-output checks (ADR-0088).
|
||||
COPY tools/ci/prepare-kustomize.sh tools/ci/
|
||||
RUN bash tools/ci/prepare-kustomize.sh /usr/local/bin
|
||||
COPY deploy/ deploy/
|
||||
COPY tools/ci/deploy-policy-check.py tools/ci/deploy-policy-selftest.py tools/ci/
|
||||
COPY tools/ops/ tools/ops/
|
||||
|
||||
@@ -3,7 +3,10 @@
|
||||
# name void.aeternalisgames.com resolves to (never the control-plane address).
|
||||
# - externalTrafficPolicy Local keeps client source addresses, so the NetworkPolicy's
|
||||
# ipBlock (same CIDRs) is enforceable; new connections reach only a ready pod.
|
||||
# - allocateLoadBalancerNodePorts false: no node ports (the quota forbids them).
|
||||
# - allocateLoadBalancerNodePorts false: no node port for the session data. Local
|
||||
# still makes Kubernetes allocate spec.healthCheckNodePort, an HTTP endpoint kube-proxy
|
||||
# serves on every node; it carries no session traffic and must stay closed at the edge
|
||||
# firewall. Its allocation and quota accounting are verified before enabling (ADR-0087).
|
||||
# - loadBalancerSourceRanges duplicates the CIDR at the Service; the edge firewall is
|
||||
# the third layer. TLS 1.3, the dev CA and single-use grants remain mandatory.
|
||||
apiVersion: v1
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
|
||||
## Status
|
||||
Accepted for M9.8f as design (2026-10-02), evolution-dev only. The route is committed
|
||||
**disabled**; nothing was applied to the cluster. Enabling it is blocked by policy until
|
||||
**disabled**; nothing was applied to the cluster. The M9.8f gate stays open until the
|
||||
route is enabled and connectivity is recorded. Enabling it is blocked by policy until
|
||||
the inputs and server-side window enforcement below exist. Narrowly supersedes
|
||||
ADR-0083 (loopback-only listener, never a Service) and ADR-0069 (no LoadBalancer
|
||||
Services) for evolution-dev, and only once enabled.
|
||||
@@ -23,8 +24,13 @@ fronts one LoadBalancer Service `evo-region-session` in evolution-dev, which for
|
||||
the region pod's port 7400. TLS 1.3 terminates in the region pod; its dev-CA server
|
||||
certificate carries `void.aeternalisgames.com`, which the client verifies. No HTTP
|
||||
Ingress, no TLS termination at nginx, no PROXY-protocol prefix. The Service uses
|
||||
`externalTrafficPolicy: Local` (client addresses preserved), no node ports, a pinned
|
||||
`metallb.io/loadBalancerIPs` annotation and `loadBalancerSourceRanges`. New connections
|
||||
`externalTrafficPolicy: Local` (client addresses preserved), no node port for the
|
||||
session data (`allocateLoadBalancerNodePorts: false`), a pinned
|
||||
`metallb.io/loadBalancerIPs` annotation and `loadBalancerSourceRanges`.
|
||||
**Health-check exception:** with `Local`, Kubernetes still allocates
|
||||
`spec.healthCheckNodePort`, an HTTP endpoint kube-proxy serves on every node reporting
|
||||
local endpoint health. It carries no session traffic, must stay closed at the edge
|
||||
firewall, and `services.nodeports: "0"` is not assumed to prevent or account for it. New connections
|
||||
reach only a ready pod; after a replacement, clients reconnect with a fresh grant and
|
||||
baseline (ADR-0083).
|
||||
|
||||
@@ -54,18 +60,37 @@ Expiry is enforced **inside evo-server**, not by a delayed cleanup job:
|
||||
closed Git configuration. Grant expiry alone does not stop the generator.
|
||||
|
||||
## Enabling (all required; `tools/ci/deploy-policy-check.py` refuses otherwise)
|
||||
1. The VIP (private, from the MetalLB pool) and permitted client CIDR(s) (private, no
|
||||
wider than /16) replace the REPLACE markers; Service ranges equal the policy's.
|
||||
0. `ROUTE_ENABLING_PERMITTED` in the policy check is `False` and is changed only together
|
||||
with an amendment here that records the implemented window argument/configuration
|
||||
contract and its runtime expiry tests (expiry during an active session, restart after
|
||||
expiry, delayed application, extend/close races, clock movement, synthetic inputs
|
||||
stopping while pending commits drain). A flag string in a manifest is not evidence.
|
||||
1. The VIP (from the MetalLB pool) and permitted client CIDR(s) (no wider than /16)
|
||||
replace the REPLACE markers; Service ranges equal the policy's. Addresses must be
|
||||
RFC 1918 or in a range approved here and listed in `APPROVED_INTERNAL` (none yet;
|
||||
the recorded cluster LAN addresses, 172.10.1.x, are not RFC 1918).
|
||||
2. `deploy/kubernetes/platform/dev` raises `services.loadbalancers` to exactly 1;
|
||||
node ports stay 0; stage and World0 stay 0.
|
||||
3. evo-server implements the session window (`--session-window`) as above, with tests
|
||||
and an amendment to this ADR, and the base deployment passes it.
|
||||
4. Only `region/dev` includes the component. Then: internal DNS for void to the VIP, the
|
||||
edge firewall rule, and a server-side dry-run (quota accounting of the Local health
|
||||
check port is to be confirmed there), a new dev server certificate for
|
||||
void.aeternalisgames.com, and live connectivity recorded separately in VALIDATION.md.
|
||||
edge firewall rule, a new dev server certificate for void.aeternalisgames.com, and live
|
||||
connectivity recorded separately in VALIDATION.md.
|
||||
5. Health-check port: a server-side dry-run and the applied object show the allocated
|
||||
`healthCheckNodePort` and how the quota accounts for it; the firewall handling is
|
||||
recorded. `services.nodeports` is not raised without that evidence.
|
||||
6. The rendered-output checks (ADR-0088) pass for every overlay with the route included.
|
||||
|
||||
## Open points for the maintainer
|
||||
Review recommendations (M9.8f review, 2026-10-02), pending the maintainer's decision:
|
||||
keep manual Argo sync for the first delivery and report a workflow-created window as
|
||||
pending until applied, with an absolute expiry that delayed sync never shifts (Close
|
||||
also needs propagation; the server deadline is the upper bound); keep the existing void
|
||||
HTTP endpoint unless explicitly retired, possibly choosing a dedicated session hostname
|
||||
instead (DNS cannot select by TCP port), before issuing the certificate; confirm the
|
||||
actual MetalLB pool and client addresses; set the maximum window length and clock
|
||||
handling before implementing windows.
|
||||
|
||||
- **Argo delivery.** The aevolution-dev Application has no automated sync, and the
|
||||
policy forbids auto-sync outside the platform app (ADR-0069). A workflow commit alone
|
||||
will not deploy; either a manual sync per window or a scoped dev exception must be
|
||||
@@ -83,6 +108,10 @@ Expiry is enforced **inside evo-server**, not by a delayed cleanup job:
|
||||
- NodePort: opens a port on every node; forbidden by ADR-0069.
|
||||
- Always-on sessions in dev: continuous synthetic history; rejected for now.
|
||||
|
||||
Policy evaluation (ADR-0088): the check renders every overlay with the pinned kustomize
|
||||
and validates the effective objects, so inline or JSON patches and namespace changes
|
||||
cannot enable listeners, sessions, inputs or session ports outside the route.
|
||||
|
||||
## Consequences
|
||||
One reviewed exception to "no LoadBalancer" for evolution-dev. The policy check and its
|
||||
fail-closed self-test (`tools/ci/deploy-policy-selftest.py`) encode the route's shape and
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
# ADR-0088: Rendered deployment-policy checks with pinned kustomize
|
||||
|
||||
## Status
|
||||
Accepted 2026-10-02 (M9.8f review correction). Extends ADR-0069's offline policy check.
|
||||
|
||||
## Context
|
||||
`tools/ci/deploy-policy-check.py` validated manifest files one by one. Kustomize
|
||||
inline strategic-merge or JSON patches, components and namespace transformations
|
||||
change the effective objects without appearing as checkable files; the M9.8f review
|
||||
reproduced inline patches enabling the session listener, sessions, synthetic inputs
|
||||
and a session port while the check passed.
|
||||
|
||||
## Decision
|
||||
- The check renders every non-component kustomization under `deploy/` with a pinned
|
||||
kustomize and validates the effective output: overlay namespaces; evo-region's
|
||||
session/input/listener environment, the single listener argument and container ports;
|
||||
Service types and the session port; route Service shape; NetworkPolicies admitting the
|
||||
session port; quotas; Ingresses; and the absence of Secrets. Source-file checks stay.
|
||||
- Kustomizations may use only the keys in use today (resources, components, patches,
|
||||
images, labels, common labels/annotations, namespace, ConfigMap generators);
|
||||
generators/transformers plugins, Helm inflation and Secret generators are refused.
|
||||
- Dependency: kustomize **v5.8.1** (kubernetes-sigs, Apache-2.0), official Linux amd64
|
||||
release archive, SHA-256
|
||||
029a7f0f4e1932c52a0476cf02a0fd855c0bb85694b82c338fc648dcb53a819d, matching the
|
||||
release's checksums.txt. Maintenance evidence (GitHub releases, 2026-10-02): v5.8.0
|
||||
2025-11-09, v5.8.1 2026-02-09, v5.8.2 2026-09-30. v5.8.1 is pinned rather than the
|
||||
two-day-old v5.8.2; updates change the version and checksum together.
|
||||
- Acquisition is explicit: `tools/ci/prepare-kustomize.sh [dir]` downloads, verifies and
|
||||
installs it (default `.cache/tools/kustomize-v5.8.1`); the CI deploy image runs it.
|
||||
The check uses `$KUSTOMIZE`, that cache path or `kustomize` on PATH, and fails unless
|
||||
the binary reports exactly the pinned version. No build step downloads it implicitly.
|
||||
|
||||
## Alternatives considered
|
||||
- `kubectl kustomize`: its embedded kustomize version follows kubectl releases.
|
||||
- Parsing patches in Python: re-implements kustomize semantics incompletely.
|
||||
- kubeconform/schema validation: complements but does not evaluate policy.
|
||||
|
||||
## Consequences
|
||||
Running the policy check locally needs the pinned binary (Linux x86_64 is pinned).
|
||||
Rendering adds seconds to the deploy suite. The self-test covers inline and JSON patches,
|
||||
namespace changes and unsupported features.
|
||||
@@ -97,6 +97,7 @@ design. No unadopted candidate is a link dependency of the proof.
|
||||
- [ADR-0083: Dev region sessions](0083-dev-region-sessions.md) - session server in evo-server, durable no-effect intentions on synthetic carriers, empty organism presentation, reconnect with fresh baselines, loopback-only drill-time dev deployment.
|
||||
|
||||
- [ADR-0084: Engineering UI toolkit](0084-engineering-ui-toolkit.md) - Dear ImGui 1.92.9b pinned (M9.8a) and integrated as the engineering client overlay with toolkit-first input routing (M9.8c).
|
||||
- [ADR-0088: Rendered deployment policy](0088-rendered-deployment-policy.md) - the deploy policy check renders every overlay with pinned kustomize v5.8.1 (checksum-verified, explicit preparation) and validates effective objects; unsupported kustomize features refused.
|
||||
- [ADR-0087: Dev session route](0087-dev-session-route.md) - evolution-dev only: void.aeternalisgames.com:7400 via a dedicated internal MetalLB VIP to the region pod, TLS end to end, CIDR-limited, operator-opened windows enforced by evo-server; committed disabled with fail-closed policy tests.
|
||||
- [ADR-0086: Windows client package](0086-windows-client-package.md) - static-runtime Windows x64 ZIP with explicit allowlist, licence texts, build identity, SHA256 manifest, credential guards and deterministic layout; local packaging until a Windows runner exists.
|
||||
- [ADR-0085: Rebindable engineering controls](0085-rebindable-engineering-controls.md) - SDL-free action/HID-key bindings, conflict validation, versioned all-or-nothing controls file, binding capture before toolkit/world.
|
||||
|
||||
@@ -4,9 +4,10 @@ Status: M9.8a probe, M9.8b owned state/capture/event bounds, M9.8c launcher,
|
||||
legend, labelled inspector and click selection, and M9.8d rebindable controls
|
||||
complete, 2026-10-02. M9.8e local Windows ZIP complete (ADR-0086); its Gitea
|
||||
Actions publication waits for a Windows runner and its clean-VM run for M9.8g.
|
||||
M9.8f route design, disabled manifests and fail-closed policy tests complete
|
||||
(ADR-0087); enabling waits for the VIP, client CIDR, DNS, dev quota and server-enforced
|
||||
session windows. M9.8g remains.
|
||||
M9.8f design, disabled manifests and rendered fail-closed policy checks delivered
|
||||
(ADR-0087, ADR-0088); the M9.8f gate stays open until the route is enabled (VIP, client
|
||||
CIDR, DNS, dev quota, health-check port evidence and server-enforced session windows)
|
||||
and connectivity is recorded. M9.8g remains.
|
||||
No application UI or deployment completion is claimed. See ADR-0084 for the toolkit boundary.
|
||||
|
||||
## Confirmed scope
|
||||
|
||||
@@ -3930,3 +3930,35 @@ were run directly), any cluster/server dry-run, MetalLB allocation, DNS, firewal
|
||||
connectivity. Server-enforced session windows and the Gitea window workflow are not
|
||||
implemented; Argo delivery for windows and the existing void HTTP Ingress are open
|
||||
maintainer decisions recorded in ADR-0087.
|
||||
|
||||
|
||||
## 2026-10-02 — M9.8f review corrections (.cache/reviews/M9.8f-9f622cf-review.md)
|
||||
|
||||
P1 (rendered configuration): the policy check now renders all 17 non-component
|
||||
kustomizations with pinned kustomize v5.8.1 (ADR-0088; official archive, SHA-256
|
||||
029a7f0f…a819d checked against the release checksums) and validates the effective
|
||||
objects: overlay namespaces, evo-region session/input/listener environment, a single
|
||||
listener argument, container ports, Service types and session port, route shape, session
|
||||
NetworkPolicies, quotas, Ingresses and no Secrets. Unsupported kustomize features
|
||||
(plugin generators/transformers, Helm, Secret generators) are refused. Route enabling is
|
||||
gated by `ROUTE_ENABLING_PERMITTED = False`, changed only with an ADR-0087 amendment
|
||||
recording the implemented window contract and runtime expiry tests; the earlier
|
||||
`--session-window` string test is gone. Route addresses must be RFC 1918 or listed in
|
||||
`APPROVED_INTERNAL` (empty; 172.10.1.x is not RFC 1918).
|
||||
P2 (health-check port): manifest comment and ADR-0087 now state that the Local policy
|
||||
allocates `healthCheckNodePort`; allocation, quota accounting and firewall handling are
|
||||
enabling preconditions. Review recommendations (manual Argo sync, preserving the void
|
||||
endpoint, address confirmation, window limits and tests) are recorded in ADR-0087 as
|
||||
pending maintainer decisions. The M9.8f gate stays open.
|
||||
|
||||
Executed (WSL Ubuntu, Python 3, PyYAML 6.0.1, kustomize v5.8.1 via the new prepare
|
||||
script, idempotent on rerun): policy check 70 source objects / 126 rendered objects in
|
||||
17 overlays, 0 failures; self-test 33/33 refused (20 earlier cases plus non-RFC 1918 VIP,
|
||||
the reviewer's combined inline patches, inline listener/session/input patches, JSON
|
||||
patches adding a session port, a LoadBalancer, a container port and a second listener
|
||||
argument, namespace transformation, inline stage quota, Secret generator, Helm inflation);
|
||||
the reviewer's m98f-policy-repro.py now exits 1 with the listener, session/input and
|
||||
ops-port failures in both region/dev and dev; the check fails without kustomize and with
|
||||
kustomize v5.4.3; ops-lib and ci-path fixtures pass.
|
||||
Not run: the ci-deploy Docker stage (no Docker here; its commands were run directly,
|
||||
including prepare-kustomize.sh), any cluster dry-run or live action.
|
||||
|
||||
@@ -89,5 +89,6 @@ ADR-0087 designs `void.aeternalisgames.com:7400` through a dedicated internal Me
|
||||
VIP to the region pod (TLS end to end, development/test CIDR only), opened only in
|
||||
operator windows that evo-server itself expires. The manifests are in
|
||||
`deploy/kubernetes/region/dev-session-route` and are not included anywhere; the policy
|
||||
check refuses to enable them until the VIP, client CIDR, dev quota and the server-side
|
||||
`--session-window` exist. Until then use the port-forward procedures above.
|
||||
check refuses to enable them until the VIP, client CIDR, dev quota and server-enforced
|
||||
session windows exist (ADR-0087 enabling list). Until then use the port-forward
|
||||
procedures above.
|
||||
|
||||
@@ -46,7 +46,7 @@ case "$suite" in
|
||||
pattern="$common|$native|tests/|$postgres_tools|third_party/mbedtls/|tools/prepare-mbedtls\.cmake|tools/mbedtls-pin\.cmake|deploy/docker/Dockerfile"
|
||||
;;
|
||||
deploy)
|
||||
pattern="$common|deploy/|tools/ops/|tools/backup/|tools/ci/deploy-policy-(check|selftest)\.py|tools/ci/monitoring-check\.sh|tools/ci/ops-lib-check\.sh"
|
||||
pattern="$common|deploy/|tools/ops/|tools/backup/|tools/ci/deploy-policy-(check|selftest)\.py|tools/ci/prepare-kustomize\.sh|tools/ci/monitoring-check\.sh|tools/ci/ops-lib-check\.sh"
|
||||
;;
|
||||
server-image)
|
||||
pattern="$common|$native|migrations/|tools/prepare-postgres\.sh|tools/ci/server-image-smoke\.sh|deploy/docker/Dockerfile"
|
||||
|
||||
@@ -28,10 +28,31 @@ environment-isolation or World0-protection invariant is broken:
|
||||
- the dev session route (ADR-0087): loopback listener everywhere except the
|
||||
evolution-dev route component; that component is the only LoadBalancer and the
|
||||
only Service on the session port, narrowly shaped, and it cannot be enabled while
|
||||
markers are unresolved or before server-enforced session windows exist.
|
||||
markers are unresolved or before server-enforced session windows exist;
|
||||
- rendered output (ADR-0088): every overlay is built with the pinned kustomize and
|
||||
the effective objects are checked (namespaces, listener/session/input environment,
|
||||
arguments, ports, Services, quotas, policies, Ingresses, Secrets), so inline or
|
||||
JSON patches and namespace transformations cannot bypass the source checks.
|
||||
"""
|
||||
import ipaddress
|
||||
import re
|
||||
|
||||
# Enabling the dev session route needs ADR-0087 to be amended with the implemented
|
||||
# session-window argument contract and runtime expiry tests (expiry during a session,
|
||||
# restart after expiry, delayed application, extend/close races, clock movement,
|
||||
# synthetic inputs stopping while commits drain). Until then this stays False; a
|
||||
# string such as --session-window in a manifest is not proof of enforcement.
|
||||
ROUTE_ENABLING_PERMITTED = False
|
||||
# Addresses the route may use: RFC 1918 plus internal ranges the maintainer has
|
||||
# explicitly approved in ADR-0087 (none yet; the cluster LAN 172.10.0.0/16 is not
|
||||
# RFC 1918 and needs that approval before use).
|
||||
APPROVED_INTERNAL = []
|
||||
INTERNAL = [ipaddress.ip_network(n) for n in ('10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16')] + \
|
||||
[ipaddress.ip_network(n) for n in APPROVED_INTERNAL]
|
||||
|
||||
|
||||
def internal(network):
|
||||
return any(network.version == r.version and network.subnet_of(r) for r in INTERNAL)
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
@@ -315,7 +336,9 @@ for path, doc in objects:
|
||||
check(len(route_ranges) >= 1, f'{path}: loadBalancerSourceRanges must name the permitted CIDRs')
|
||||
if not vip.startswith('REPLACE'):
|
||||
try:
|
||||
check(ipaddress.ip_address(vip).is_private, f'{path}: the VIP must be an internal address')
|
||||
address = ipaddress.ip_address(vip)
|
||||
check(internal(ipaddress.ip_network(f'{address}/{address.max_prefixlen}')),
|
||||
f'{path}: the VIP must be an internal address (RFC 1918 or approved in ADR-0087)')
|
||||
except ValueError:
|
||||
check(False, f'{path}: the VIP must be one IP address')
|
||||
for cidr in route_ranges:
|
||||
@@ -323,7 +346,7 @@ for path, doc in objects:
|
||||
continue
|
||||
try:
|
||||
network = ipaddress.ip_network(cidr)
|
||||
check(network.is_private and network.prefixlen >= (16 if network.version == 4 else 48),
|
||||
check(internal(network) and network.prefixlen >= (16 if network.version == 4 else 48),
|
||||
f'{path}: {cidr} must be a private range no wider than /16 (IPv6 /48)')
|
||||
except ValueError:
|
||||
check(False, f'{path}: {cidr} is not a CIDR')
|
||||
@@ -354,11 +377,9 @@ if route_enabled:
|
||||
for name in os.listdir(os.path.join(ROOT, ROUTE)):
|
||||
with open(os.path.join(ROOT, ROUTE, name), encoding='utf-8') as handle:
|
||||
check('REPLACE' not in handle.read(), f'{ROUTE}{name}: unresolved REPLACE marker with the route enabled')
|
||||
base_script = ' '.join(' '.join(c.get('args', [])) for p, d in objects
|
||||
if p == 'deploy/kubernetes/region/base/deployment.yaml' and d['kind'] == 'Deployment'
|
||||
for c in d['spec']['template']['spec']['containers'])
|
||||
check('--session-window' in base_script,
|
||||
'the dev session route needs server-enforced session windows (--session-window, ADR-0087)')
|
||||
check(ROUTE_ENABLING_PERMITTED,
|
||||
'the dev session route cannot be enabled until ADR-0087 records the implemented, tested '
|
||||
'server-enforced session windows')
|
||||
dev_quota = [d.get('spec', {}).get('hard', {}).get('services.loadbalancers') for p, d in objects
|
||||
if d['kind'] == 'ResourceQuota' and p.startswith(f'{platform}/dev/')]
|
||||
check(dev_quota == ['1'], 'the dev quota must allow exactly one LoadBalancer for the route')
|
||||
@@ -541,7 +562,123 @@ for tool, args, component in generators:
|
||||
f'{tool}: component {component} must be admitted by the database egress policy')
|
||||
check(pod['metadata'].get('namespace') == 'evolution-dev', f'{tool}: renders into its environment')
|
||||
|
||||
|
||||
# ADR-0088: the effective configuration is what kustomize renders. Every overlay is
|
||||
# built with the pinned binary and its output checked, independent of how patches,
|
||||
# components or namespace transformations produced it.
|
||||
def kustomize_binary():
|
||||
with open(os.path.join(ROOT, 'tools', 'ci', 'prepare-kustomize.sh'), encoding='utf-8') as f:
|
||||
version = re.search(r'^version=(v[0-9.]+)$', f.read(), re.M).group(1)
|
||||
candidates = [os.environ.get('KUSTOMIZE', ''),
|
||||
os.path.join(ROOT, '.cache', 'tools', f'kustomize-{version}', 'kustomize'),
|
||||
shutil.which('kustomize') or '']
|
||||
for candidate in candidates:
|
||||
if candidate and os.path.isfile(candidate):
|
||||
reported = subprocess.run([candidate, 'version'], capture_output=True, text=True).stdout.strip()
|
||||
if reported == version:
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
SUPPORTED_KEYS = {'apiVersion', 'kind', 'namespace', 'resources', 'components', 'patches', 'images',
|
||||
'labels', 'commonAnnotations', 'commonLabels', 'configMapGenerator', 'generatorOptions'}
|
||||
for path, ds in manifests.items():
|
||||
if path.endswith('kustomization.yaml') and ds:
|
||||
unsupported = set(ds[0]) - SUPPORTED_KEYS
|
||||
check(not unsupported, f'{path}: unsupported kustomize features {sorted(unsupported)} (ADR-0088)')
|
||||
EXPECTED_NAMESPACE = {'deploy/kubernetes/dev': 'evolution-dev', 'deploy/kubernetes/stage': 'evolution-stage',
|
||||
'deploy/kubernetes/monitoring': 'aevolution-monitoring'}
|
||||
for env in ('dev', 'stage', 'world0'):
|
||||
for component in ('platform', 'region', 'stores'):
|
||||
EXPECTED_NAMESPACE[f'deploy/kubernetes/{component}/{env}'] = f'evolution-{env}'
|
||||
EXPECTED_NAMESPACE['deploy/kubernetes/platform/monitoring'] = 'aevolution-monitoring'
|
||||
CLUSTER_KINDS = {'Namespace', 'StorageClass', 'ValidatingAdmissionPolicy', 'ValidatingAdmissionPolicyBinding',
|
||||
'ClusterRole', 'ClusterRoleBinding', 'PersistentVolume'}
|
||||
binary = kustomize_binary()
|
||||
check(binary is not None, 'the pinned kustomize is required: run tools/ci/prepare-kustomize.sh (ADR-0088)')
|
||||
renders = {}
|
||||
if binary:
|
||||
for path, ds in sorted(manifests.items()):
|
||||
if not path.endswith('kustomization.yaml') or not ds or ds[0].get('kind') == 'Component':
|
||||
continue
|
||||
overlay = os.path.dirname(path)
|
||||
result = subprocess.run([binary, 'build', os.path.join(ROOT, overlay)], capture_output=True, text=True)
|
||||
check(result.returncode == 0, f'{overlay}: kustomize build failed: {result.stderr.strip()[:300]}')
|
||||
if result.returncode == 0:
|
||||
renders[overlay] = [d for d in yaml.safe_load_all(result.stdout) if isinstance(d, dict)]
|
||||
for overlay, docs in renders.items():
|
||||
expected = EXPECTED_NAMESPACE.get(overlay)
|
||||
route_services = [d for d in docs if d['kind'] == 'Service' and d['metadata']['name'] == 'evo-region-session']
|
||||
route_active = bool(route_services)
|
||||
for doc in docs:
|
||||
kind, name = doc['kind'], doc['metadata'].get('name', '')
|
||||
namespace = doc['metadata'].get('namespace')
|
||||
where = f'{overlay} (rendered) {kind}/{name}'
|
||||
if expected and kind not in CLUSTER_KINDS:
|
||||
check(namespace == expected, f'{where}: renders into {namespace}, expected {expected}')
|
||||
check(kind != 'Secret', f'{where}: rendered Secrets never come from Git')
|
||||
if kind == 'Deployment' and name == 'evo-region':
|
||||
pod = doc['spec']['template']['spec']
|
||||
for container in pod.get('containers', []) + pod.get('initContainers', []):
|
||||
env = {v.get('name'): v.get('value') for v in container.get('env', [])}
|
||||
text = ' '.join(container.get('command', []) + container.get('args', []))
|
||||
if container.get('name') == 'evo-server':
|
||||
check(env.get('REGION_SESSION') == '' and env.get('REGION_INPUTS') == '',
|
||||
f'{where}: sessions and synthetic inputs stay off in Git')
|
||||
listen = '0.0.0.0:7400' if route_active else '127.0.0.1:7400'
|
||||
check(env.get('REGION_SESSION_LISTEN') == listen,
|
||||
f'{where}: REGION_SESSION_LISTEN must be {listen} (route {"on" if route_active else "off"})')
|
||||
check(text.count('--session-listen') == 1 and '--session-listen "$REGION_SESSION_LISTEN"' in text,
|
||||
f'{where}: one session listener, addressed only by REGION_SESSION_LISTEN')
|
||||
else:
|
||||
check('--session-listen' not in text and '--synthetic-inputs' not in text,
|
||||
f'{where}: only evo-server may listen for sessions')
|
||||
ports = sorted((p.get('name'), p.get('containerPort')) for p in container.get('ports', []))
|
||||
allowed = [('ops', 9100), ('session', 7400)] if route_active else [('ops', 9100)]
|
||||
if container.get('name') == 'evo-server':
|
||||
check(ports == sorted(allowed), f'{where}: container ports must be {allowed}')
|
||||
if kind == 'Service':
|
||||
spec = doc['spec']
|
||||
session = any(p.get('port') == 7400 or p.get('targetPort') in (7400, 'session')
|
||||
for p in spec.get('ports', []))
|
||||
if name == 'evo-region-session':
|
||||
check(namespace == 'evolution-dev' and spec.get('type') == 'LoadBalancer'
|
||||
and spec.get('externalTrafficPolicy') == 'Local'
|
||||
and spec.get('allocateLoadBalancerNodePorts') is False
|
||||
and spec.get('ports') == [{'name': 'session', 'port': 7400, 'targetPort': 'session',
|
||||
'protocol': 'TCP'}],
|
||||
f'{where}: the route keeps its reviewed shape in evolution-dev')
|
||||
else:
|
||||
check(spec.get('type', 'ClusterIP') == 'ClusterIP' and not session,
|
||||
f'{where}: only the route may be a LoadBalancer or expose the session port')
|
||||
if kind == 'Ingress':
|
||||
check('7400' not in yaml.safe_dump(doc) and 'session' not in yaml.safe_dump(doc),
|
||||
f'{where}: the dev session is never an Ingress')
|
||||
if kind == 'NetworkPolicy':
|
||||
for rule in doc['spec'].get('ingress', []):
|
||||
if any(p.get('port') == 7400 for p in rule.get('ports', [])):
|
||||
ranges = route_services[0]['spec'].get('loadBalancerSourceRanges', []) if route_services else []
|
||||
blocks = [peer.get('ipBlock', {}) for peer in rule.get('from', [])]
|
||||
check(name == 'evo-region-session-ingress' and route_active and blocks
|
||||
and all(set(b) == {'cidr'} for b in blocks) and len(blocks) == len(rule.get('from', []))
|
||||
and sorted(b['cidr'] for b in blocks) == sorted(ranges),
|
||||
f'{where}: only the route policy admits TCP 7400, from exactly its source ranges')
|
||||
if kind == 'ResourceQuota':
|
||||
hard = doc['spec'].get('hard', {})
|
||||
check(hard.get('services.nodeports') == '0', f'{where}: node ports stay forbidden')
|
||||
limit = ('0', '1') if namespace == 'evolution-dev' and route_enabled else ('0',)
|
||||
check(hard.get('services.loadbalancers') in limit,
|
||||
f'{where}: LoadBalancers only in evolution-dev with the enabled route')
|
||||
if route_active:
|
||||
check(overlay in ('deploy/kubernetes/region/dev', 'deploy/kubernetes/dev'),
|
||||
f'{overlay}: renders the dev session route outside evolution-dev')
|
||||
check(ROUTE_ENABLING_PERMITTED,
|
||||
f'{overlay}: renders the dev session route, which cannot be enabled until ADR-0087 records '
|
||||
'the implemented, tested server-enforced session windows')
|
||||
check('REPLACE' not in yaml.safe_dump_all(docs), f'{overlay}: unresolved REPLACE marker in the rendered route')
|
||||
|
||||
for message in failures:
|
||||
print('FAIL', message)
|
||||
print(f'deploy policy: {len(objects)} objects in {len(manifests)} files, {len(failures)} failures')
|
||||
print(f'deploy policy: {len(objects)} objects in {len(manifests)} files, '
|
||||
f'{sum(len(d) for d in renders.values())} rendered objects in {len(renders)} overlays, {len(failures)} failures')
|
||||
sys.exit(1 if failures else 0)
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
"""M9.8f fail-closed tests for deploy-policy-check.py (ADR-0087). Needs Python 3 and
|
||||
PyYAML only. Copies the manifests and the tools the check runs into a temporary tree,
|
||||
applies one unsafe change at a time and requires the check to fail with the expected
|
||||
message; the unmodified copy must pass.
|
||||
"""M9.8f fail-closed tests for deploy-policy-check.py (ADR-0087, ADR-0088). Needs
|
||||
Python 3, PyYAML and the pinned kustomize (tools/ci/prepare-kustomize.sh). Copies the
|
||||
manifests and the tools the check runs into a temporary tree, applies one unsafe change
|
||||
at a time and requires the check to fail with the expected message; the unmodified copy
|
||||
must pass. Includes inline strategic-merge and JSON patches and namespace changes, which
|
||||
only the rendered-output checks can see.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
@@ -41,6 +43,26 @@ def write(tree, path, text):
|
||||
f.write(text)
|
||||
|
||||
|
||||
def inline_patch(tree, path, patch, target=None):
|
||||
"""Appends an inline patch (strategic merge, or JSON 6902 with a target)."""
|
||||
import yaml
|
||||
full = os.path.join(tree, path)
|
||||
with open(full, encoding='utf-8') as f:
|
||||
data = yaml.safe_load(f)
|
||||
entry = {'patch': patch}
|
||||
if target:
|
||||
entry['target'] = target
|
||||
data.setdefault('patches', []).append(entry)
|
||||
with open(full, 'w', encoding='utf-8') as f:
|
||||
yaml.safe_dump(data, f, sort_keys=False)
|
||||
|
||||
|
||||
def env_patch(name, value):
|
||||
return ('apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: evo-region\nspec:\n template:\n'
|
||||
' spec:\n containers:\n - name: evo-server\n env:\n'
|
||||
f' - {{name: {name}, value: "{value}"}}\n')
|
||||
|
||||
|
||||
def enable(tree):
|
||||
append(tree, DEV, 'components:\n - ../dev-session-route\n')
|
||||
|
||||
@@ -59,7 +81,47 @@ def quota_one(tree):
|
||||
CASES = [
|
||||
('enabling with unresolved markers', lambda t: enable(t), 'unresolved REPLACE marker'),
|
||||
('enabling before server-enforced windows', lambda t: (enable(t), resolve(t), quota_one(t)),
|
||||
'server-enforced session windows'),
|
||||
'cannot be enabled until ADR-0087'),
|
||||
('non-RFC 1918 VIP without approval', lambda t: edit(t, ROUTE + 'service.yaml', 'REPLACE-dev-session-vip',
|
||||
'172.10.1.210'), 'internal address'),
|
||||
# Review M9.8f P1: inline/JSON patches and namespace changes, seen only when rendered.
|
||||
('reviewer reproduction (inline listener, sessions, inputs, ops port)', lambda t: (
|
||||
inline_patch(t, DEV, env_patch('REGION_SESSION_LISTEN', '0.0.0.0:7400')),
|
||||
inline_patch(t, DEV, env_patch('REGION_SESSION', 'dev')),
|
||||
inline_patch(t, DEV, env_patch('REGION_INPUTS', 'development-v1')),
|
||||
inline_patch(t, DEV, 'apiVersion: v1\nkind: Service\nmetadata:\n name: evo-region-ops\nspec:\n ports:\n'
|
||||
' - {name: session, port: 7400, targetPort: 7400, protocol: TCP}\n')),
|
||||
'REGION_SESSION_LISTEN must be 127.0.0.1:7400'),
|
||||
('inline patch: listener on the pod interface', lambda t: inline_patch(
|
||||
t, DEV, env_patch('REGION_SESSION_LISTEN', '0.0.0.0:7400')), 'REGION_SESSION_LISTEN must be 127.0.0.1:7400'),
|
||||
('inline patch: sessions on', lambda t: inline_patch(t, DEV, env_patch('REGION_SESSION', 'dev')),
|
||||
'sessions and synthetic inputs stay off'),
|
||||
('inline patch: synthetic inputs on', lambda t: inline_patch(
|
||||
t, 'deploy/kubernetes/dev/kustomization.yaml', env_patch('REGION_INPUTS', 'development-v1')),
|
||||
'sessions and synthetic inputs stay off'),
|
||||
('JSON patch: session port on the ops Service', lambda t: inline_patch(
|
||||
t, DEV, '- op: add\n path: /spec/ports/-\n value: {name: session, port: 7400, targetPort: 7400, protocol: TCP}\n',
|
||||
{'kind': 'Service', 'name': 'evo-region-ops'}), 'only the route may be a LoadBalancer'),
|
||||
('JSON patch: LoadBalancer ops Service', lambda t: inline_patch(
|
||||
t, 'deploy/kubernetes/stage/kustomization.yaml', '- op: replace\n path: /spec/type\n value: LoadBalancer\n',
|
||||
{'kind': 'Service', 'name': 'evo-region-ops'}), 'only the route may be a LoadBalancer'),
|
||||
('JSON patch: container port 7400', lambda t: inline_patch(
|
||||
t, DEV, '- op: add\n path: /spec/template/spec/containers/0/ports/-\n'
|
||||
' value: {name: session, containerPort: 7400, protocol: TCP}\n',
|
||||
{'kind': 'Deployment', 'name': 'evo-region'}), 'container ports must be'),
|
||||
('JSON patch: second listener argument', lambda t: inline_patch(
|
||||
t, DEV, '- op: add\n path: /spec/template/spec/containers/0/args/-\n value: --session-listen 0.0.0.0:7401\n',
|
||||
{'kind': 'Deployment', 'name': 'evo-region'}), 'one session listener'),
|
||||
('namespace transformation', lambda t: edit(t, DEV, 'namespace: evolution-dev', 'namespace: evolution-stage'),
|
||||
'expected evolution-dev'),
|
||||
('inline quota patch in stage', lambda t: inline_patch(
|
||||
t, 'deploy/kubernetes/platform/stage/kustomization.yaml',
|
||||
'apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: aevolution-quota\nspec:\n hard:\n'
|
||||
' services.loadbalancers: "1"\n'), 'LoadBalancers only in evolution-dev'),
|
||||
('secret generator', lambda t: append(t, DEV, 'secretGenerator:\n - name: leaked\n literals: [a=b]\n'),
|
||||
'unsupported kustomize features'),
|
||||
('helm chart inflation', lambda t: append(t, DEV, 'helmCharts:\n - name: x\n repo: https://example.invalid\n'),
|
||||
'unsupported kustomize features'),
|
||||
('enabling without the dev quota', lambda t: (enable(t), resolve(t)), 'exactly one LoadBalancer'),
|
||||
('route included by stage', lambda t: append(t, 'deploy/kubernetes/region/stage/kustomization.yaml',
|
||||
'components:\n - ../dev-session-route\n'),
|
||||
@@ -118,12 +180,27 @@ def copy_tree(target):
|
||||
for directory in ('deploy', 'tools/ops', 'tools/backup'):
|
||||
shutil.copytree(os.path.join(ROOT, directory), os.path.join(target, directory))
|
||||
os.makedirs(os.path.join(target, 'tools', 'ci'))
|
||||
shutil.copy(os.path.join(ROOT, 'tools', 'ci', 'deploy-policy-check.py'), os.path.join(target, 'tools', 'ci'))
|
||||
for name in ('deploy-policy-check.py', 'prepare-kustomize.sh'):
|
||||
shutil.copy(os.path.join(ROOT, 'tools', 'ci', name), os.path.join(target, 'tools', 'ci'))
|
||||
|
||||
|
||||
def pinned_kustomize():
|
||||
with open(os.path.join(ROOT, 'tools', 'ci', 'prepare-kustomize.sh'), encoding='utf-8') as f:
|
||||
version = re.search(r'^version=(v[0-9.]+)$', f.read(), re.M).group(1)
|
||||
for candidate in (os.environ.get('KUSTOMIZE', ''),
|
||||
os.path.join(ROOT, '.cache', 'tools', f'kustomize-{version}', 'kustomize'),
|
||||
shutil.which('kustomize') or ''):
|
||||
if candidate and os.path.isfile(candidate):
|
||||
return os.path.abspath(candidate)
|
||||
raise SystemExit('selftest: the pinned kustomize is required (tools/ci/prepare-kustomize.sh)')
|
||||
|
||||
|
||||
KUSTOMIZE = pinned_kustomize()
|
||||
|
||||
|
||||
def run(tree):
|
||||
result = subprocess.run([sys.executable, os.path.join(tree, 'tools', 'ci', 'deploy-policy-check.py')],
|
||||
capture_output=True, text=True)
|
||||
capture_output=True, text=True, env=dict(os.environ, KUSTOMIZE=KUSTOMIZE))
|
||||
return result.returncode, result.stdout + result.stderr
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env bash
|
||||
# M9.8f review: pinned kustomize for rendered deployment-policy checks (ADR-0088).
|
||||
# tools/ci/prepare-kustomize.sh [install-dir] (default .cache/tools/kustomize-<version>)
|
||||
# Explicit preparation only: downloads the official Linux amd64 release archive,
|
||||
# verifies the pinned SHA-256 and installs the single binary. Idempotent.
|
||||
set -euo pipefail
|
||||
version=v5.8.1
|
||||
sha256=029a7f0f4e1932c52a0476cf02a0fd855c0bb85694b82c338fc648dcb53a819d
|
||||
root="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
target="${1:-$root/.cache/tools/kustomize-$version}"
|
||||
if [ -x "$target/kustomize" ] && "$target/kustomize" version | grep -qx "$version"; then
|
||||
echo "kustomize $version: $target/kustomize"
|
||||
exit 0
|
||||
fi
|
||||
case "$(uname -s)/$(uname -m)" in
|
||||
Linux/x86_64) ;;
|
||||
*) echo "prepare-kustomize: only Linux x86_64 is pinned (got $(uname -s)/$(uname -m))" >&2; exit 2 ;;
|
||||
esac
|
||||
work="$(mktemp -d)"
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
archive="kustomize_${version}_linux_amd64.tar.gz"
|
||||
curl -fsSL --retry 3 -o "$work/$archive" \
|
||||
"https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F$version/$archive"
|
||||
echo "$sha256 $work/$archive" | sha256sum -c --quiet -
|
||||
tar -xzf "$work/$archive" -C "$work" kustomize
|
||||
mkdir -p "$target"
|
||||
install -m 0755 "$work/kustomize" "$target/kustomize"
|
||||
"$target/kustomize" version | grep -qx "$version"
|
||||
echo "kustomize $version: $target/kustomize"
|
||||
Reference in New Issue
Block a user