# SOR.Cdn 12-Factor Refactor Plan Date: 2026-06-10 Scope reviewed: - `Server/src/SOR.Cdn` - CDN registration and usage in `Server/src/SOR.Admin` - CDN proxy endpoint in `Server/src/SOR.Host` - CDN-related catalog and bazaar entities under `Server/src/SOR.Database` Reference: https://12factor.net/ ## Executive Summary `SOR.Cdn` is a small infrastructure module that wraps S3-compatible object storage, currently Ceph RadosGW, behind `ICdnClient`. It is primarily used by `SOR.Admin` to upload card art, animation assets, bazaar product images, cosmetic assets, and content assets. `SOR.Host` also exposes a `/cdn/{path}` proxy that forwards browser requests to the public CDN base URL and adds permissive CORS headers. The module already has several 12-Factor-friendly traits: - CDN access is configured through the `Cdn` configuration section. - Credentials are intended to come from environment variables or Kubernetes secrets. - The S3 client is registered through DI and hidden behind `ICdnClient` for upload/delete operations. - The module does not keep local filesystem state. - Public URLs are derived from a configured `PublicBaseUrl`. - Upload operations return structured results with object key, public URL, error, and ETag. The main gaps are around config validation, operational boundaries, and consistent abstraction: - `CdnOptions` requires `EndpointUrl`, `AccessKey`, `SecretKey`, and `BucketName`, but current appsettings comments still refer to older `UploadBaseUrl` and `ApiKey` keys. - `AddCdnClient` binds options but does not validate required values, URL formats, bucket names, or credential presence. - Admin pages sometimes inject raw `IAmazonS3` directly for browsing and deleting objects, bypassing `ICdnClient`. - Upload path construction accepts caller-supplied folder and file names with little normalization policy. - Uploads force `PublicRead` ACL and immutable caching without a clear environment policy. - There is no central content validation policy for allowed MIME types, max sizes, folder prefixes, or asset classes. - Deletes are hard deletes with minimal audit or reference checks. - CDN health/readiness is not checked, even though Admin depends on it for asset management. The plan is to keep `SOR.Cdn` as an infrastructure module, but make it a complete object-storage boundary: typed and validated config, controlled key generation, asset policy enforcement, browsing/listing APIs, reference-aware deletion, audit metadata, and dependency-aware readiness. ## Current Structure Observations ### Strong Foundations - `SOR.Cdn` has a narrow public interface: `ICdnClient`. - `S3CdnClient` uses `IAmazonS3`, which keeps provider-specific code localized. - `AmazonS3Client` is registered as a singleton, matching the SDK connection-pool model. - `ForcePathStyle = true` is explicitly set for Ceph RadosGW compatibility. - Uploads set cache-control headers for immutable assets. - Upload results include the S3 `ETag`. - Admin already uses `ICdnClient` for many upload workflows. - The database stores CDN URLs and keys in catalog, bazaar, and identity-related entities instead of storing binary asset data. ### Main Problems - `CdnOptions` is not validated at startup. - Config comments in `SOR.Admin` and `SOR.Host` appsettings mention `UploadBaseUrl` and `ApiKey`, which no longer match `CdnOptions`. - `AuthenticationRegion` is hardcoded to `us-east-1`. - Public-read ACL is hardcoded. - Immutable cache-control is hardcoded. - Path generation is simple string concatenation: `{folder.Trim('/')}/{fileName}`. - File names are supplied by callers, and different admin pages choose folder/key conventions independently. - Direct `IAmazonS3` usage in Admin pages duplicates listing, URL building, and delete behavior. - There is no first-class list/copy/head/presign API in `ICdnClient`. - There is no central asset-policy layer for file extensions, content types, max size, malware scanning hooks, or allowed prefixes. - Delete operations do not check whether database records still reference the object key or URL. - Upload success logs include public URLs but do not include stable operation metadata such as key, ETag, size, content type, caller, or asset type. - The CDN proxy in `SOR.Host` forwards GET requests without cache headers, range support, size limits, request timeout policy, or dependency health. ## Target CDN Shape Keep `SOR.Cdn` as a reusable infrastructure module, but make it the only object-storage boundary: ```text Server/src/SOR.Cdn/ Configuration/ CdnOptions.cs CdnUploadOptions.cs CdnSecurityOptions.cs CdnOptionsValidator.cs Bootstrap/ CdnRegistration.cs CdnHealthChecks.cs Storage/ ICdnClient.cs S3CdnClient.cs CdnObjectInfo.cs CdnUploadResult.cs CdnDeleteResult.cs CdnListResult.cs Policy/ ICdnKeyBuilder.cs CdnKeyBuilder.cs ICdnAssetPolicy.cs CdnAssetPolicy.cs CdnContentTypePolicy.cs References/ ICdnReferenceScanner.cs CdnReferenceReport.cs Audit/ ICdnAuditSink.cs CdnAuditEvent.cs ``` Desired consumer shape: - Admin pages upload, list, inspect, and delete objects through `ICdnClient` or higher-level content services. - No admin component builds public URLs by hand. - No admin component injects `IAmazonS3` directly. - Object keys are generated by policy, not raw user filenames. - Deletion requires a reference check or explicit force-delete path. ## 12-Factor Gap Map ### 1. Codebase Mostly aligned. `SOR.Cdn` is a small module in the server codebase, but CDN behavior is spread into Admin pages and Host proxy code. Plan: - Keep one CDN module. - Move object listing, public URL building, key normalization, and delete semantics into `SOR.Cdn`. - Keep Host proxy registration separate but backed by shared CDN options. ### 2. Dependencies Mostly aligned. The AWS S3 SDK dependency is explicit in `SOR.Cdn.csproj`. Plan: - Keep `AWSSDK.S3` isolated in `SOR.Cdn`. - Remove direct `IAmazonS3` usage from Admin pages over time. - Add interfaces for object listing, metadata/head checks, and safe deletion. - Keep provider-specific SDK types out of admin application code. ### 3. Config Partially aligned. CDN config is externalized but not validated, and current appsettings comments do not match the code. Plan: - Update config names to match `CdnOptions`. - Add validation for required values and URL formats. - Move region, path-style behavior, ACL, cache-control, max upload size, allowed prefixes, and allowed content types into typed options. - Validate required CDN config only in processes that actually use CDN write/list features. Candidate config shape: ```json { "Cdn": { "PublicBaseUrl": "https://cdn.example.com", "EndpointUrl": "https://s3.example.internal", "AccessKey": "", "SecretKey": "", "BucketName": "sor-assets", "AuthenticationRegion": "us-east-1", "ForcePathStyle": true, "DefaultAcl": "PublicRead", "CacheControl": "public, max-age=31536000, immutable", "MaxUploadBytes": 104857600, "AllowedPrefixes": [ "cardart/", "models/", "cosmetics/", "bazaar/", "audio/" ] } } ``` ### 4. Backing Services Mostly aligned. Ceph RadosGW/S3 is an attached backing service. Plan: - Treat object storage as a required attached service for Admin asset workflows. - Add readiness checks for bucket existence and credential permissions. - Document minimum required S3 permissions: put, get/head, list, delete, and optional copy. - Keep CDN/public serving independent from application build artifacts. ### 5. Build, Release, Run Partially aligned. Asset binaries are already external to the app build, but asset promotion and cache policy are not formalized. Plan: - Treat CDN assets as release-managed content artifacts. - Store object metadata in catalog/admin records with ETag, content type, size, uploader, and uploaded timestamp. - Add import/export or promotion commands for content releases. - Make cache-control and immutable URL policy explicit in release docs. - Avoid mutable object keys unless the release process invalidates caches. ### 6. Processes Partially aligned. CDN upload/list/delete work runs inside Admin. Host also has a CDN proxy endpoint. Plan: - Keep normal asset management in Admin. - Add separate admin/job processes for bulk import, orphan scanning, reference validation, and content promotion. - Keep CDN proxy concerns in Host or a dedicated edge service, not mixed into `SOR.Cdn` upload logic. ### 7. Port Binding Aligned for `SOR.Cdn` itself. The module does not bind ports. Plan: - Keep port binding in Admin and Host. - Move shared CDN proxy config and health checks into reusable registration helpers. - Add timeout, response-header, and range-request policy to the Host proxy if it remains there. ### 8. Concurrency Partially aligned. S3 operations are naturally concurrent, but key collisions and deletes need policy. Plan: - Generate collision-resistant object keys centrally. - Add optional conditional put behavior where overwrites are disallowed. - Make delete operations idempotent but audited. - Prevent deleting keys that are still referenced unless a force flag is explicitly used. - Add retry policy for transient S3/RadosGW failures. ### 9. Disposability Mostly aligned. Upload and delete methods accept cancellation tokens. Plan: - Pass cancellation tokens from all Admin upload/list/delete calls. - Configure SDK timeouts. - Avoid long-running upload work tied only to Blazor circuit lifetime for large files. - Consider direct-to-object-storage upload flows or background upload jobs if file sizes grow. ### 10. Dev/Prod Parity Partially aligned. The same client can target Ceph/S3-compatible storage, but local config examples are stale. Plan: - Provide a local MinIO or Ceph-compatible dev profile. - Keep the same bucket/prefix policy locally and in production. - Document dev/prod differences for public URL, path-style access, ACL support, and CORS. - Add a simple smoke test command for upload, head, list, and delete. ### 11. Logs Partially aligned. Upload and delete failures are logged, but audit and metrics are thin. Plan: - Add structured logs for upload, delete, list failure, rejected content type, rejected prefix, and reference-check failures. - Add metrics for upload count, upload bytes, upload latency, failures by operation, delete count, and proxy failures. - Avoid logging secret values or signed URLs. Suggested fields: - `cdnKey` - `bucket` - `contentType` - `contentLength` - `etag` - `assetType` - `adminAccountId` - `operation` - `forceDelete` ### 12. Admin Processes Partially aligned. Admin handles uploads, but bulk maintenance operations are not first-class. Plan: - Add admin/job processes for orphan scans, reference validation, asset promotion, cache audit, and bulk delete. - Record upload/delete audit events in the catalog database or a dedicated audit sink. - Add a dry-run mode for destructive maintenance. - Provide a runbook for restoring accidentally deleted assets from object-store versioning/backups if available. ## Refactor Phases ### Phase 0: Config Cleanup and Baseline Tests Goals: - Fix documentation/config drift without behavior changes. Work: - Update appsettings comments to mention `Cdn__EndpointUrl`, `Cdn__AccessKey`, `Cdn__SecretKey`, and `Cdn__BucketName`. - Add tests for public URL construction, key construction, missing config validation, and upload failure handling. - Add a small fake `ICdnClient` for Admin tests. Done when: - Config documentation matches the actual `CdnOptions`. - Existing upload behavior is covered by tests. ### Phase 1: Typed Validation Goals: - Fail fast when CDN config is missing or malformed. Work: - Add `CdnOptionsValidator`. - Validate `PublicBaseUrl`, `EndpointUrl`, `BucketName`, and credentials when CDN write/list features are active. - Add options for region, force path style, ACL, cache-control, and max upload size. - Keep current defaults where practical. Done when: - Admin cannot start with incomplete CDN config unless CDN features are explicitly disabled. - Secret values remain externalized and are never logged. ### Phase 2: Central Key and URL Policy Goals: - Stop every caller from inventing object key conventions. Work: - Add `ICdnKeyBuilder`. - Normalize path separators, reject traversal, reject unsafe characters, and preserve safe extensions. - Generate collision-resistant keys with asset type, entity ID, timestamp or content hash, and sanitized filename. - Add a single `BuildPublicUrl` implementation. Done when: - Admin pages no longer concatenate CDN URLs manually. - Upload keys are stable, safe, and predictable. ### Phase 3: Asset Upload Policy Goals: - Enforce upload rules centrally. Work: - Add `ICdnAssetPolicy`. - Validate content type, extension, prefix, and max size by asset class. - Add optional image/model/audio-specific validation hooks. - Return policy errors before contacting object storage. Done when: - Unsafe or unsupported uploads are rejected consistently. - New asset types require policy configuration, not scattered component edits. ### Phase 4: Complete CDN Client Surface Goals: - Remove raw `IAmazonS3` usage from Admin pages. Work: - Add list, head, metadata, and safe delete methods to `ICdnClient`. - Add paged list responses with continuation tokens. - Add object metadata DTOs with key, size, ETag, last modified, content type, and public URL. - Refactor `CdnBrowser`, `CardEditor`, and `TileModelManagerPage` to use the CDN abstraction. Done when: - `IAmazonS3` is only used inside `SOR.Cdn`. - Admin object browsing and deletion share one behavior path. ### Phase 5: Reference-Aware Delete and Audit Goals: - Prevent accidental content breakage. Work: - Add `ICdnReferenceScanner` that checks catalog, bazaar, identity, and content records for a key or URL. - Add delete modes: soft/unreferenced-only, force, dry-run. - Record upload and delete audit events. - Surface reference reports in Admin before destructive actions. Done when: - Operators can see what a CDN object is used by before deletion. - Destructive operations are auditable and reversible where object storage supports it. ### Phase 6: Readiness and Resilience Goals: - Make CDN dependency health visible. Work: - Add a health check that can verify bucket access with a cheap head/list operation. - Configure SDK timeouts and retry policy. - Add metrics for operation latency, errors, and bytes uploaded. - Add Host CDN proxy timeout and error metrics. Done when: - Admin readiness accounts for CDN availability when asset management is enabled. - CDN failures are visible before users report broken asset workflows. ### Phase 7: Bulk Maintenance and Release Workflows Goals: - Treat assets as managed content artifacts. Work: - Add jobs for orphan scanning, reference validation, bulk import, and content promotion. - Add dry-run outputs for all destructive jobs. - Document cache policy and immutable key expectations. - Define backup/restore expectations for the bucket. Done when: - Content releases can validate their object-store dependencies. - Operators can clean up unused assets safely. ## Suggested Work Order 1. Fix CDN config documentation drift. 2. Add `CdnOptionsValidator`. 3. Add key builder and public URL builder. 4. Add asset upload policy. 5. Expand `ICdnClient` with list/head/safe delete. 6. Refactor Admin pages away from direct `IAmazonS3`. 7. Add reference-aware delete and audit events. 8. Add CDN health checks, metrics, and retry/timeout policy. 9. Add bulk maintenance jobs and runbooks. ## Completion Criteria The refactor is complete when: - CDN config is typed, validated, and accurately documented. - `SOR.Cdn` is the only module that uses S3 SDK types directly. - Upload keys and public URLs are generated centrally. - Uploads are validated by asset policy before storage writes. - Deletes are reference-aware, auditable, and support dry-run behavior. - Admin CDN browsing/listing uses the same abstraction as upload/delete. - CDN readiness checks account for bucket access and credentials. - Asset promotion, orphan cleanup, and reference validation exist as explicit admin processes. ## Immediate Low-Risk Next Steps - Update `SOR.Admin` and `SOR.Host` appsettings comments to match current `CdnOptions`. - Add startup validation for `PublicBaseUrl`, `EndpointUrl`, `AccessKey`, `SecretKey`, and `BucketName`. - Add a central `BuildPublicUrl` helper to stop duplicating URL construction in Admin pages. - Add a central key sanitizer before changing upload behavior. - Document current CDN assumptions: Ceph RadosGW, path-style access, public-read objects, immutable cache headers, and one configured bucket.