Files
sor/SOR.Content.12FactorPlan.md
2026-07-02 13:09:05 -04:00

445 lines
16 KiB
Markdown

# SOR.Content 12-Factor Refactor Plan
Date: 2026-06-10
Scope reviewed:
- `Server/src/SOR.Content`
- Catalog persistence in `Server/src/SOR.Database/Contexts/CatalogDbContext.cs`
- Board-template reads from `Server/src/SOR.Database/Contexts/GameDbContext.cs`
- Content wiring in `Server/src/SOR.Host`
- CDN-backed asset references stored in catalog data
Reference: https://12factor.net/
## Executive Summary
`SOR.Content` is a read-heavy service module registered by `SOR.Host` when `ENABLED_SERVICES` includes `content`. It serves card-definition sync, card sets, single card definitions, tile model bindings, tile bottom models, skybox config, highlight config, board templates, and game audio config. Most data comes from the catalog schema; board templates come from the game schema.
The service already has useful 12-Factor foundations:
- It is enabled through host process formation.
- It is stateless from the process perspective.
- It stores durable content in PostgreSQL rather than local files.
- Card sync is cursor-based through `DefinitionVersion`.
- Content assets are referenced by CDN URLs/keys rather than stored in the service.
- Handlers are small compared with mutation-heavy modules.
- The service has no hosted worker and no local filesystem state.
The main 12-Factor gaps are around operational hardening and content-release boundaries:
- Handlers directly query EF contexts and map DTOs inline or through a static mapper.
- Sync page-size limits are hardcoded.
- There is no typed `ContentOptions` for sync limits, cache TTLs, publication behavior, or payload limits.
- Content publication rules are not explicit: some endpoints filter active/enabled rows, while card sync/card lookup expose published/retired flags without a clear client-visible contract.
- There is no visible cache strategy for high-frequency read endpoints.
- CDN asset references are returned without readiness/reference validation.
- Large payload endpoints, especially card sync and board templates, lack explicit size, compression, and pagination strategy.
- Content and game board-template reads are registered together, but dependency/readiness checks do not distinguish catalog and game schema requirements.
- There is no visible test project covering sync cursor behavior or DTO compatibility.
The plan is to keep `SOR.Content` as a read-only content delivery module, but split direct handlers into query/read-model services, add typed config and readiness, define a publication/sync contract, add cache support, and create admin/release validation jobs for CDN-backed content.
## Current Structure Observations
### Strong Foundations
- `AddContentService` registers all content handlers from one place.
- `ContentSyncHandler` pages card definitions by `DefinitionVersion`.
- Sync fetches one extra row to calculate `HasMore` without a count query.
- Sync uses `AsSplitQuery` to avoid cartesian explosion across assets and animation bindings.
- Several endpoints use `AsNoTracking` for read-only queries.
- Catalog schema has indexes for published cards, sets, definition version, asset bindings, tile models, skyboxes, highlight configs, and audio configs.
- DTO contracts are centralized in `ContentMessages.cs`.
- `ContentDtoMapper` centralizes card definition DTO mapping.
- CDN cache-busting is represented through `ContentETag` on card assets.
### Main Problems
- `ContentSyncHandler` clamps page size to `500` directly.
- `SyncCardDefinitionsRequest` default page size is `200`, but limits are not configurable.
- `ContentGetCardDefinitionHandler` returns any card by ID, without an explicit published/retired visibility decision.
- `ContentSyncHandler` syncs all cards above the version watermark, without an explicit published-only or include-retired contract.
- `ContentGetHighlightConfigHandler` returns all rows and comments say this helps admin visibility, but the endpoint is registered as a client content endpoint.
- Tile model bindings do not filter by an active/published concept.
- Board templates include full `TilesJson` and `StartingZonesJson` with no payload-size or pagination policy.
- Readiness is host-level only; content needs CatalogDbContext, and board templates need GameDbContext.
- There is no local cache, Redis cache, ETag/manifest response, or content snapshot/read-model layer.
- CDN-backed URLs are returned even if the objects are missing or invalid.
- Content contracts are broad and likely coupled tightly to client expectations.
## Target Content Shape
Keep `SOR.Content` as the read-only content service module, but split it into focused areas:
```text
Server/src/SOR.Content/
Configuration/
ContentOptions.cs
ContentSyncOptions.cs
ContentCacheOptions.cs
ContentPublicationOptions.cs
ContentOptionsValidator.cs
Bootstrap/
ContentRegistration.cs
ContentHandlersRegistration.cs
ContentHealthChecks.cs
Contracts/
ContentMessages.cs
ContentErrors.cs
ContentManifest.cs
Application/
Cards/
CardContentQueryService.cs
CardSyncService.cs
CardSetQueryService.cs
CardDefinitionMapper.cs
World/
TileModelContentService.cs
SkyboxContentService.cs
HighlightContentService.cs
BoardTemplateContentService.cs
Audio/
GameAudioContentService.cs
Validation/
ContentReferenceValidator.cs
ContentPublicationValidator.cs
Persistence/
ICatalogContentReader.cs
IGameContentReader.cs
IContentVersionReader.cs
Caching/
IContentCache.cs
ContentCacheKeys.cs
ContentSnapshotBuilder.cs
Jobs/
ValidateContentReferencesJob.cs
RebuildContentSnapshotsJob.cs
```
Desired handler shape:
- Deserialize request.
- Call one query service.
- Return a stable content DTO or error.
- Avoid direct EF queries and ad hoc filtering in handlers.
## 12-Factor Gap Map
### 1. Codebase
Mostly aligned. `SOR.Content` is one module, but content query concerns are spread across handlers.
Plan:
- Keep one content module.
- Extract card, world, audio, and board-template query services.
- Keep DTO contracts stable while introducing read-model services underneath.
- Add tests for sync and DTO compatibility.
### 2. Dependencies
Mostly aligned. Dependencies are explicit through `SOR.Database` and networking abstractions.
Plan:
- Keep EF Core access isolated behind readers/query services.
- Keep Catalog and Game database dependencies explicit.
- Treat CDN as an indirect dependency through stored asset references and validation jobs.
- Avoid adding provider-specific CDN dependencies to this read service.
### 3. Config
Weakly aligned. Sync limits and publication behavior are hardcoded or implicit.
Plan:
- Add typed `ContentOptions`.
- Configure sync defaults, max page sizes, cache TTLs, publication scope, payload limits, and optional board-template limits.
- Validate options at startup.
Candidate config shape:
```json
{
"Content": {
"Sync": {
"DefaultPageSize": 200,
"MaxPageSize": 500,
"IncludeUnpublished": false,
"IncludeRetiredTombstones": true
},
"Cache": {
"Enabled": true,
"CardSetsTtlSeconds": 300,
"WorldConfigTtlSeconds": 300,
"BoardTemplatesTtlSeconds": 300
},
"Payloads": {
"MaxBoardTemplates": 200,
"MaxBoardTemplateJsonBytes": 1048576
}
}
}
```
### 4. Backing Services
Partially aligned. Content relies on PostgreSQL catalog and game schemas, and indirectly on CDN object availability.
Plan:
- Treat CatalogDbContext and GameDbContext as attached backing services.
- Add readiness checks per enabled endpoint group.
- Add release validation for CDN object references.
- Document which content endpoints can remain healthy if GameDbContext is unavailable.
### 5. Build, Release, Run
Partially aligned. Content data is external to the app build, but content release validation is not formalized.
Plan:
- Treat card definitions, world config, audio config, board templates, and CDN references as release-managed content.
- Add validation jobs before publishing content.
- Add content manifest/version reporting so clients can reason about what they have cached.
- Keep content migrations outside normal runtime startup.
### 6. Processes
Mostly aligned. Content is a stateless request/response read service.
Plan:
- Keep content serving in the `content` process role.
- Add optional `content-jobs` role for validation, snapshot generation, and cache warmup.
- Do not run content validation jobs in normal request workers unless explicitly enabled.
### 7. Port Binding
Aligned through `SOR.Host`. Content does not bind ports directly.
Plan:
- Keep port binding in Host/Gateway.
- Expose content readiness through host health checks.
- Do not add a content-specific listener.
### 8. Concurrency
Mostly aligned because Content is read-only. The main risk is consistency across admin writes and client sync reads.
Plan:
- Define publication semantics for partial admin edits.
- Use versioned publication transactions for cards/assets if clients require atomic content snapshots.
- Ensure sync cursor behavior handles updates, retirements, and unpublished drafts intentionally.
- Add tests for concurrent update/sync behavior.
### 9. Disposability
Mostly aligned. Handlers are short and pass cancellation tokens through EF operations.
Plan:
- Continue cancellation-aware reads.
- Add bounded cache/snapshot rebuild jobs if introduced.
- Avoid request-time expensive validation against CDN.
### 10. Dev/Prod Parity
Partially aligned. The same DB-backed content model runs everywhere, but dev content and CDN references can drift.
Plan:
- Provide local seed content for card sets, cards, world config, audio, and board templates.
- Provide a local CDN/reference validation profile.
- Keep sync/version behavior identical across environments.
### 11. Logs
Weakly aligned. Content handlers do not visibly emit structured operational metrics.
Plan:
- Add metrics for sync requests, rows returned, payload size, cache hit/miss, query latency, stale cache usage, and validation failures.
- Log content validation failures with stable content IDs and keys, not full payloads.
- Add correlation fields for content version and endpoint.
Suggested fields:
- `endpoint`
- `sinceVersion`
- `currentVersion`
- `pageSize`
- `rowsReturned`
- `hasMore`
- `cacheHit`
- `contentId`
- `contentType`
- `assetUrl`
### 12. Admin Processes
Partially aligned. Content is authored through Admin, but validation/publish processes are not part of `SOR.Content`.
Plan:
- Add admin/job processes for content reference validation, publication validation, snapshot rebuilds, and cache warmup.
- Add dry-run validation reports for cards, animations, tile models, skyboxes, highlights, audio, and board templates.
- Integrate validation results into Admin before publishing or enabling content.
## Refactor Phases
### Phase 0: Baseline Tests
Goals:
- Preserve current response shapes and sync behavior.
Work:
- Add tests for `content.sync` initial sync, pagination, `HasMore`, and `CurrentVersion`.
- Add tests for card DTO mapping with assets and animation bindings.
- Add tests for card set counts.
- Add tests for tile models, tile bottoms, skybox, highlight, audio, and board-template response filters.
Done when:
- Existing content contracts are covered before extraction.
### Phase 1: Typed Options
Goals:
- Make sync and content delivery behavior configurable.
Work:
- Add `ContentOptions`, `ContentSyncOptions`, `ContentCacheOptions`, and `ContentPublicationOptions`.
- Replace hardcoded page-size limits with options.
- Add startup validation.
- Keep defaults matching current behavior until publication rules are deliberately changed.
Done when:
- Content limits and cache settings are externalized.
- Invalid content config fails startup for active content roles.
### Phase 2: Query Services and Thin Handlers
Goals:
- Make handlers transport adapters.
Work:
- Extract `CardSyncService`, `CardContentQueryService`, `CardSetQueryService`, `WorldContentService`, `AudioContentService`, and `BoardTemplateContentService`.
- Move DTO mapping into injectable or testable mappers.
- Keep message contracts unchanged.
Done when:
- Handlers contain minimal request/response plumbing.
- Query behavior is independently testable.
### Phase 3: Publication Contract
Goals:
- Define exactly what clients should receive.
Work:
- Decide whether `content.sync` returns:
- published cards only
- retired tombstones
- unpublished draft cards
- admin-only records
- Decide whether `content.card` can return unpublished content.
- Decide whether highlight config should expose partially configured rows to clients.
- Encode the decision in options, services, and tests.
Done when:
- Client-visible content is not an accidental byproduct of raw table state.
- Sync/update/retire behavior is documented and tested.
### Phase 4: Cache and Manifest Strategy
Goals:
- Reduce repeated database load and give clients stable cache cues.
Work:
- Add in-memory or Redis-backed caching for low-churn endpoints.
- Add a content manifest with current card version, asset/config versions, and optional hashes.
- Add cache invalidation hooks or TTLs.
- Consider precomputed snapshots for full card sync if payload size grows.
Done when:
- High-frequency content endpoints have predictable DB load.
- Clients can identify when their cached content is current.
### Phase 5: Content Validation Jobs
Goals:
- Catch broken content before clients download it.
Work:
- Add validation for card JSON fields, ability payloads, animation bindings, CDN URLs, tile model bindings, skyboxes, highlights, audio config, and board-template JSON.
- Add dry-run reports.
- Add a release/admin command for validation.
Done when:
- Bad content can be detected before publish/enable.
- Admin can see validation failures with actionable IDs.
### Phase 6: CDN Reference Policy
Goals:
- Make asset references reliable without coupling runtime reads to CDN checks.
Work:
- Add `ContentReferenceValidator` that checks stored CDN URLs/keys through CDN tooling or HTTP HEAD/job process.
- Store validation status or reports where Admin can inspect them.
- Decide whether invalid references block publication.
- Keep runtime content reads fast and avoid request-time CDN validation.
Done when:
- Missing assets are caught by validation jobs, not by players.
### Phase 7: Payload and Version Hardening
Goals:
- Keep content sync scalable as catalog size grows.
Work:
- Measure sync payload sizes.
- Add response-size metrics.
- Consider splitting cards, assets, animations, and world config into separate manifests.
- Add board-template pagination or versioning if needed.
- Add compatibility versioning for DTO contracts.
Done when:
- Content payload growth has limits, metrics, and a migration path.
### Phase 8: Readiness and Operations
Goals:
- Make content delivery diagnosable in production.
Work:
- Add readiness checks for CatalogDbContext and GameDbContext.
- Add metrics for query latency, cache hit rate, sync pages, payload sizes, validation status, and manifest version.
- Add runbooks for stale content, broken assets, failed validation, and sync incompatibility.
Done when:
- Operators can answer what content version is live and whether clients can sync it safely.
## Suggested Work Order
1. Add baseline tests for sync and DTO mapping.
2. Add typed `ContentOptions` with current defaults.
3. Extract query services behind existing handlers.
4. Define and test publication visibility rules.
5. Add cache/manifest support for stable content responses.
6. Add content validation jobs and dry-run reports.
7. Add CDN reference validation jobs.
8. Add payload metrics, readiness checks, and runbooks.
## Completion Criteria
The refactor is complete when:
- Content handlers are thin transport adapters.
- Sync limits, publication rules, cache TTLs, and payload limits are typed config.
- Client-visible publication behavior is explicit and tested.
- Content queries can be served from read-model services or cache.
- CDN references and JSON payloads are validated before publish.
- Readiness checks distinguish Catalog and Game dependencies.
- Metrics show sync latency, payload size, cache behavior, and validation health.
- Admin/release workflows can validate content before it becomes client-visible.
## Immediate Low-Risk Next Steps
- Add `ContentOptions` with current sync defaults.
- Add tests for `ContentSyncHandler` cursor pagination.
- Extract `CardSyncService` without changing behavior.
- Document current publication behavior for `content.sync`, `content.card`, and highlight config.
- Add a validation report for missing CDN URLs/keys across catalog content.