445 lines
16 KiB
Markdown
445 lines
16 KiB
Markdown
# SOR.Content 12-Factor Refactor Plan
|
|
|
|
Date: 2026-06-10
|
|
|
|
Scope reviewed:
|
|
- `Server/src/SOR.Content`
|
|
- Catalog persistence in `Server/src/SOR.Database/Contexts/CatalogDbContext.cs`
|
|
- Board-template reads from `Server/src/SOR.Database/Contexts/GameDbContext.cs`
|
|
- Content wiring in `Server/src/SOR.Host`
|
|
- CDN-backed asset references stored in catalog data
|
|
|
|
Reference: https://12factor.net/
|
|
|
|
## Executive Summary
|
|
|
|
`SOR.Content` is a read-heavy service module registered by `SOR.Host` when `ENABLED_SERVICES` includes `content`. It serves card-definition sync, card sets, single card definitions, tile model bindings, tile bottom models, skybox config, highlight config, board templates, and game audio config. Most data comes from the catalog schema; board templates come from the game schema.
|
|
|
|
The service already has useful 12-Factor foundations:
|
|
|
|
- It is enabled through host process formation.
|
|
- It is stateless from the process perspective.
|
|
- It stores durable content in PostgreSQL rather than local files.
|
|
- Card sync is cursor-based through `DefinitionVersion`.
|
|
- Content assets are referenced by CDN URLs/keys rather than stored in the service.
|
|
- Handlers are small compared with mutation-heavy modules.
|
|
- The service has no hosted worker and no local filesystem state.
|
|
|
|
The main 12-Factor gaps are around operational hardening and content-release boundaries:
|
|
|
|
- Handlers directly query EF contexts and map DTOs inline or through a static mapper.
|
|
- Sync page-size limits are hardcoded.
|
|
- There is no typed `ContentOptions` for sync limits, cache TTLs, publication behavior, or payload limits.
|
|
- Content publication rules are not explicit: some endpoints filter active/enabled rows, while card sync/card lookup expose published/retired flags without a clear client-visible contract.
|
|
- There is no visible cache strategy for high-frequency read endpoints.
|
|
- CDN asset references are returned without readiness/reference validation.
|
|
- Large payload endpoints, especially card sync and board templates, lack explicit size, compression, and pagination strategy.
|
|
- Content and game board-template reads are registered together, but dependency/readiness checks do not distinguish catalog and game schema requirements.
|
|
- There is no visible test project covering sync cursor behavior or DTO compatibility.
|
|
|
|
The plan is to keep `SOR.Content` as a read-only content delivery module, but split direct handlers into query/read-model services, add typed config and readiness, define a publication/sync contract, add cache support, and create admin/release validation jobs for CDN-backed content.
|
|
|
|
## Current Structure Observations
|
|
|
|
### Strong Foundations
|
|
|
|
- `AddContentService` registers all content handlers from one place.
|
|
- `ContentSyncHandler` pages card definitions by `DefinitionVersion`.
|
|
- Sync fetches one extra row to calculate `HasMore` without a count query.
|
|
- Sync uses `AsSplitQuery` to avoid cartesian explosion across assets and animation bindings.
|
|
- Several endpoints use `AsNoTracking` for read-only queries.
|
|
- Catalog schema has indexes for published cards, sets, definition version, asset bindings, tile models, skyboxes, highlight configs, and audio configs.
|
|
- DTO contracts are centralized in `ContentMessages.cs`.
|
|
- `ContentDtoMapper` centralizes card definition DTO mapping.
|
|
- CDN cache-busting is represented through `ContentETag` on card assets.
|
|
|
|
### Main Problems
|
|
|
|
- `ContentSyncHandler` clamps page size to `500` directly.
|
|
- `SyncCardDefinitionsRequest` default page size is `200`, but limits are not configurable.
|
|
- `ContentGetCardDefinitionHandler` returns any card by ID, without an explicit published/retired visibility decision.
|
|
- `ContentSyncHandler` syncs all cards above the version watermark, without an explicit published-only or include-retired contract.
|
|
- `ContentGetHighlightConfigHandler` returns all rows and comments say this helps admin visibility, but the endpoint is registered as a client content endpoint.
|
|
- Tile model bindings do not filter by an active/published concept.
|
|
- Board templates include full `TilesJson` and `StartingZonesJson` with no payload-size or pagination policy.
|
|
- Readiness is host-level only; content needs CatalogDbContext, and board templates need GameDbContext.
|
|
- There is no local cache, Redis cache, ETag/manifest response, or content snapshot/read-model layer.
|
|
- CDN-backed URLs are returned even if the objects are missing or invalid.
|
|
- Content contracts are broad and likely coupled tightly to client expectations.
|
|
|
|
## Target Content Shape
|
|
|
|
Keep `SOR.Content` as the read-only content service module, but split it into focused areas:
|
|
|
|
```text
|
|
Server/src/SOR.Content/
|
|
Configuration/
|
|
ContentOptions.cs
|
|
ContentSyncOptions.cs
|
|
ContentCacheOptions.cs
|
|
ContentPublicationOptions.cs
|
|
ContentOptionsValidator.cs
|
|
Bootstrap/
|
|
ContentRegistration.cs
|
|
ContentHandlersRegistration.cs
|
|
ContentHealthChecks.cs
|
|
Contracts/
|
|
ContentMessages.cs
|
|
ContentErrors.cs
|
|
ContentManifest.cs
|
|
Application/
|
|
Cards/
|
|
CardContentQueryService.cs
|
|
CardSyncService.cs
|
|
CardSetQueryService.cs
|
|
CardDefinitionMapper.cs
|
|
World/
|
|
TileModelContentService.cs
|
|
SkyboxContentService.cs
|
|
HighlightContentService.cs
|
|
BoardTemplateContentService.cs
|
|
Audio/
|
|
GameAudioContentService.cs
|
|
Validation/
|
|
ContentReferenceValidator.cs
|
|
ContentPublicationValidator.cs
|
|
Persistence/
|
|
ICatalogContentReader.cs
|
|
IGameContentReader.cs
|
|
IContentVersionReader.cs
|
|
Caching/
|
|
IContentCache.cs
|
|
ContentCacheKeys.cs
|
|
ContentSnapshotBuilder.cs
|
|
Jobs/
|
|
ValidateContentReferencesJob.cs
|
|
RebuildContentSnapshotsJob.cs
|
|
```
|
|
|
|
Desired handler shape:
|
|
|
|
- Deserialize request.
|
|
- Call one query service.
|
|
- Return a stable content DTO or error.
|
|
- Avoid direct EF queries and ad hoc filtering in handlers.
|
|
|
|
## 12-Factor Gap Map
|
|
|
|
### 1. Codebase
|
|
|
|
Mostly aligned. `SOR.Content` is one module, but content query concerns are spread across handlers.
|
|
|
|
Plan:
|
|
- Keep one content module.
|
|
- Extract card, world, audio, and board-template query services.
|
|
- Keep DTO contracts stable while introducing read-model services underneath.
|
|
- Add tests for sync and DTO compatibility.
|
|
|
|
### 2. Dependencies
|
|
|
|
Mostly aligned. Dependencies are explicit through `SOR.Database` and networking abstractions.
|
|
|
|
Plan:
|
|
- Keep EF Core access isolated behind readers/query services.
|
|
- Keep Catalog and Game database dependencies explicit.
|
|
- Treat CDN as an indirect dependency through stored asset references and validation jobs.
|
|
- Avoid adding provider-specific CDN dependencies to this read service.
|
|
|
|
### 3. Config
|
|
|
|
Weakly aligned. Sync limits and publication behavior are hardcoded or implicit.
|
|
|
|
Plan:
|
|
- Add typed `ContentOptions`.
|
|
- Configure sync defaults, max page sizes, cache TTLs, publication scope, payload limits, and optional board-template limits.
|
|
- Validate options at startup.
|
|
|
|
Candidate config shape:
|
|
|
|
```json
|
|
{
|
|
"Content": {
|
|
"Sync": {
|
|
"DefaultPageSize": 200,
|
|
"MaxPageSize": 500,
|
|
"IncludeUnpublished": false,
|
|
"IncludeRetiredTombstones": true
|
|
},
|
|
"Cache": {
|
|
"Enabled": true,
|
|
"CardSetsTtlSeconds": 300,
|
|
"WorldConfigTtlSeconds": 300,
|
|
"BoardTemplatesTtlSeconds": 300
|
|
},
|
|
"Payloads": {
|
|
"MaxBoardTemplates": 200,
|
|
"MaxBoardTemplateJsonBytes": 1048576
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 4. Backing Services
|
|
|
|
Partially aligned. Content relies on PostgreSQL catalog and game schemas, and indirectly on CDN object availability.
|
|
|
|
Plan:
|
|
- Treat CatalogDbContext and GameDbContext as attached backing services.
|
|
- Add readiness checks per enabled endpoint group.
|
|
- Add release validation for CDN object references.
|
|
- Document which content endpoints can remain healthy if GameDbContext is unavailable.
|
|
|
|
### 5. Build, Release, Run
|
|
|
|
Partially aligned. Content data is external to the app build, but content release validation is not formalized.
|
|
|
|
Plan:
|
|
- Treat card definitions, world config, audio config, board templates, and CDN references as release-managed content.
|
|
- Add validation jobs before publishing content.
|
|
- Add content manifest/version reporting so clients can reason about what they have cached.
|
|
- Keep content migrations outside normal runtime startup.
|
|
|
|
### 6. Processes
|
|
|
|
Mostly aligned. Content is a stateless request/response read service.
|
|
|
|
Plan:
|
|
- Keep content serving in the `content` process role.
|
|
- Add optional `content-jobs` role for validation, snapshot generation, and cache warmup.
|
|
- Do not run content validation jobs in normal request workers unless explicitly enabled.
|
|
|
|
### 7. Port Binding
|
|
|
|
Aligned through `SOR.Host`. Content does not bind ports directly.
|
|
|
|
Plan:
|
|
- Keep port binding in Host/Gateway.
|
|
- Expose content readiness through host health checks.
|
|
- Do not add a content-specific listener.
|
|
|
|
### 8. Concurrency
|
|
|
|
Mostly aligned because Content is read-only. The main risk is consistency across admin writes and client sync reads.
|
|
|
|
Plan:
|
|
- Define publication semantics for partial admin edits.
|
|
- Use versioned publication transactions for cards/assets if clients require atomic content snapshots.
|
|
- Ensure sync cursor behavior handles updates, retirements, and unpublished drafts intentionally.
|
|
- Add tests for concurrent update/sync behavior.
|
|
|
|
### 9. Disposability
|
|
|
|
Mostly aligned. Handlers are short and pass cancellation tokens through EF operations.
|
|
|
|
Plan:
|
|
- Continue cancellation-aware reads.
|
|
- Add bounded cache/snapshot rebuild jobs if introduced.
|
|
- Avoid request-time expensive validation against CDN.
|
|
|
|
### 10. Dev/Prod Parity
|
|
|
|
Partially aligned. The same DB-backed content model runs everywhere, but dev content and CDN references can drift.
|
|
|
|
Plan:
|
|
- Provide local seed content for card sets, cards, world config, audio, and board templates.
|
|
- Provide a local CDN/reference validation profile.
|
|
- Keep sync/version behavior identical across environments.
|
|
|
|
### 11. Logs
|
|
|
|
Weakly aligned. Content handlers do not visibly emit structured operational metrics.
|
|
|
|
Plan:
|
|
- Add metrics for sync requests, rows returned, payload size, cache hit/miss, query latency, stale cache usage, and validation failures.
|
|
- Log content validation failures with stable content IDs and keys, not full payloads.
|
|
- Add correlation fields for content version and endpoint.
|
|
|
|
Suggested fields:
|
|
|
|
- `endpoint`
|
|
- `sinceVersion`
|
|
- `currentVersion`
|
|
- `pageSize`
|
|
- `rowsReturned`
|
|
- `hasMore`
|
|
- `cacheHit`
|
|
- `contentId`
|
|
- `contentType`
|
|
- `assetUrl`
|
|
|
|
### 12. Admin Processes
|
|
|
|
Partially aligned. Content is authored through Admin, but validation/publish processes are not part of `SOR.Content`.
|
|
|
|
Plan:
|
|
- Add admin/job processes for content reference validation, publication validation, snapshot rebuilds, and cache warmup.
|
|
- Add dry-run validation reports for cards, animations, tile models, skyboxes, highlights, audio, and board templates.
|
|
- Integrate validation results into Admin before publishing or enabling content.
|
|
|
|
## Refactor Phases
|
|
|
|
### Phase 0: Baseline Tests
|
|
|
|
Goals:
|
|
- Preserve current response shapes and sync behavior.
|
|
|
|
Work:
|
|
- Add tests for `content.sync` initial sync, pagination, `HasMore`, and `CurrentVersion`.
|
|
- Add tests for card DTO mapping with assets and animation bindings.
|
|
- Add tests for card set counts.
|
|
- Add tests for tile models, tile bottoms, skybox, highlight, audio, and board-template response filters.
|
|
|
|
Done when:
|
|
- Existing content contracts are covered before extraction.
|
|
|
|
### Phase 1: Typed Options
|
|
|
|
Goals:
|
|
- Make sync and content delivery behavior configurable.
|
|
|
|
Work:
|
|
- Add `ContentOptions`, `ContentSyncOptions`, `ContentCacheOptions`, and `ContentPublicationOptions`.
|
|
- Replace hardcoded page-size limits with options.
|
|
- Add startup validation.
|
|
- Keep defaults matching current behavior until publication rules are deliberately changed.
|
|
|
|
Done when:
|
|
- Content limits and cache settings are externalized.
|
|
- Invalid content config fails startup for active content roles.
|
|
|
|
### Phase 2: Query Services and Thin Handlers
|
|
|
|
Goals:
|
|
- Make handlers transport adapters.
|
|
|
|
Work:
|
|
- Extract `CardSyncService`, `CardContentQueryService`, `CardSetQueryService`, `WorldContentService`, `AudioContentService`, and `BoardTemplateContentService`.
|
|
- Move DTO mapping into injectable or testable mappers.
|
|
- Keep message contracts unchanged.
|
|
|
|
Done when:
|
|
- Handlers contain minimal request/response plumbing.
|
|
- Query behavior is independently testable.
|
|
|
|
### Phase 3: Publication Contract
|
|
|
|
Goals:
|
|
- Define exactly what clients should receive.
|
|
|
|
Work:
|
|
- Decide whether `content.sync` returns:
|
|
- published cards only
|
|
- retired tombstones
|
|
- unpublished draft cards
|
|
- admin-only records
|
|
- Decide whether `content.card` can return unpublished content.
|
|
- Decide whether highlight config should expose partially configured rows to clients.
|
|
- Encode the decision in options, services, and tests.
|
|
|
|
Done when:
|
|
- Client-visible content is not an accidental byproduct of raw table state.
|
|
- Sync/update/retire behavior is documented and tested.
|
|
|
|
### Phase 4: Cache and Manifest Strategy
|
|
|
|
Goals:
|
|
- Reduce repeated database load and give clients stable cache cues.
|
|
|
|
Work:
|
|
- Add in-memory or Redis-backed caching for low-churn endpoints.
|
|
- Add a content manifest with current card version, asset/config versions, and optional hashes.
|
|
- Add cache invalidation hooks or TTLs.
|
|
- Consider precomputed snapshots for full card sync if payload size grows.
|
|
|
|
Done when:
|
|
- High-frequency content endpoints have predictable DB load.
|
|
- Clients can identify when their cached content is current.
|
|
|
|
### Phase 5: Content Validation Jobs
|
|
|
|
Goals:
|
|
- Catch broken content before clients download it.
|
|
|
|
Work:
|
|
- Add validation for card JSON fields, ability payloads, animation bindings, CDN URLs, tile model bindings, skyboxes, highlights, audio config, and board-template JSON.
|
|
- Add dry-run reports.
|
|
- Add a release/admin command for validation.
|
|
|
|
Done when:
|
|
- Bad content can be detected before publish/enable.
|
|
- Admin can see validation failures with actionable IDs.
|
|
|
|
### Phase 6: CDN Reference Policy
|
|
|
|
Goals:
|
|
- Make asset references reliable without coupling runtime reads to CDN checks.
|
|
|
|
Work:
|
|
- Add `ContentReferenceValidator` that checks stored CDN URLs/keys through CDN tooling or HTTP HEAD/job process.
|
|
- Store validation status or reports where Admin can inspect them.
|
|
- Decide whether invalid references block publication.
|
|
- Keep runtime content reads fast and avoid request-time CDN validation.
|
|
|
|
Done when:
|
|
- Missing assets are caught by validation jobs, not by players.
|
|
|
|
### Phase 7: Payload and Version Hardening
|
|
|
|
Goals:
|
|
- Keep content sync scalable as catalog size grows.
|
|
|
|
Work:
|
|
- Measure sync payload sizes.
|
|
- Add response-size metrics.
|
|
- Consider splitting cards, assets, animations, and world config into separate manifests.
|
|
- Add board-template pagination or versioning if needed.
|
|
- Add compatibility versioning for DTO contracts.
|
|
|
|
Done when:
|
|
- Content payload growth has limits, metrics, and a migration path.
|
|
|
|
### Phase 8: Readiness and Operations
|
|
|
|
Goals:
|
|
- Make content delivery diagnosable in production.
|
|
|
|
Work:
|
|
- Add readiness checks for CatalogDbContext and GameDbContext.
|
|
- Add metrics for query latency, cache hit rate, sync pages, payload sizes, validation status, and manifest version.
|
|
- Add runbooks for stale content, broken assets, failed validation, and sync incompatibility.
|
|
|
|
Done when:
|
|
- Operators can answer what content version is live and whether clients can sync it safely.
|
|
|
|
## Suggested Work Order
|
|
|
|
1. Add baseline tests for sync and DTO mapping.
|
|
2. Add typed `ContentOptions` with current defaults.
|
|
3. Extract query services behind existing handlers.
|
|
4. Define and test publication visibility rules.
|
|
5. Add cache/manifest support for stable content responses.
|
|
6. Add content validation jobs and dry-run reports.
|
|
7. Add CDN reference validation jobs.
|
|
8. Add payload metrics, readiness checks, and runbooks.
|
|
|
|
## Completion Criteria
|
|
|
|
The refactor is complete when:
|
|
|
|
- Content handlers are thin transport adapters.
|
|
- Sync limits, publication rules, cache TTLs, and payload limits are typed config.
|
|
- Client-visible publication behavior is explicit and tested.
|
|
- Content queries can be served from read-model services or cache.
|
|
- CDN references and JSON payloads are validated before publish.
|
|
- Readiness checks distinguish Catalog and Game dependencies.
|
|
- Metrics show sync latency, payload size, cache behavior, and validation health.
|
|
- Admin/release workflows can validate content before it becomes client-visible.
|
|
|
|
## Immediate Low-Risk Next Steps
|
|
|
|
- Add `ContentOptions` with current sync defaults.
|
|
- Add tests for `ContentSyncHandler` cursor pagination.
|
|
- Extract `CardSyncService` without changing behavior.
|
|
- Document current publication behavior for `content.sync`, `content.card`, and highlight config.
|
|
- Add a validation report for missing CDN URLs/keys across catalog content.
|