17 KiB
SOR.Networking 12-Factor Refactor Plan
Date: 2026-06-10
Scope reviewed:
Server/src/SOR.Networking.AbstractionsServer/src/SOR.Networking.WebSocketServer/src/SOR.Networking.NatsServer/src/SOR.Networking.Redis- Networking wiring in
Server/src/SOR.Host - Gateway-facing usage in
Server/src/SOR.Gateway - Shared subject/channel contracts in
Server/src/SOR.Shared
Reference: https://12factor.net/
Executive Summary
SOR.Networking is the shared transport and messaging layer for the server. It defines the abstractions used by service modules and supplies concrete implementations for WebSocket client transport, NATS event/RPC messaging, and Redis Pub/Sub client push.
The module already has strong 12-factor foundations:
- Application code depends on abstractions:
ITransport,IClientConnection,IEventBus, andIClientPush. - Concrete providers are isolated in separate projects: WebSocket, NATS, and Redis.
SOR.Hostis the composition root that wires concrete infrastructure to abstractions.- Redis and NATS are treated as attached backing services.
- WebSocket, NATS, and Redis lifetimes are mostly singleton/process-scoped.
- Event bus delivery semantics are documented as core NATS at-most-once.
- Redis push delivery semantics are documented as best-effort.
- NATS RPC responders use queue groups for handler request distribution.
The main 12-factor gaps are operational rather than architectural:
- WebSocket transport uses an unbounded accept queue, so overload can become memory pressure.
- WebSocket options are bound but not validated at startup.
- There is no explicit max connection, max pending accept, max send size, or outbound backpressure policy.
- NATS URL and Redis connection are configured in
SOR.Host, but dependency health/readiness is still host-level and minimal. - NATS fan-out subscriptions use no queue group by default, which is correct for broadcast but risky if consumers accidentally scale a singleton-style worker.
- NATS RPC subject and queue naming is duplicated with Gateway dispatch logic instead of centralized in shared contracts.
- Redis push ignores subscriber count by design, but metrics do not currently expose dropped/offline delivery volume.
RedisClientPush.PushAsyncaccepts a cancellation token but does not pass it toPublishAsync, because StackExchange.Redis Pub/Sub does not use that token directly.NatsMessageRpcResponderstarts listener tasks directly and logs unexpected listener termination, but does not restart failed listeners.- Readiness, metrics, and integration smoke tests are not packaged with the networking modules.
The plan is to preserve the current abstraction split, then add typed provider options, validation, bounded queues, health checks, delivery-semantic documentation in interfaces, central subject/channel naming helpers, and contract/integration tests around the transport and broker boundaries.
Current Structure Observations
Strong Foundations
SOR.Networking.Abstractionskeeps service code decoupled from WebSocket, NATS, and Redis.SOR.Networking.WebSocketimplementsITransportandIClientConnectionwithout leaking ASP.NET Core WebSocket types through the abstraction boundary.WebSocketClientConnectionserializes concurrent sends withSemaphoreSlim.WebSocketClientConnectionenforcesMaxMessageSizeByteswhile assembling frames.WebSocketTransportuses a channel to hand accepted ASP.NET Core WebSocket connections to the Gateway accept loop.SOR.Networking.Natswraps NATS behindIEventBus.NatsEventBusexplicitly documents core NATS at-most-once semantics and reserves JetStream for durable delivery needs.NatsMessageRpcResponderexposes localIMessageHandlerimplementations over NATS request/reply so Gateway can relay client requests to worker processes.NatsMessageRpcResponderuses queue groups per message type for competing consumers.SOR.Networking.Rediskeeps push publishing separate from Gateway's Redis subscription/read side.IClientPushandRedisClientPushboth document best-effort/offline-drop behavior.SOR.Shared.Nats.NatsSubjectsandSOR.Shared.Redis.RedisKeysalready provide shared naming contracts for event subjects and Redis push channels.
Main Problems
WebSocketTransportusesChannel.CreateUnbounded<IClientConnection>, so the process has no configured accept backlog limit.WebSocketTransportOptionscontains path, ping interval, and max message size, but no validator prevents invalid values.appsettings.jsonincludesWebSocket:AuthTimeoutMsandWebSocket:MaxMessagesPerSecond, butWebSocketTransportOptionsonly defines path, ping interval, and max message size. That creates config drift between host settings and transport settings.- WebSocket keep-alive is configured in
SOR.Hostseparately from the application-level ping loop inWebSocketClientConnection. - The ping loop is fire-and-forget; it catches/logs exceptions, but it is not represented as a hosted lifecycle primitive.
- There is no explicit per-connection outbound queue limit or send timeout policy.
NatsEventBus.SubscribeAsyncalways usesqueueGroup: null, which is good for true fan-out but should be hard to misuse for singleton worker workflows.- NATS RPC subject/queue naming exists in both Gateway dispatch and
NatsMessageRpcResponderas local helper methods. NatsMessageRpcRespondercreates one listener task per handler type. If a listener exits unexpectedly, it is logged but not restarted.NatsMessageRpcResponderdiscovers duplicate handlers at startup, but there is no diagnostics endpoint or test that verifies the registered RPC surface.RedisClientPushpublishes bytes and intentionally ignores subscriber count; this should be observable assubscriber_count=0, not invisible.IClientPushdoes not expose or record delivery hints such as subscriber count, offline drop, or publish latency.SOR.Hosthas only a minimal/healthcheck and a TODO for Redis, NATS, and Postgres dependency checks.- There are no visible contract tests proving message serialization, RPC relay behavior, WebSocket frame limits, cancellation, or shutdown semantics.
Target Networking Shape
Keep the provider split, but make operational policy explicit:
Server/src/SOR.Networking.Abstractions/
EventBus/
IEventBus.cs
EventSubscriptionOptions.cs
DeliverySemantics.cs
Messaging/
IMessageDispatcher.cs
IMessageHandler.cs
IMessageSerializer.cs
Push/
IClientPush.cs
ClientPushResult.cs
Rpc/
RpcSubjects.cs
RemoteDispatchOptions.cs
Transport/
ITransport.cs
IClientConnection.cs
TransportLimits.cs
Server/src/SOR.Networking.WebSocket/
Configuration/
WebSocketTransportOptions.cs
WebSocketTransportOptionsValidator.cs
Transport/
WebSocketTransport.cs
WebSocketClientConnection.cs
Health/
WebSocketTransportHealthCheck.cs
Metrics/
WebSocketMetrics.cs
Server/src/SOR.Networking.Nats/
Configuration/
NatsMessagingOptions.cs
NatsMessagingOptionsValidator.cs
EventBus/
NatsEventBus.cs
Rpc/
NatsMessageRpcResponder.cs
Health/
NatsHealthCheck.cs
Metrics/
NatsMetrics.cs
Server/src/SOR.Networking.Redis/
Configuration/
RedisPushOptions.cs
RedisPushOptionsValidator.cs
Push/
RedisClientPush.cs
Health/
RedisHealthCheck.cs
Metrics/
RedisPushMetrics.cs
12-Factor Assessment
1. Codebase
Status: mostly aligned.
Keep SOR.Networking as shared infrastructure code, not a deployable process. The process boundary remains SOR.Host and role selection remains driven by ENABLED_SERVICES.
Plan:
- Keep abstractions and provider implementations in separate projects.
- Move duplicated NATS RPC subject/queue naming into one shared helper.
- Keep Gateway-specific Redis subscription behavior in
SOR.Gateway, but document the full push contract in Networking. - Add networking contract tests that can run without service modules.
2. Dependencies
Status: mostly aligned.
Provider dependencies are isolated: NATS packages stay in SOR.Networking.Nats, Redis packages stay in SOR.Networking.Redis, and ASP.NET Core WebSocket types stay in SOR.Networking.WebSocket.
Plan:
- Keep provider packages out of
SOR.Networking.Abstractions. - Add test projects per provider where integration dependencies are needed.
- Avoid introducing direct references from service modules to provider-specific networking projects.
3. Config
Status: partially aligned.
Config is environment-compatible through SOR.Host, but provider options are incomplete and not validated.
Plan:
- Add validators for
WebSocketTransportOptions,NatsMessagingOptions, andRedisPushOptions. - Reconcile config drift around
AuthTimeoutMsandMaxMessagesPerSecond; either move them to Gateway options or define them in transport options if transport owns them. - Promote NATS RPC timeout, queue group prefix, subject prefix, and listener restart policy into typed options.
- Promote WebSocket accept backlog, max active connections, send timeout, ping interval, and max payload size into typed options.
- Promote Redis push channel prefix and metrics behavior into typed options.
- Fail fast on invalid or missing production config.
4. Backing Services
Status: aligned in concept, incomplete operationally.
Redis and NATS are attached backing services. They should be replaceable by changing config and provider registration, not changing business services.
Plan:
- Add Redis and NATS health checks that can be included in readiness.
- Separate liveness from readiness: liveness checks process health, readiness checks broker connectivity.
- Document Redis Pub/Sub as best-effort push, not durable state.
- Document core NATS as at-most-once eventing, and define when JetStream is required.
5. Build, Release, Run
Status: mostly aligned.
Networking projects are library code loaded by the host at runtime.
Plan:
- Keep build artifacts environment-neutral.
- Keep provider selection and provider URLs in config.
- Add startup validation so bad broker URLs, invalid WebSocket limits, or missing options fail during release/run startup rather than at first message.
6. Processes
Status: mostly aligned.
The networking modules keep connection/process state but do not own durable application state. WebSocket connections are inherently process-local; Redis and NATS externalize cross-process communication.
Plan:
- Treat WebSocket connections as disposable process state.
- Treat Redis/NATS as the only cross-process communication boundary.
- Keep user/game/session state outside Networking.
- Make offline reconnect behavior rely on persistent service state and client sync, not push replay.
7. Port Binding
Status: aligned through SOR.Host.
Networking libraries do not bind ports directly. SOR.Host owns Kestrel, HTTP, gRPC, and WebSocket upgrade configuration.
Plan:
- Keep WebSocket path and Kestrel behavior in host-level config.
- Keep transport registration behind
AddWebSocketTransport. - Ensure gateway-only WebSocket registration remains role-gated by
ENABLED_SERVICES.
8. Concurrency
Status: partially aligned.
The code uses async APIs, channels, and cancellation, but overload behavior is not explicit.
Plan:
- Replace the unbounded WebSocket accept channel with a bounded channel and a configured full-mode policy.
- Add metrics for pending accepts, active connections, rejected upgrades, receive payload size, send latency, and disconnect reason.
- Add per-connection send timeout and max outbound message size.
- Make NATS subscription type explicit: fan-out subscription vs queue-group subscription.
- Add listener restart policy or process-fatal behavior for unexpected NATS RPC listener exits.
- Add tests for duplicate handler registration and RPC queue group distribution.
9. Disposability
Status: partially aligned.
Cancellation is present, but shutdown and drain behavior should be more explicit.
Plan:
- On shutdown, stop accepting new WebSocket upgrades before waiting for active connections.
- Add a configured graceful connection drain timeout.
- Ensure pending accepts are completed/cancelled cleanly.
- Ensure NATS RPC subscriptions stop cleanly and report listener shutdown.
- Ensure Redis push publishes fail fast when Redis is unavailable rather than blocking request paths indefinitely.
- Add integration tests for shutdown cancellation in WebSocket, NATS responder, and Redis push.
10. Dev/Prod Parity
Status: partially aligned.
The host uses local default Redis/NATS config, but parity depends on developers running equivalent backing services.
Plan:
- Provide a local compose profile for Redis and NATS.
- Keep local broker behavior close to production defaults.
- Add a smoke-test command that verifies WebSocket upgrade, NATS publish/subscribe, NATS RPC, and Redis push publish.
- Avoid hidden in-memory fallbacks in production service registration.
11. Logs
Status: partially aligned.
Logging exists, but operational signals need to be structured and measurable.
Plan:
- Add structured fields for transport name, connection ID, user ID, subject, queue group, channel, message type, subscriber count, payload size, latency, and failure reason.
- Add counters for WebSocket accepted/rejected/closed, NATS publish/subscribe failures, RPC requests/errors/timeouts, and Redis push subscriber counts.
- Keep logs on stdout/stderr through host logging.
- Avoid logging payload bodies unless a redaction policy exists.
12. Admin Processes
Status: early.
Networking has no explicit one-off diagnostics or repair commands.
Plan:
- Add broker diagnostics that can run as admin commands or smoke-test jobs.
- Add a handler registry dump for NATS RPC responder startup.
- Add an optional subject/channel inventory command for known
NatsSubjects, RPC subjects, and Redis keys. - If durable messaging is introduced later, add replay/dead-letter admin tooling at that time.
Refactor Phases
Phase 0: Baseline Contracts
- Add tests for
IMessageSerializerwire compatibility across WebSocket, Redis push, and NATS RPC payloads. - Add tests for WebSocket max message size handling.
- Add tests for duplicate
IMessageHandlerdetection inNatsMessageRpcResponder. - Add tests for Redis push channel naming.
- Add tests that Gateway and responder compute the same RPC subject/queue names.
Phase 1: Configuration Cleanup
- Add typed option classes for NATS messaging and Redis push.
- Add
IValidateOptions<T>validators for WebSocket, NATS, and Redis options. - Resolve
WebSocketconfig drift by moving auth/rate-limit options to Gateway or implementing them in transport. - Add startup validation for required broker URLs and limits.
Phase 2: Backpressure and Limits
- Replace unbounded pending WebSocket accepts with bounded channel config.
- Add
MaxActiveConnections. - Add explicit reject behavior for overload.
- Add send timeout and optional max outbound payload size.
- Add metrics for pending accepts and rejected upgrades.
Phase 3: Subject and Channel Contracts
- Centralize NATS RPC subject and queue naming in shared code.
- Keep event subjects in
NatsSubjects. - Keep Redis push channel names in
RedisKeys. - Add tests that prevent accidental naming drift across Gateway, responder, and publishers.
Phase 4: Broker Hardening
- Add NATS health check.
- Add Redis health check.
- Add NATS RPC listener restart/fatal policy.
- Add explicit subscription options for fan-out vs queue-group consumption.
- Define when core NATS is sufficient and when JetStream is required.
Phase 5: Observability
- Add metrics wrappers around WebSocket accepts/sends/receives.
- Add NATS publish, subscribe, and RPC metrics.
- Add Redis push metrics including subscriber count.
- Add structured logging fields across transport, push, event bus, and RPC.
Phase 6: Disposability
- Add graceful WebSocket drain behavior.
- Add tests for cancellation and shutdown paths.
- Ensure NATS and Redis dependencies fail quickly when unavailable.
- Add readiness checks that remove the process from traffic before shutdown drains active work.
Phase 7: Dev/Prod Parity and Runbooks
- Add local broker startup instructions.
- Add smoke tests for WebSocket, Redis, NATS event bus, and NATS RPC.
- Document expected production env vars.
- Document operational delivery semantics for events, RPC, and push.
Immediate Next Steps
- Add
WebSocketTransportOptionsValidatorand reconcile unused WebSocket config keys. - Centralize RPC subject/queue naming used by Gateway and
NatsMessageRpcResponder. - Add Redis/NATS readiness checks and wire them into host readiness.
- Add metrics for Redis push subscriber count and NATS RPC handler latency.
- Replace the WebSocket accept queue with a bounded, configurable channel.
Definition of Done
- All networking provider options are typed, validated, and environment-overridable.
- WebSocket overload behavior is bounded and observable.
- Redis, NATS, and WebSocket readiness/health behavior is explicit.
- Event, push, and RPC delivery semantics are documented in interfaces and runbooks.
- NATS RPC naming cannot drift between Gateway and responder.
- Integration tests cover WebSocket frame limits, NATS event publish/subscribe, NATS RPC, Redis push publish, and shutdown cancellation.
- No service module directly depends on concrete provider implementations.