Files
d5a0efdb77 Implement AI Phase 5: Python neural training pipeline
- tools/train_neural.py: PyTorch MLP training script that reads JSONL from
  GameReplayExporter and exports a compliant ONNX model. Architecture is
  1024->512->256->128->1 (tanh) with LayerNorm + Dropout. AdamW optimizer
  with cosine LR annealing, gradient clipping, best-checkpoint restore.
  Post-export validation via onnxruntime confirms tensor names and shapes.
- tools/requirements.txt: pinned deps (torch 2.3.1, onnx 1.16.1,
  onnxruntime 1.18.0 matching the C# package, numpy 1.26.4)
- tools/README.md: end-to-end workflow docs covering self-play, data export,
  training options, deployment (hot-swap vs restart), ONNX contract table,
  and iterative training guidance

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
2026-04-12 21:26:45 -04:00
..

SOR Neural AI Training Tools

This directory contains the Python training pipeline for the SOR neural decision engine (tiers 9-10). The workflow is:

self-play (admin) -> JSONL export (admin) -> train_neural.py -> model.onnx -> hot-swap (admin)

Prerequisites

Python 3.11+ with pip:

pip install -r tools/requirements.txt

GPU training is supported via CUDA. Use --cpu to force CPU-only mode.


Step 1 — Generate self-play data (Admin portal)

  1. Navigate to Admin -> AI -> Calibration.
  2. Click Run All Pairs (or a specific difficulty pair) to trigger in-process self-play games.
  3. Wait for the run to complete — check the calibration table for results.

Alternatively, trigger a full calibration run via the API:

POST /api/ai/calibration/run-all

Step 2 — Export training data (Admin portal)

  1. Navigate to Admin -> AI -> Calibration.
  2. Click Export Training Data and enter an output path on the server (e.g. /var/sor/training/data.jsonl).
  3. The exporter replays completed game sessions and writes one JSON line per action:
{
  "GameSessionId": "3fa85f64...",
  "TurnNumber": 5,
  "ActorAccountId": "...",
  "Features": [0.0, 1.0, ...],
  "ActionType": "AiPlayCardAction",
  "ActionJson": "{}",
  "Outcome": 1.0
}

Outcome is +1.0 if the acting player won the game, -1.0 if they lost.


Step 3 — Train the model

cd /path/to/SOR
python tools/train_neural.py     --data /var/sor/training/data.jsonl     --out  /var/sor/models/model.onnx     --epochs 50     --batch-size 512     --lr 1e-3

Options

Flag Default Description
--data (required) Path to the JSONL export file
--out model.onnx Output ONNX file path
--epochs 50 Number of training epochs
--batch-size 512 Mini-batch size
--lr 0.001 AdamW initial learning rate
--val-split 0.1 Fraction of data held out for validation
--dropout 0.2 Dropout probability in the MLP
--seed 42 Random seed
--cpu off Force CPU training (ignores CUDA)

The script prints per-epoch train/val loss and marks the best checkpoint with *. It restores the best weights before exporting.

Model architecture

Input   [1, 1024] float32  (FeatureExtractor output)
Linear  1024 -> 512  + LayerNorm + ReLU + Dropout
Linear   512 -> 256  + LayerNorm + ReLU + Dropout
Linear   256 -> 128  + LayerNorm + ReLU
Linear   128 ->   1  + Tanh
Output  [1, 1]    float32  (score in [-1, 1])

Total parameters: ~920k


Step 4 — Deploy the model

Option A: Hot-swap via admin portal (no restart)

  1. Copy the model.onnx file to the path configured in appsettings:
    "Ai": { "NeuralModelPath": "/var/sor/models/model.onnx" }
    
  2. Navigate to Admin -> AI -> Neural Model.
  3. Click Hot-swap Model. The old session is disposed atomically and the new model is loaded from disk. All subsequent tier 9-10 decisions use the new model.

Option B: Restart the service

The model is loaded at startup by AiNeuralStartupService. If the file exists at the configured path, it will be loaded automatically on the next deploy.


ONNX contract

The exported model must satisfy:

Property Value
Input tensor name features
Input shape [1, 1024] (or [batch, 1024] with dynamic batch)
Input dtype float32
Output tensor name score
Output shape [1, 1] (or [batch, 1])
Output dtype float32
Output range [-1, 1] (tanh)

These names are defined as constants in NeuralModelRegistry.cs (InputName, OutputName). The Python script and the C# inference code must agree on these names exactly.


Iterative training

For best results, run multiple rounds:

  1. Start with random-engine games (all difficulties) to generate diverse data.
  2. Train an initial model and deploy it.
  3. Run additional self-play with the neural engine enabled (tiers 9-10 play each other).
  4. Append new exports to the existing JSONL and retrain with more epochs.
  5. Monitor calibration win rates in the admin portal after each deployment.