- tools/train_neural.py: PyTorch MLP training script that reads JSONL from GameReplayExporter and exports a compliant ONNX model. Architecture is 1024->512->256->128->1 (tanh) with LayerNorm + Dropout. AdamW optimizer with cosine LR annealing, gradient clipping, best-checkpoint restore. Post-export validation via onnxruntime confirms tensor names and shapes. - tools/requirements.txt: pinned deps (torch 2.3.1, onnx 1.16.1, onnxruntime 1.18.0 matching the C# package, numpy 1.26.4) - tools/README.md: end-to-end workflow docs covering self-play, data export, training options, deployment (hot-swap vs restart), ONNX contract table, and iterative training guidance Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
4.1 KiB
SOR Neural AI Training Tools
This directory contains the Python training pipeline for the SOR neural decision engine (tiers 9-10). The workflow is:
self-play (admin) -> JSONL export (admin) -> train_neural.py -> model.onnx -> hot-swap (admin)
Prerequisites
Python 3.11+ with pip:
pip install -r tools/requirements.txt
GPU training is supported via CUDA. Use --cpu to force CPU-only mode.
Step 1 — Generate self-play data (Admin portal)
- Navigate to Admin -> AI -> Calibration.
- Click Run All Pairs (or a specific difficulty pair) to trigger in-process self-play games.
- Wait for the run to complete — check the calibration table for results.
Alternatively, trigger a full calibration run via the API:
POST /api/ai/calibration/run-all
Step 2 — Export training data (Admin portal)
- Navigate to Admin -> AI -> Calibration.
- Click Export Training Data and enter an output path on the server
(e.g.
/var/sor/training/data.jsonl). - The exporter replays completed game sessions and writes one JSON line per action:
{
"GameSessionId": "3fa85f64...",
"TurnNumber": 5,
"ActorAccountId": "...",
"Features": [0.0, 1.0, ...],
"ActionType": "AiPlayCardAction",
"ActionJson": "{}",
"Outcome": 1.0
}
Outcome is +1.0 if the acting player won the game, -1.0 if they lost.
Step 3 — Train the model
cd /path/to/SOR
python tools/train_neural.py --data /var/sor/training/data.jsonl --out /var/sor/models/model.onnx --epochs 50 --batch-size 512 --lr 1e-3
Options
| Flag | Default | Description |
|---|---|---|
--data |
(required) | Path to the JSONL export file |
--out |
model.onnx |
Output ONNX file path |
--epochs |
50 |
Number of training epochs |
--batch-size |
512 |
Mini-batch size |
--lr |
0.001 |
AdamW initial learning rate |
--val-split |
0.1 |
Fraction of data held out for validation |
--dropout |
0.2 |
Dropout probability in the MLP |
--seed |
42 |
Random seed |
--cpu |
off | Force CPU training (ignores CUDA) |
The script prints per-epoch train/val loss and marks the best checkpoint with *.
It restores the best weights before exporting.
Model architecture
Input [1, 1024] float32 (FeatureExtractor output)
Linear 1024 -> 512 + LayerNorm + ReLU + Dropout
Linear 512 -> 256 + LayerNorm + ReLU + Dropout
Linear 256 -> 128 + LayerNorm + ReLU
Linear 128 -> 1 + Tanh
Output [1, 1] float32 (score in [-1, 1])
Total parameters: ~920k
Step 4 — Deploy the model
Option A: Hot-swap via admin portal (no restart)
- Copy the
model.onnxfile to the path configured inappsettings:"Ai": { "NeuralModelPath": "/var/sor/models/model.onnx" } - Navigate to Admin -> AI -> Neural Model.
- Click Hot-swap Model. The old session is disposed atomically and the new model is loaded from disk. All subsequent tier 9-10 decisions use the new model.
Option B: Restart the service
The model is loaded at startup by AiNeuralStartupService. If the file exists at the
configured path, it will be loaded automatically on the next deploy.
ONNX contract
The exported model must satisfy:
| Property | Value |
|---|---|
| Input tensor name | features |
| Input shape | [1, 1024] (or [batch, 1024] with dynamic batch) |
| Input dtype | float32 |
| Output tensor name | score |
| Output shape | [1, 1] (or [batch, 1]) |
| Output dtype | float32 |
| Output range | [-1, 1] (tanh) |
These names are defined as constants in NeuralModelRegistry.cs (InputName, OutputName).
The Python script and the C# inference code must agree on these names exactly.
Iterative training
For best results, run multiple rounds:
- Start with random-engine games (all difficulties) to generate diverse data.
- Train an initial model and deploy it.
- Run additional self-play with the neural engine enabled (tiers 9-10 play each other).
- Append new exports to the existing JSONL and retrain with more epochs.
- Monitor calibration win rates in the admin portal after each deployment.