# SOR Neural AI Training Tools This directory contains the Python training pipeline for the SOR neural decision engine (tiers 9-10). The workflow is: ``` self-play (admin) -> JSONL export (admin) -> train_neural.py -> model.onnx -> hot-swap (admin) ``` --- ## Prerequisites Python 3.11+ with pip: ```bash pip install -r tools/requirements.txt ``` GPU training is supported via CUDA. Use `--cpu` to force CPU-only mode. --- ## Step 1 — Generate self-play data (Admin portal) 1. Navigate to **Admin -> AI -> Calibration**. 2. Click **Run All Pairs** (or a specific difficulty pair) to trigger in-process self-play games. 3. Wait for the run to complete — check the calibration table for results. Alternatively, trigger a full calibration run via the API: ``` POST /api/ai/calibration/run-all ``` --- ## Step 2 — Export training data (Admin portal) 1. Navigate to **Admin -> AI -> Calibration**. 2. Click **Export Training Data** and enter an output path on the server (e.g. `/var/sor/training/data.jsonl`). 3. The exporter replays completed game sessions and writes one JSON line per action: ```json { "GameSessionId": "3fa85f64...", "TurnNumber": 5, "ActorAccountId": "...", "Features": [0.0, 1.0, ...], "ActionType": "AiPlayCardAction", "ActionJson": "{}", "Outcome": 1.0 } ``` `Outcome` is `+1.0` if the acting player won the game, `-1.0` if they lost. --- ## Step 3 — Train the model ```bash cd /path/to/SOR python tools/train_neural.py --data /var/sor/training/data.jsonl --out /var/sor/models/model.onnx --epochs 50 --batch-size 512 --lr 1e-3 ``` ### Options | Flag | Default | Description | |------|---------|-------------| | `--data` | *(required)* | Path to the JSONL export file | | `--out` | `model.onnx` | Output ONNX file path | | `--epochs` | `50` | Number of training epochs | | `--batch-size` | `512` | Mini-batch size | | `--lr` | `0.001` | AdamW initial learning rate | | `--val-split` | `0.1` | Fraction of data held out for validation | | `--dropout` | `0.2` | Dropout probability in the MLP | | `--seed` | `42` | Random seed | | `--cpu` | off | Force CPU training (ignores CUDA) | The script prints per-epoch train/val loss and marks the best checkpoint with `*`. It restores the best weights before exporting. ### Model architecture ``` Input [1, 1024] float32 (FeatureExtractor output) Linear 1024 -> 512 + LayerNorm + ReLU + Dropout Linear 512 -> 256 + LayerNorm + ReLU + Dropout Linear 256 -> 128 + LayerNorm + ReLU Linear 128 -> 1 + Tanh Output [1, 1] float32 (score in [-1, 1]) ``` Total parameters: ~920k --- ## Step 4 — Deploy the model ### Option A: Hot-swap via admin portal (no restart) 1. Copy the `model.onnx` file to the path configured in `appsettings`: ```json "Ai": { "NeuralModelPath": "/var/sor/models/model.onnx" } ``` 2. Navigate to **Admin -> AI -> Neural Model**. 3. Click **Hot-swap Model**. The old session is disposed atomically and the new model is loaded from disk. All subsequent tier 9-10 decisions use the new model. ### Option B: Restart the service The model is loaded at startup by `AiNeuralStartupService`. If the file exists at the configured path, it will be loaded automatically on the next deploy. --- ## ONNX contract The exported model must satisfy: | Property | Value | |----------|-------| | Input tensor name | `features` | | Input shape | `[1, 1024]` (or `[batch, 1024]` with dynamic batch) | | Input dtype | `float32` | | Output tensor name | `score` | | Output shape | `[1, 1]` (or `[batch, 1]`) | | Output dtype | `float32` | | Output range | `[-1, 1]` (tanh) | These names are defined as constants in `NeuralModelRegistry.cs` (`InputName`, `OutputName`). The Python script and the C# inference code must agree on these names exactly. --- ## Iterative training For best results, run multiple rounds: 1. Start with random-engine games (all difficulties) to generate diverse data. 2. Train an initial model and deploy it. 3. Run additional self-play with the neural engine enabled (tiers 9-10 play each other). 4. Append new exports to the existing JSONL and retrain with more epochs. 5. Monitor calibration win rates in the admin portal after each deployment.