First commit

README.md

@@ -0,0 +1,60 @@
+# Ultimate Tic Tac Toe Deep Learning Bot
+
+**Usage**
+Run `python run.py --help` for help.
+
+**Shared flags**
+- --device: Torch device string. If omitted, it auto-picks "cuda" when available, otherwise "cpu".
+- --checkpoint: Path to the model checkpoint file. Default is latest.pth. It is loaded for eval, play, and checkpoint-based arena,
+and used as the save path for accepted training checkpoints.
+
+**Training parameters**
+- --resume: Loads model and optimizer state from --checkpoint before continuing training.
+- --num-simulations default 100: MCTS rollouts per move during self-play. Higher is stronger/slower.
+- --num-iters default 50: Number of outer training iterations. Each iteration generates new self-play games, trains, then arena-tests
+    the new model.
+- --num-eps default 20: Self-play games per iteration.
+- --epochs default 5: Passes over the current replay-buffer training set per iteration.
+- --batch-size default 64: Mini-batch size for gradient updates.
+- --lr default 5e-4: Adam learning rate.
+- --weight-decay default 1e-4: Adam weight decay (L2-style regularization).
+- --replay-buffer-size default 50000: Maximum number of training examples retained across iterations. Older examples are dropped.
+- --value-loss-weight default 1.0: Multiplier on the value-head loss in total training loss. Total loss is policy_KL +
+    value_loss_weight * value_loss.
+- --grad-clip-norm default 5.0: Global gradient norm clipping threshold before optimizer step.
+- --temperature-threshold default 10: In self-play, moves before this step use stochastic sampling from MCTS visit counts; later
+    moves use greedy selection.
+- --root-dirichlet-alpha default 0.3: Dirichlet noise alpha added to root priors during self-play MCTS to force exploration.
+- --root-exploration-fraction default 0.25: How much of that root prior is replaced by Dirichlet noise.
+- --arena-compare-games default 6: Number of head-to-head games between candidate and previous model after each iteration. If <= 0,
+    every candidate is accepted.
+- --arena-accept-threshold default 0.55: Minimum average points needed in arena to keep the new model. Win = 1, draw = 0.5.
+- --arena-compare-simulations default 8: MCTS simulations per move during those arena comparison games. Separate from self-play
+    --num-simulations. 
+
+**Evaluation parameters**
+
+- --moves default "": Comma-separated move list to reach a position from the starting board, e.g. 0,10,4.
+- --top-k default 10: How many highest-probability legal moves to print from the model policy.
+- --with-mcts: Also run MCTS on that position and print the best move, instead of only raw network policy/value.
+- --num-simulations default 100: Only matters with --with-mcts; controls MCTS search depth for that evaluation.
+
+**Play parameters**
+
+- --human-player default 1: Which side you control. 1 means X, -1 means O.
+- --num-simulations default 100: MCTS simulations the AI uses for each move.
+
+**Arena parameters**
+- --games default 20: Number of matches to run.
+- --num-simulations default 100: MCTS simulations per move for checkpoint-based players. 
+- --x-player / --o-player: Either checkpoint or random. Chooses the agent type for each side.
+- --x-checkpoint / --o-checkpoint: Checkpoint path for that side when its player type is checkpoint. Ignored for random.
+
+A few practical examples:
+
+```bash
+python run.py train --num-iters 100 --num-eps 50 --resume
+python run.py eval --checkpoint latest.pth --moves 0,10,4 --with-mcts --num-simulations 200
+python run.py play --human-player -1 --num-simulations 300
+python run.py arena --games 50 --x-player checkpoint --o-player random
+```