Unit 0 — Setup & First Run

Course home

Time

Reading: ~20 min · Training: ~15 min GPU / ~1 h CPU

Install the Godot .NET editor, Python toolchain, and godot-rl-agents plugin. Run your first training session with the BallChase example and confirm the Godot ↔ Python socket works.

---

First success (one session)

1. Godot — agent moving (gdrl --viz + Play Scene)
2. Python — terminal shows steps; ep_rew_mean trends up
3. TensorBoard (optional) — tensorboard --logdir=logs shows a curve
4. Neural Foundations — you will build visible neurons before the RL loop

Three ways to see your AI (every unit)

Godot behavior · TensorBoard curves · what you change in AIController

Short on time?

Follow the first evening script (~2½–3 h) to finish Unit 0 and start Neural Foundations 1 in one sitting.

---

1 · Split architecture

Two runtimes talk over a local socket — Godot sends observations and receives actions; Python runs the training loop.

Component	Role	Runtime
Godot	Physics, observations, rewards	Godot 4 .NET + GDScript
Plugin (C#)	Sync node, ONNX bridge	.NET / MSBuild
Python	PPO / DQN training (SB3)	Conda, Python 3.10

Godot .NET edition required

The standard Godot build cannot load the plugin's C# / NuGet dependencies. Download the .NET build from godotengine.org and install the .NET SDK.

---

2 · Conda environment

First time here?

Full installation instructions — Miniconda, godot_env, and the Godot plugin — are in Setup. Complete that page first, then return here.

Quick reminder for every new terminal:

conda activate godot_env

---

3 · Godot project & plugin

The training environments come from the separate godot_rl_agents_examples repo — not the course repo's own examples/ folder. Clone it as a sibling of the course repo:

cd ..   # step out of godot-rl-course
git clone https://github.com/edbeeching/godot_rl_agents_examples.git

No git? Use Code → Download ZIP on github.com/edbeeching/godot_rl_agents_examples and unpack it next to the course repo.

Then:

1. Godot → Import → browse to godot_rl_agents_examples/examples/BallChase/project.godot
2. Project → Project Settings → Plugins → enable Godot RL Agents
3. Wait for MSBuild to finish

---

4 · First training run

In-editor

Terminal 1 — start Python listener:

gdrl --experiment_name=BallChase_Mac --viz \
  --save_model_path=ballchase_brain \
  --onnx_export_path=ballchase_brain.onnx

Wait for "Waiting for connection from Godot…"

Terminal 2 optional: tensorboard --logdir=logs

Godot — open the training scene, press F6 (Play Scene). The agent should connect and learn.

Success criteria

Agent visible in Godot; episode reward trends upward; no socket errors; TensorBoard curve optional but recommended.

---

5 · Training modes for this course

Phase	Units	Default
Explore	0–2	Editor or --viz — see physics and reward bugs
Scale	3–8	Exported binary + --headless — faster rollouts
Ship	9–10	ONNX in Sync node — no Python at runtime

ONNX preview (optional)

After training, copy ballchase_brain.onnx into the Godot project. On the Sync node: Control Mode → ONNX Inference, set ONNX Model Path. Play scene — agent runs without Python.

---

First evening script (~2½–3 hours)

One sitting: tooling works, agent learns, you change a reward. Times are guides — install steps vary by machine.

Block	Time	Do this	Done when
1 · Install	45–75 min	Section 2 — install Miniconda → create godot_env → pip install Section 3 — clone examples repo, open BallChase in Godot	import godot_rl prints ok; BallChase project opens with plugin enabled
2 · First train	30–45 min	Section 4 — gdrl --viz + Godot F6 Second terminal: tensorboard --logdir=logs	Agent moves; ep_rew_mean rises; no socket errors
3 · Start Foundations 1	45–60 min	Open Neural Foundations 1 Read Section 1 (what a neuron computes), then predict the hand calculation in Section 2 (~15 min) → run the research plot or the Godot jumper scene While exploring: read Sections 3–4	You can name each contribution, weighted sum, and activation output

Minimal command cheat sheet (Block 2)

conda activate godot_env
tensorboard --logdir=logs &

gdrl --experiment_name=evening_ballchase --viz \
  --save_model_path=ballchase_brain \
  --onnx_export_path=ballchase_brain.onnx

# Godot: BallChase training scene → F6 (Play Scene)

End-of-evening checklist

• [ ] Godot — saw the agent act
• [ ] Python — training ran without connection errors
• [ ] TensorBoard — opened at least once (localhost:6006)
• [ ] Code — ran one neuron forward-pass test or visual example

Tomorrow: finish Foundations 1–2, then RL Essentials (BallChase reward tweak) and Unit 2 Phase A (SimpleReachGoal).

Stuck?

Most first-evening blockers: wrong Godot build (need .NET), plugin not enabled, Python started before Godot F6, or firewall blocking localhost socket. Re-read Section 1 if the split architecture is unclear.

---

Stretch Goals

Inspect the exported ONNX. Open ballchase_brain.onnx in Netron (drag the file into the browser tab — no install). Identify the input tensor shape, the output tensor shape, and the activation between layers. This is the file Godot loads at inference time in Phase 3 — knowing what's inside it now pays off in Unit 9.

Try a second example from the repo. Open examples/JumperHard in Godot, run a short gdrl --viz session, and compare reward curves on TensorBoard. The goal isn't to train it well — it's to confirm your install handles more than one environment.

What's next

Tooling works. In Neural Foundations 1 you'll build one visible neuron, then connect networks to the RL loop in later units.

Two optional refreshers sit between here and there: Math Foundations 1 covers vectors and dot products, Math Foundations 2 probability and expected value. Skip them if the arithmetic in Neural Foundations 1 feels comfortable, and come back when a later unit's notation stops making sense.

→ Math Foundations 1 · skip to Neural Foundations 1