Troubleshooting & FAQ
This page collects common errors, warnings, and questions you may encounter while setting up and training RL agents with godot-native-rl, stable-baselines3, and Python — plus the legacy godot-rl-agents examples used by not-yet-migrated units. Each entry explains the root cause and the fix. Check here first before opening a GitHub issue.
Setup & Installation
NcnnSync / NcnnAIController3D not found in Add Node
Cause: The addon is not installed. It is not committed to the course repo, so a fresh clone has no addons/godot_native_rl/ at all — or you opened a different project, or the first import has not finished.
Fix:
scripts/fetch-native-runner.sh # installs the pinned addon release
scripts/fetch-native-runner.sh --check # shows pinned vs. installed
Then in Godot: Import → godot-rl-course/examples/neural_foundations/game/project.godot, let the first import finish, and retry Add Node → search NcnnSync. Nothing is compiled, so the import is quick.
If you are using the addon in your own project, confirm addons/godot_native_rl/ was copied into that project's addons/ folder.
See also: Unit 0 § 3
"NcnnRunner" class missing / native inference fails to load
Cause: Native ncnn inference needs a platform library (GDExtension). Those libraries are not committed to the course repo — they are larger than everything else in it — so a fresh clone has none until you install them.
Fix: Run the installer from the repo root:
It ships macOS Apple Silicon, Windows x86_64 and Linux x86_64, so every desktop platform is covered. If it still fails afterwards, check your Godot version: the extension requires 4.5 or newer and silently does not register on older builds. Training is unaffected either way — the training bridge is GDScript and needs no native library.
Godot version compatibility error
Cause: The bundled ncnn GDExtension declares compatibility_minimum = 4.5; older Godot versions refuse to load the project cleanly.
Fix:
# Download Godot 4.5 or newer (Standard build) from godotengine.org
# Verify on the command line:
godot --version
The legacy godot-rl-agents examples used from RL Essentials onward have their own version constraints — check the godot-rl-agents repository if an example project misbehaves.
ModuleNotFoundError: No module named 'godot_rl' — or gdrl: command not found
Cause: The Python godot-rl package was not installed, or your conda environment is not activated. (The course does not use the gdrl command itself — see Reference — but its absence is the same symptom.)
Fix:
# Activate your conda environment
conda activate godot_env
# Install using the pinned requirements file (recommended)
pip install -r requirements-course.txt
# Verify (expect 0.8.2 — there is no `gdrl --version`)
python -c "import importlib.metadata as m; print(m.version('godot-rl'))"
Package name is godot-rl, not godot-rl-agents
The PyPI package is godot-rl (install with pip install godot-rl or via requirements-course.txt). The old name godot-rl-agents is no longer the canonical package name. The Python import is still import godot_rl.
ModuleNotFoundError: No module named 'stable_baselines3'
Cause: stable-baselines3 is not installed in your active Python environment.
Fix:
If you get a permission error, use pip install --user stable-baselines3 or check that you own /usr/local/lib/python3.x/.
ModuleNotFoundError: No module named 'godot_rl'
Cause: The godot-rl Python package is not installed in your active conda environment.
Fix:
Verify:
Python version incompatibility (godot-rl requires Python 3.8+)
Cause: Your Python environment is older than 3.8, or you are using Python 2.
Fix:
# Check your Python version
python --version
# If < 3.8, create a new conda environment with Python 3.10
conda create -n godot_env python=3.10
conda activate godot_env
pip install -r requirements-course.txt
onnxruntime not installed (ONNX inference fails)
Cause: The onnxruntime package is required for exporting and running policies as ONNX, but was not installed with stable-baselines3.
Fix:
Training fails to start
NcnnSync: minor version mismatch (got 3, expected 7)
Cause: Your godot_env still has an old godot-rl. Protocol 0.7 is what NcnnSync and the godot_rl_agents examples both speak; godot-rl releases before 0.8 speak 0.3. The handshake warns and continues, so training still works — but the environment no longer matches the course.
Fix: Reinstall the pinned versions:
conda activate godot_env
pip install -r requirements-course.txt
python -c "import godot_rl.core.godot_env as g; print(g.GodotEnv.MINOR_VERSION)" # expect 7
Treat an actual stall (no rollout tables after a minute) as a different problem — see the next entry.
See also: Unit 0 § 4
WebSocket connection refused / ConnectionRefusedError: [Errno 111]
Cause: The Python training script is trying to connect to Godot on port 11008 (default), but Godot is not running, not listening, or is on a different port.
Fix:
# 1. Start the PYTHON side first — it opens the socket and waits.
# Godot is the side that connects, so nothing can connect to a server
# that is not listening yet.
conda activate godot_env
python stable_baselines3_example.py --experiment_name=debug --viz
# 2. Wait for "waiting for remote GODOT connection on port 11008",
# then press F6 (Play Scene) in the Godot editor — or pass
# --env_path=./godot_binary in step 1 to let the script launch it.
# 3. If using a custom port, ensure both sides match
# In Python: env = GodotEnv(..., port=12000)
# In Godot: AIController.port = 12000
Check that no other process is blocking port 11008:
Windows: antivirus may silently block port 11008
On Windows, Windows Defender and third-party antivirus programs sometimes block the godot-rl socket port without showing an error. If Godot never connects even though the port appears free, add firewall exceptions for python.exe and the Godot executable. See Windows first run in the setup guide for step-by-step instructions.
"Action space mismatch" / ValueError: Action space size mismatch
Cause: The number of actions your AIController returns (n_actions) does not match the n_actions argument passed to your SB3 algorithm.
Fix:
# In AIController.cs (or your Godot script)
# Count the exact actions you return
public override int GetActionSpaceSize() {
return 2; // e.g., 2 for move_x, move_y
}
# In your Python training script, match exactly
model = PPO(
"MlpPolicy",
env,
n_steps=2048,
...
)
Print both to be sure:
print("Python action space:", env.action_space.shape)
print("Godot n_actions:", env._get_obs()['n_actions']) # or query your AIController
KeyError: 'obs' when starting training
Cause: Your observation dictionary from Godot does not include the required 'obs' key, or is using the wrong key name.
Fix:
# In your AIController, ensure you return a dictionary with 'obs' key
public override Dictionary<string, object> GetObservation() {
return new Dictionary<string, object> {
{ "obs", new float[] { position.X, position.Y, ... } }
};
}
# In Python, access it as
obs = env.reset()
print(obs.keys()) # should include 'obs'
If you are returning a flat array instead of a dict, wrap it:
Training freezes after first rollout (silent Godot crash)
Cause: The Godot process crashed or hung, but the Python side did not receive the error. This commonly happens due to an exception in _process() or _physics_process().
Fix:
# 1. Run Godot itself in the foreground to see stderr/logs
# (`gdrl` is the Python side — it never starts Godot)
godot --path examples/BallChase 2>&1 | tee godot.log
# 2. Check the Godot log for exceptions
# Common culprits: accessing null nodes, division by zero, infinite loops in reward calculation
# 3. Add debug prints in AIController
GD.Print($"Step {_step_count}: obs={obs}, action={action}, reward={reward}");
RuntimeError: Expected all tensors to be on the same device (PyTorch error)
Cause: Your observation or reward is a PyTorch tensor on CPU, but PyTorch expects all tensors on the same device (CPU or GPU).
Fix:
# Ensure all observations and rewards are NumPy arrays or the same tensor device
def _get_obs(self):
obs = np.array([...], dtype=np.float32) # Use NumPy, not torch
return obs
# Or, if using PyTorch tensors, ensure they are all on the same device
obs = obs.to(device) # Move to correct device
Training runs but doesn't learn
NaN loss after a few thousand steps
Cause: The learning rate is too high, the reward is unbounded, or pixel observations are not normalized. This causes exploding gradients.
Fix:
# Reduce the learning rate
model = PPO("MlpPolicy", env, learning_rate=1e-4) # Instead of 1e-3 or higher
# Ensure rewards are bounded (e.g., -1 to 1)
reward = np.clip(reward, -1.0, 1.0)
# Normalize pixel observations to [0, 1] or [-1, 1]
observation = observation.astype(np.float32) / 255.0
ep_rew_mean stays flat at 0 or negative forever
Cause: Your reward function returns 0 (or a very small value) every step, so the agent has no learning signal. Check the sign — negative rewards discourage the agent.
Fix:
// In AIController: reward should be non-zero when the agent makes progress
float reward = 0.0f;
if (Vector2.Distance(transform.GlobalPosition, target.GlobalPosition) < 1.0f) {
reward = 1.0f; // Goal reached
} else {
reward = -0.01f; // Small step cost to encourage progress
}
return reward;
Print rewards to TensorBoard to debug:
# In your training script, log rewards per episode
episode_rewards.append(total_reward)
logger.record("custom/episode_reward", total_reward)
ep_rew_mean oscillates wildly and never improves
Cause: The learning rate is too high, or n_steps is too small for PPO to accumulate enough experience before updating.
Fix:
# Increase n_steps (experience per update)
model = PPO(
"MlpPolicy",
env,
n_steps=4096, # Instead of 2048
learning_rate=3e-4, # Reduce if still unstable
ent_coef=0.01, # Increase entropy coefficient to encourage exploration
)
Train longer and check TensorBoard for trends over 100k+ steps, not just the first 10k.
RuntimeError: CUDA out of memory (GPU error)
Cause: Your batch size is too large for the GPU memory, or you are running multiple environments in parallel without enough VRAM.
Fix:
# Reduce the buffer size and batch size
model = PPO(
"MlpPolicy",
env,
n_steps=1024, # Smaller rollout buffer
batch_size=64, # Smaller training batch
)
# Or, use CPU only
env = DummyVecEnv([lambda: GodotEnv(..., use_discrete_actions=False)]) # CPU training
Agent spins in place / does nothing (action ignored)
Cause: The action space is incorrectly mapped in Godot, or actions are not being applied to the physics body.
Fix:
// In AIController: ensure actions are applied each frame
public override void SetAction(float[] action) {
// action[0] and action[1] are continuous or discrete actions
velocity.X = action[0] * max_speed;
velocity.Y = action[1] * max_speed;
_body.Velocity = velocity;
_body.MoveAndSlide();
}
// Check that you call MoveAndSlide() in _physics_process
public override void _PhysicsProcess(float delta) {
SetAction(_last_action);
_body.Velocity = velocity;
_body.MoveAndSlide();
}
Entropy collapses to near-zero early (policy becomes deterministic)
Cause: The entropy coefficient ent_coef is too low, or the policy has converged to a deterministic behavior before learning the task.
Fix:
# Increase entropy coefficient to encourage exploration
model = PPO(
"MlpPolicy",
env,
ent_coef=0.1, # Increase from default 0.0
learning_rate=3e-4,
)
# Or, use a schedule to decay entropy over time
from stable_baselines3.common.callbacks import EvalCallback
model = PPO(
"MlpPolicy",
env,
ent_coef=0.05,
policy_kwargs={"net_arch": [128, 128]}, # Larger network = more exploration needed
)
Behavior & Reward debugging
TensorBoard reward goes up but agent looks wrong in Godot
Cause: The reward function is not aligned with the desired behavior. The agent is maximizing the numerical reward without solving the task (reward hacking).
Fix:
// Example: agent may "cheat" by moving quickly rather than reaching the goal
// BAD: only reward for velocity
float reward = velocity.Length() * 0.1f;
// BETTER: reward progress toward goal AND reaching goal
float distance_to_goal = Vector2.Distance(transform.GlobalPosition, goal.GlobalPosition);
float reward = -distance_to_goal * 0.1f; // Progress penalty
if (distance_to_goal < 1.0f) {
reward += 1.0f; // Goal bonus
}
Watch the agent in Godot while training and compare to the TensorBoard curve. They should match in spirit.
Agent ignores obstacles / targets that are clearly visible
Cause: The observation does not include or update information about obstacles/targets, or observations are cached and not refreshed after each step.
Fix:
// In AIController: ensure observations are updated every frame
public override Dictionary<string, object> GetObservation() {
// Observations MUST be recomputed each call, not cached
var obs = new float[] {
transform.GlobalPosition.X,
transform.GlobalPosition.Y,
target.GlobalPosition.X,
target.GlobalPosition.Y,
// ... include obstacle positions
};
return new Dictionary<string, object> { { "obs", obs } };
}
// Call in _PhysicsProcess, not _Process (to stay in sync with physics)
public override void _PhysicsProcess(float delta) {
var obs = GetObservation();
// Reset position next frame if goal reached
}
Agent learns to exploit reward instead of solving the task
Cause: Reward hacking — the agent found a way to maximize the numerical reward without actually achieving the intended goal.
Fix:
// Example: agent may stay in place and get stuck if you only reward reaching goal
// Add a small step cost to push the agent forward
float reward = -0.01f; // Step penalty
// Penalize if agent gets stuck
if (stuck_timer > max_stuck_time) {
reward -= 0.5f;
}
// Reward goal
if (distance < goal_radius) {
reward += 1.0f;
}
return reward;
Manually verify that the highest-reward actions match your intuition about the task.
rollout/ep_len_mean always equals max_episode_steps (agent never reaches goal)
Cause: The done condition is never triggered — either the agent never succeeds, or you are not returning done=True when the episode should end.
Fix:
// In AIController: check and return done
public override bool IsDone() {
// Done when goal is reached
bool goal_reached = Vector2.Distance(transform.GlobalPosition, goal.GlobalPosition) < 1.0f;
// Or when out of bounds
bool out_of_bounds = transform.GlobalPosition.Length() > max_distance;
// Or when max steps exceeded (framework handles this, but you can override)
bool timeout = step_count >= max_steps;
return goal_reached || out_of_bounds || timeout;
}
Debug by printing:
# In Python
done_reasons = {"goal": 0, "timeout": 0, "oob": 0}
for episode in range(100):
obs, info = env.reset()
while True:
action, _ = model.predict(obs)
obs, reward, terminated, truncated, info = env.step(action)
done = terminated or truncated
if done:
if info.get("goal_reached"):
done_reasons["goal"] += 1
else:
done_reasons["timeout"] += 1
break
print(done_reasons)
ONNX export & deployment
ONNX export succeeds but Godot inference produces garbage actions
Cause: The ONNX export did not include the output tanh squashing, or the actor was exported alone without the log-std layer (SAC).
Fix:
# For PPO, export the full policy (includes tanh output layer)
model.policy.actor.to("cpu")
model.policy.actor.eval()
# Use the stable_baselines3 export utility
from stable_baselines3.common.policies import BasePolicy
input_dict = {"obs": np.zeros((1, obs_space.shape[0]), dtype=np.float32)}
model.policy.actor.to("cpu")
# Export with opset 11 or 15 to ensure compatibility
import onnx
onnx_model, _ = convert_pytorch(model.policy.actor, input_dict, opset_version=11)
onnx.save(onnx_model, "policy.onnx")
Test in Python before deploying to Godot:
import onnxruntime as ort
sess = ort.InferenceSession("policy.onnx")
action = sess.run(None, {"obs": obs})
print(action[0].shape, action[0].min(), action[0].max()) # Should be [-1, 1] for tanh
ONNX model in Godot returns wrong tensor shape
Cause: The input_names or output_names do not match the exported ONNX model, or the input/output order is swapped.
Fix:
// In Godot C#, when loading ONNX:
var model = new OnnxModel();
model.LoadModel("res://policy.onnx");
// Get the input/output node names from the exported model
// Use a tool like Netron to inspect the ONNX file:
// - Input node: "obs" (float32, shape [1, n_obs])
// - Output node: "output_0" (float32, shape [1, n_actions])
var input_dict = new Dictionary<string, Tensor> {
{ "obs", Tensor.FromArray<float>(obs_array) }
};
var outputs = model.Run(input_dict);
var actions = outputs["output_0"]; // or "output_name" from your export
Inspect your ONNX file with Netron (online tool at netron.app) to verify node names and shapes.
ONNX model inference is slow in Godot
Cause: The model is too large (e.g., CNN policy for visual obs), or you are using a GPU-incompatible runtime.
Fix:
# Use MlpPolicy for real-time inference, not CnnPolicy
model = PPO("MlpPolicy", env, ...) # Good for real-time
# Avoid: PPO("CnnPolicy", env, ...) # Slow in game engine
# For visual observations, either:
# 1. Train with CnnPolicy in Python, then distill to MlpPolicy
# 2. Use very small image sizes (e.g., 32x32 instead of 84x84)
Typical inference time for MlpPolicy on Godot: 1–5 ms. For CnnPolicy: 50–200 ms.
onnxruntime.InvalidGraph: Load model from ... failed
Cause: The ONNX opset version is incompatible with your onnxruntime, or the model was not exported correctly.
Fix:
# Export with opset version 11 or 15 (both widely supported)
from stable_baselines3.common.policies import BasePolicy
# When exporting, specify opset_version=11
# Or, update your onnxruntime
pip install --upgrade onnxruntime
Also check that the ONNX file is valid:
python -c "import onnx; model = onnx.load('policy.onnx'); onnx.checker.check_model(model); print('Valid')"
Still stuck?
If your issue is not listed here, check the following:
- GitHub Issues — Search for your error message in the godot-rl-agents repository.
- Godot Docs — For Godot-specific errors (physics, signals, etc.).
- SB3 Docs — For stable-baselines3 algorithm details and hyperparameters.
- Course units — Each unit includes detailed examples and common pitfalls in its own Troubleshooting sections.
- Logging & debugging — Add print statements to both Godot and Python, and use TensorBoard to visualize training metrics.
Before opening an issue, please provide:
- Your Python version (python --version)
- Your Godot version
- The exact error message (full traceback)
- Your training script (sanitized)
- A minimal reproducible example (if possible)