Reinforcement Learning

Kangaroo’s locomotion controllers are trained using reinforcement learning (RL) rather than hand-tuned by hand. A policy is trained in simulation to map robot observations (joint states, IMU, commanded velocity, etc.) to joint commands, and is then deployed on the real robot.

mjlab

Training is done with mjlab, a GPU-accelerated RL framework built on top of MuJoCo. It follows the same environment/task API popularised by NVIDIA Isaac Lab, but runs on MuJoCo’s physics engine instead of Isaac Sim. This lets thousands of environments be simulated in parallel on a single GPU, which is what makes training locomotion policies from scratch practical.

Because Kangaroo is also simulated with MuJoCo (see MuJoCo Simulation), a policy trained in mjlab uses the same physics and contact model it will later run against in simulation, minimising the sim-to-sim gap before moving to the real robot.

pal_mjlab

Kangaroo-specific mjlab environments — rewards, observations, terrain, and domain randomization tailored to the robot — live in the pal_mjlab repository. Start there for training or modifying locomotion policies for Kangaroo; its README covers setup and usage.