RL Policies Execution

Alongside the model-based controllers, Kangaroo can be driven by locomotion policies trained with reinforcement learning (see Reinforcement Learning). These policies are trained in simulation with pal_mjlab, exported to ONNX, and deployed on the real robot through the pal_policy_deployer framework.

The pal_policy_deployer Framework

pal_policy_deployer is a modular ROS 2 framework for deploying trained machine-learning policies (e.g. ONNX models) on a robot. The two packages relevant to Kangaroo are:

Package	Responsibility
pal_policy_deployer	Installation entry point. Contains the launch files and the trained models, and ties the other packages together.
pal_policy_inferer	Runs the policy inference on the observation vector and publishes the resulting commands.

Trained models are placed under pal_policy_deployer/models/, together with their metadata (observation order, joint structure) and the YAML configuration for the robot-specific settings.

The kang_mjlab Inferer Node

pal_policy_inferer provides an implemented node — kang_mjlab_node — that is specially fitted for the policies trained on pal_mjlab. Most of the information the node needs to run a policy is read directly from the metadata embedded in the ONNX model and then configured automatically. This includes fields such as joint_names, default_joint_pos, action_scale, observation_names, joint_stiffness, and joint_damping, so a policy can be swapped simply by pointing the node at a different model.

At runtime the node builds its observation vector from the robot data published on /controller_manager/introspection_data (together with commands such as /cmd_vel for locomotion policies), runs inference, and writes the resulting joint commands to the controller command topics.

ROS 2 Interfaces

The tables below list the topics, actions, and services exposed by the kang_mjlab_node.

Subscriptions (inputs used to build the observation vector):

Topic	Purpose
/controller_manager/introspection_data/names	Names of the introspected robot signals (published once / on change).
/controller_manager/introspection_data/values	Values of the introspected signals; their arrival triggers a new inference step.
/cmd_vel	Commanded base velocity for locomotion policies.
/odom	Odometry, subscribed only when the observation set includes a base linear-velocity term.

Publishers (outputs):

Topic	Type	Purpose
/subscriber_controller/desired_state	control_msgs/msg/MultiDOFCommand	Joint commands produced by the policy. Remapped by the launch file to /rl_impedance_controller/reference.
pal_policy_deployer_statistics	pal_statistics_msgs (introspection)	Diagnostics such as inference elapsed times and command values.

Actions:

Action	Type	Purpose
~/switch_policy (i.e. /pal_policy_deployer/switch_policy)	pal_policy_inferer/action/SwitchPolicy	Switches the active policy by policy_index. See Multi-Policy Architecture.

Launching a Policy

A policy is launched by passing its model tag to the deployer launch file:

ros2 launch pal_policy_deployer kang_mjlab_policy_deployer.launch.py model:=v636

By default, the second index (index 1) is the jumping (imitation) policy, and additional policies loaded alongside it take the following indices.

Launch Arguments

The kang_mjlab_policy_deployer.launch.py file exposes one argument per policy slot, so both the primary (locomotion) policy and the secondary (imitation) policy can be selected without editing any file:

Argument	Default	Description
task	loco	Task name of the primary policy (index 0).
model	v315	Model version of the primary policy.
imitation_task	imitation	Task name of the secondary policy (index 1).
imitation_model	v719	Model version of the secondary (imitation) policy.
imitation_frames	147	Frame budget the imitation policy runs for before automatically reverting to the primary policy.
robot_type	lower_body	Hardware configuration; selects the controller config config/<robot_type>.yaml.
use_sim_time	false	Use the simulation clock.

For example, launching with model:=v636 keeps the imitation slot at its default (v719) and only replaces the primary policy with the forward stair-climbing model.

How the Launch File Defines the Policies

Internally the launch file turns those arguments into a list of policies and a set of per-policy parameters passed to the kang_mjlab_node. Each policy is declared by a name in a policies list, and its configuration is nested under that name:

policies: ["loco", "imitation"]   # order defines the switch index (0, 1, ...)

loco:
  task:  loco
  model: v315

imitation:
  task:   imitation
  model:  v719
  frames: 147

The position of a name in the policies list is exactly the policy_index used by the switch action: loco → 0, imitation → 1.

Before the node starts, the launch file resolves each policy to a model directory of the form:

<pal_policy_deployer_share>/models/<task>/<task>_<model>_mjlab/

and verifies that the directory contains at least one .onnx file, so a missing or mistyped model fails fast at launch time. It also loads the rl_impedance_controller from config/<robot_type>.yaml and remaps the node’s desired_state output onto /rl_impedance_controller/reference.

Mandatory Parameters per Task Type

The parameters that must be defined depend on the task type of the policy:

Task type	Mandatory parameters	Purpose
loco (locomotion)	task, model	Selects the task family and the ONNX model to run.
imitation	task, model, frames	In addition to task/model, frames sets the frame budget after which the policy auto-reverts to the primary (locomotion) policy so the robot can stabilize.

In other words, task and model are required for every policy regardless of type, while frames is required only for imitation tasks — it is what gives the imitation policy its bounded, self-terminating behavior.

Available Policies

The policies are grouped by the task they solve. Each is identified by its version tag.

Walking on Flat Terrain

Flat-terrain locomotion is handled by policies v315 and v534. These are the default policies for commanding Kangaroo to walk on level ground.

Climbing Stairs

Stair climbing is split by direction of travel:

Policies	Task
v619, v630	Climb stairs backward
v636, v639	Climb stairs forward

Jumping (Imitation)

Jumping is realized through an imitation policy, v719.

Multi-Policy Architecture

The deployer can load more than one policy at a time and switch between them at runtime. This is what enables combining a task policy with a stabilizing policy.

For imitation tasks (such as jumping with v719), it is recommended to launch the imitation policy together with a flat-terrain policy. The imitation policy performs the task, and once the task is finished the deployer switches back to the flat-terrain policy so it can stabilize the robot after the maneuver.

Policies are switched through an action. Each loaded policy has an index, and the goal selects which policy to activate by that index:

ros2 action send_goal /pal_policy_deployer/switch_policy \
  pal_policy_inferer/action/SwitchPolicy "policy_index: 1"

Here policy_index defines the index of the policy to switch to among the loaded policies. In the imitation setup above, sending the goal switches into the imitation policy to perform the task; after it completes, control returns to the flat-terrain policy for stabilization.