Rodrigues Network for Learning Robot Actions
Jialiang Zhang, Haoran Geng, Yang You, Congyue Deng, Pieter Abbeel, Jitendra Malik, Leonidas Guibas
We design a new neural network, the Rodrigues Network (RodriNet), that addresses the kinematic structural priors in articulated robot action learning.
Abstract
Understanding and predicting articulated actions is important in robot learning. However, common architectures such as MLPs and Transformers lack inductive biases that reflect the underlying kinematic structure of articulated systems. To this end, we propose the **Neural Rodrigues Operator**, a learnable generalization of the classical forward kinematics operation, designed to inject kinematics-aware inductive bias into neural computation. Building on this operator, we design the **Rodrigues Network (RodriNet)**, a novel neural architecture specialized for processing actions. We evaluate the expressivity of our network on two synthetic tasks on kinematic and motion prediction, showing significant improvements compared to standard backbones. We further demonstrate its effectiveness in two realistic applications: (i) imitation learning on robotic benchmarks with the Diffusion Policy, and (ii) single-image 3D hand reconstruction. Our results suggest that integrating structured kinematic priors into the network architecture improves action learning in various domains.
Rodrigues Networks inject kinematics-aware inductive biases for improved action learning in articulated robot tasks.
- Neural Rodrigues Operator extending classical forward kinematics into learnable operation
- Rodrigues Network architecture specialized for processing articulated actions
- Significant improvements on kinematic and motion prediction synthetic tasks
- Effectiveness demonstrated on imitation learning and 3D hand reconstruction applications
- Neural operators
- Kinematic constraints
- Rodrigues rotation formula
- Diffusion Policy
Networks do not account for geometry of individual links that could improve contact reasoning
from the paperCurrent operator restricted to rotational joints, not translational joints
from the paperRobot learning experiments focused on imitation learning rather than reinforcement learning
from the paper
Incorporate geometry of individual links
from the paperExtend to translational joints
from the paperExplore reinforcement learning scenarios for generality testing
from the paper
Author keywords
- Robot learning
- Action understanding
- Neural architecture
Related orals
Mastering Sparse CUDA Generation through Pretrained Models and Deep Reinforcement Learning
SparseRL leverages deep RL and pretrained models to generate high-performance CUDA code for sparse matrix operations.
Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling
DECS framework reduces reasoning model overthinking by decoupling necessary from redundant tokens via curriculum scheduling.
MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
MemAgent uses RL-trained memory modules to enable LLMs to extrapolate from 8K to 3.5M token contexts with minimal performance degradation.
DiffusionNFT: Online Diffusion Reinforcement with Forward Process
DiffusionNFT enables efficient online reinforcement learning for diffusion models via forward process optimization with up to 25x efficiency gains.
Hyperparameter Trajectory Inference with Conditional Lagrangian Optimal Transport
Hyperparameter Trajectory Inference uses conditional Lagrangian optimal transport to reconstruct neural network outputs across hyperparameter spectra without expensive retraining.