SARM2: Multi-Task Reward Modeling Enhances Robotic Manipulation Policy Improvement
This work introduces RM, a multi-task, stage-aware reward model that improves value estimation accuracy by 80% for robotic manipulation tasks. RM integrates an action-primitive-based stage estimator with an MMoE value head. The SPIRAL framework leverages RM for self-improvement of VLA policies using autonomous rollouts, achieving near-perfect success rates on complex tasks like Folding Shorts and Cleaning Whiteboard, crucial for developing a stable robot data flywheel.