Independent research and ASE 389 · 2026–
SO-101 robot arms and objective inference
I built a leader–follower pair of SO-101 arms to record manipulation demonstrations. I’m developing a method to infer what a demonstrator wanted and plan new motions from it.
I assembled and calibrated both arms and set up LeRobot teleoperation with a front camera and a wrist camera.
Status: teleoperation and recording work. The inference method has no results yet.
Research question and planned method
Each LeRobot episode stores the leader’s joint commands, the follower’s joint positions and both camera streams. The SO-101 is an open-source arm design and LeRobot is Hugging Face’s open-source robotics library. My part is the build, calibration, integration, debugging, configuration, data collection and the method below.
Research question: can the robot infer the objective behind a demonstration, meaning the intended target and a few motion preferences, and use it to plan from a new starting position better than a planner given only the goal?
The plan is to fit the unknown objective so the recorded demonstrations look near-optimal under it, then plan a feasible motion for the follower from its own configuration. The approach draws on inverse-game methods from my graduate course.
J(ξ; g, w) = Σi wi φi(ξ) + terminal cost(ξ, g)
(ĝ, ŵ) = values of (g, w) under which the demonstrations ξdemo are near-optimal
- Unknowns
- Target g and a small weight vector w
- Objective family
- Weighted sum of motion-cost terms φi; the terms are still being chosen
- Constraints
- Motions must be feasible for the follower arm
- Baseline
- A planner given only the goal
- Evaluation
- Planning from new starting positions, compared with the baseline; metrics not yet fixed
