Christopher Thierauf, Ravenna Thielstrom, Bradley Oosterveld, Will Becker, Matthias Scheutz. ACM Transactions on HRI.
How can robots handle self-corrections in human commands?
Corrections are a fundamental part of human communication. If I say “go to the red table — actually, no, the blue one,” you instantly know to drop the first plan and update to the second. A robot, on the other hand, usually needs explicit cancels or starts over completely. That gap makes interactions brittle and frustrating.
Our contribution here was to build methods that let robots treat “actually…” as more than noise (that’s what we currently do for disfluencies: sounds like “um” or “uhhh”). Instead, the robot detects that this is a correction, stops what it’s doing, and replans around the updated instruction.
How we approached it
We broke the problem into two main cases:
- Corrections during teaching.
Here, a human is walking the robot through a new action sequence (“first, go to the table… actually, the counter”). Instead of storing both the wrong and corrected actions, the robot prunes the script so only the intended sequence is kept. - Corrections during execution.
This is harder. If the robot is already carrying out a plan and a correction arrives, it has to decide:- Can I just stop and switch? (e.g., stop driving to Alpha desk, head to Beta instead)
- Or do I need to undo changes I’ve already made? (e.g., I’ve already put the soda on the plate, but now the meal requires grapes — so I need to remove the soda before continuing).
We used DIARC, our cognitive robotic architecture, to implement this, since it already has the plumbing: natural language parsing, task planning with PDDL, and action execution that can be halted and replanned.
What this looked like in practice
We tested our system on two very different robots: a mobile manipulator (the Fetch) and an industrial arm with a conveyor. Scenarios ranged from simple navigation corrections (“go to Beta desk — no, Alpha desk”) to meal assembly tasks where mid-task changes forced the robot to undo and redo parts of its plan.
To validate the approach, we also ran an online study: people were given the same instruction-and-correction scenarios and asked what they thought should happen. The robot’s choices matched human expectations over 90% of the time — a strong signal we were on the right track.
Why this matters
A lot of work on language in robotics focuses on reward shaping, learning policies over time, or handling disfluencies. But corrections aren’t about vague reinforcement or noisy speech. Instead, they’re about explicit revisions of intent, which means the robot has to integrate language understanding, task planning, and execution monitoring in real time.
By showing that robots can handle corrections systematically, and not just at the parsing level but at the behavior level, we open the door to smoother, more natural collaborations. The robot doesn’t just “listen better”; it behaves more like a competent partner.