Théo Hermann, independent practitioner. MSc Robotics, EPFL. Perception and autonomy work at EPFL VITA Lab, IRI Barcelona and Roboat, 2022 to 2024.
One example, on a public recording: the robot was in one corridor, and reported, sure to a few centimetres, that it was in another.
When an autonomous robot fails in ways your logs don't explain, I find the real cause and prove it, independently.
Your tools collect the data; I deliver the verdict and the evidence your team and, from 2027, your safety assessors will ask for.
The cause: after about three minutes on a floor its map did not contain, it decided it was in the wrong corridor and stayed there.
Black: how far its reported position was from the ceiling markers. Grey: how sure the robot said it was. After the elevator, black jumps to metres; grey stays at centimetres.
A PR2 in MIT's Stata Center, from a public dataset, replayed and graded by me. Source
If your monitoring trusts the robot's own uncertainty
On the recording behind that example: the check most teams write first, tuned in hindsight, against mine, at the same rate of alarms while the robot was not lost (3%).
The evidence
| Check | Flagged while fine | Flagged while lost |
|---|---|---|
| A threshold on how sure the robot says it is of its position, picked knowing the answer | 3% | 46% |
| Four detectors (sudden jumps in that uncertainty, jumps in position, disagreement with the wheels, gaps in the sensor stream), frozen before the run | 3% | 69% |
One recording, one robot, graded against markers on the ceiling (the dataset's AprilTags). The 69% is an upper bound (57% counting only detections that start in the lost window), and neither check flagged 26% of the positions it reported while lost. Source
If your alarms were tuned on one robot
An alarm on the robot's own uncertainty, set on a simulated TurtleBot3, fired five times on a real Tiago robot in a recording where nothing was wrong.
The evidence
A threshold belongs to the machine it was measured on, so I set yours on your recordings and measure how often each fires when nothing is wrong. Source
If you run Nav2, check this in one minute
Stock Nav2 ships AMCL, the part that works out where the robot is, with both recovery terms at 0.0: a robot that gets moved or lost has no way to notice, unless you change them.
The evidence
If both print 0.0, a displaced robot has no mechanism to conclude it is lost.
ros2 param get /amcl recovery_alpha_slow
ros2 param get /amcl recovery_alpha_fastHow it works
- You send one recording from an incident (the log file the robot writes, a ROS 1 or ROS 2 bag) and a line on what the robot did.
- I read it offline. Analysis runs on your infrastructure or under NDA on mine; nothing is retained after handover. My tool reads the recording from disk and calls nothing outside it.
- You get a written account: what happened, what the evidence supports, and what it does not.
The written account of the public example above, first page. Read it in full
Before each software release, the same checks run on your recordings of the old version and the new one, and you get in writing what changed.