Work Monitor
Review task completion and the scenes supporting the assessment.
Early prototype · pre-generated with frontier models. No live analysis or robot control runs here.
Attempted task
Push the object to the target position.
Video evidence
FAILURE · Inceoglu et al. · MIT (repository)
16 published RGB frames encoded at 4 fps. Redistribution terms for the original dataset need further confirmation.
Scenes supporting the assessment
Technical details
Internal codes from the saved model assessment. Numerical model estimates are not presented as calibrated probabilities.
- Verdict code
INCORRECT- Phase code
FAILED- Advisory code
RETRY
Evaluation method and limitations
Success/failure classification on 100 task–video pairs
Recorded accuracy: 89% (89/100). This is agreement with dataset ground truth for terminal task success/failure on this frozen sample—not a field success rate or confidence in an individual assessment.
| Dataset | Setting | Success | Failure |
|---|---|---|---|
| FAILURE | Real Baxter | 16 | 16 |
| RoboFAC | Simulation | 17 | 17 |
| ViFailback | Real ALOHA | 17 | 17 |
Task instructions and videos of object-manipulation work were evaluated. Sampling was stratified by dataset, task family, and label with a fixed seed; labels were used only for scoring after inference. The two public demos are selected examples from this sample.
Results recorded September 4, 2026, without task-specific fine-tuning. Evidence quality, diagnosis, progress, recovery suggestions, and safety were not validated by this metric and require separate evaluation.
Download evaluation metadata ↗