Holo Engine on RoboDojo
Benchmark results for a dual-arm ARX X5 in our CPU-based MuJoCo replica. HoloLang selects the robot’s actions, with Holo Reason supporting recognition.
Interim results ·
Leaderboard comparison
The overall score weights five benchmark dimensions equally. The chart compares our local MuJoCo run with selected official Isaac Sim entries.
This score reflects the current Holo policy and the layouts tested so far. We’ll report new results as we refine the policy and run further evaluations.
How Holo learns and acts
We prepare examples and visual references from RoboDojo data. During each attempt, Holo Engine uses those references and fresh measurements to select the next skill and, where the skill calls for it, fit a rule for the movement.
Before evaluation
Expert demonstrations, public layouts and 3D object models.
Measure grasp and release examples. Render object views and silhouettes.
Save motion examples, task parameters and visual references.
Task knowledge feeds perception and Holo Engine during the attempt.
During each attempt
RGB views, arm poses, gripper opening and the task instruction.
Extract visual features and estimate object geometry from camera views.
- Holo Reason identifies objects from reference examples.
- HoloLang tracks progress and selects the next skill.
- Function Tree fits and evaluates movement rules for selected skills.
Build arm and gripper trajectories for the selected action.
- Local evaluation hardware
- MacBook Pro
Intel Core i9-9880H · CPU only - HoloLang decision time
- 431 ms on average
- Holo Engine peak RAM
- 88.1 MB
Peak RAM. We measured the peak resident memory of the Holo Engine process across 7 sampled tasks on CPU, starting a fresh process for each task.
HoloLang. Timing covers 2,397 logged decisions across 52 configurations, including Function Tree when called within a decision.
Function Tree. Fitting and evaluating the block stacking rule took 23 ms on average across 30 calls.
Holo Reason. In separate measurements, it took 1.56 s for two digits, 5.19 s for five objects and 21.66 s to match a text description against the full reference library. Each figure averages 3 requests.
All 42 tasks
Select a task to see the recorded run. Where a task has a randomized variant, its row averages the standard and randomized configurations.
Recorded runs
Watch the best recorded Holo attempt for each tested configuration. Select “Watch RoboDojo demo” to play the official reference video. The reference streams online, and its layout may differ from the Holo run.
Method and limits
- Evaluation. We used RoboDojo’s public layouts, assets and scoring code in our MuJoCo replica. The physics and rendered camera views come from the replica. An official result requires evaluation by RoboDojo.
- Coverage. This snapshot uses seed 0 and 3–10 attempts per tested configuration. Cloth is absent from the replica, so both clothes-folding variants were omitted and the task is scored as zero. Fluid simulation is also missing.
- Task mechanics. The conveyor uses the published dimensions and speed. The sorting demonstration follows the recorded order and grasp timing. The third arm is absent. Tic-tac-toe adds the opponent’s mark after the original number of steps. Opponent play in mahjong is incomplete.
- Perception. We built visual references for the open tasks from the replica’s images. Transfer of that recognition to Isaac Sim imagery remains unverified.