Haga: technical methodology and usage flow

This post walks through how Haga verifies physical-AI systems today: the exact pipeline from candidate intake to published results, and what teams need to provide for a reproducible benchmark.

What counts as evidence

Haga’s primary evidence comes from simulation-based evaluation, not hardware-only testing. That matters because simulation lets us run the same benchmark repeatedly with controlled perturbations, which is what reproducibility depends on.

Pipeline overview

1. Intake — teams submit the policy/world-model bundle they want evaluated.
2. Environment selection — we choose the task suite and simulation backend.
3. Stress passes — randomized seeds, contact-model sweeps, and terrain/lighting perturbations.
4. Physics consistency checks — we look for frame-to-frame plausibility violations.
5. Reporting — numeric summaries with variance, not single-point scores.

What we expect from teams

• Model load instructions or deterministic checkpoint
• Task definitions in the agreed task format
• Any simulator-specific assets required for rerun
• Acceptance criteria if the project has safety thresholds

The goal is that another team can take the published report and rerun the same benchmark without calling us.

Outputs

• Pass/fail summary per task variant
• Reproducibility notes on simulator version and seeds
• Physics consistency score and common failure modes

How to start

Evaluation intake is open at haga.mushoodhanif.com. If your benchmark numbers are used for funding, procurement, or safety reviews, independent verification should happen before those numbers become commitments.