On-device extraction
3D pose and 6-DoF object pose are extracted on site by on-device AI. Raw video never leaves the store and is in no dataset we license, and no audio is captured.
Human behavior data for Physical AI, captured from real customers in real stores — nobody recruited, nothing staged.
736,560 episodes of hand–object interaction a year, from one self-checkout lane — 2,728 hours. Real customers scanning their own items, recorded from fixed multi-view cameras. Grasp, transport, scan and retry differ from person to person, and failures and recoveries occur at their natural frequency.
A recorded lane session becomes episodes here: grasp and release are marked per hand, the stream is cut at those boundaries, and every frame carries a measured 3D hand and object pose.
Not a list designed for demonstration — behaviors were observed first, then classified.
ApproachHand enters toward the item in basket or cart
GraspOne-hand, two-hand, re-grasp included
LiftTrajectories by item weight and shape
Hand-overItem passed from one hand to the other mid-motion
TransportPath and speed to the scanner
ScanBarcode presentation pose and angle
RotateSearching for barcode orientation — branch after failure
Re-graspSlips, two-hand assist and re-alignment
Re-scanRetry after a failure signal
PlacePlacement onto bagging area or bag
and countingA class is added when a behavior shows up often enough to label. The list follows the data, not a shot list.
One clip moves through four stages at once, so you can see frame by frame where a motion holds and where it breaks. Each stage hands off a file, not a screenshot.
Third-person capture is where the data starts, not where it ends. Replayed in simulation, one episode also renders the views the robot itself would have had — per embodiment, without mounting a single new camera in a store.
3D pose and 6-DoF object pose are extracted on site by on-device AI. Raw video never leaves the store and is in no dataset we license, and no audio is captured.
Collected only at sites with a data-use agreement and customer notice in place. Store and brand identifiers are anonymized.
Commercial license for model training and evaluation. No redistribution or resale. Academic collaboration on separate terms.
Sample subsets through gated access on request. Full datasets delivered to a dedicated bucket after contract, with versioning and a change log.