CARD:Contact-Anchored Retargeting
for Dexterous Manipulation

CARD extracts, preserves, and tracks contact force targets from human demonstrations.

Abstract

Humans are dexterous, but traditional data collection paradigms such as teleoperation, kinesthetic teaching, and wearable exoskeletons force humans to act through the robot embodiment, bottlenecking their dexterity. Retargeting can avoid this. However, existing methods purely translate kinematics and ignore the real-world contact dynamics essential for task success. We present CARD: Contact-Anchored Retargeting for Dexterous Manipulation, a simulator and object-free framework that retargets human demonstrations to robot actions by explicitly preserving contact force targets. CARD unifies human data sources including video, motion capture, and tactile gloves via a shared hand–contact representation and generates explicit contact force targets in addition to kinematics. To maintain these contact targets during hardware open-loop replay, CARD also introduces a tactile tracking controller. For video which does not contain an explicit tactile source, we present Vid-HACO, our video-to-contact model that sets a new state of the art. Across our video, glove, motion-capture data and the GRAB dataset, CARD maintains contact on 43–78% of contact frames, while kinematic and object-aware retargeting methods maintain contact on <10% and 22.5% of the frames respectively. Further, integrating our formulation into physics-based retargeting improves strict task success from 12% to 32% on GRAB dataset. Finally, we show that our tactile tracker is critical to reproduce demonstrated forces during hardware replay. These results establish that contact preservation via CARD is a scalable interface for transferring human dexterity to robots.

Cunxi Dai1,2, Botao He3,2*, Francois R. Hogan2*, Wei Pan2*, Rao Fu4,2, Wen-Han Hsieh1, Ruoshi Liu3,2, Jitendra Malik4,2, Guanya Shi1,2†, Akash Sharma2†

1Carnegie Mellon University   2Amazon Frontier AI & Robotics (FAR)   3University of Maryland, College Park   4UC Berkeley

*Core contributor   †Equal advising

Videos BibTeX Paper arXiv Code
Open-loop tactile-reactive replay
CARD

Overview video

Current retargeting methods ignore contact dynamics

Kinematic retargeting maps fingertip positions or joint angles from the human hand to the robot hand. CARD instead preserves each contact’s location, push direction and force, and places the corresponding robot link on it.

Coat hangercontact force targets
Cupcontact force targets

The five stages of CARD

Unified hand–contact representation

Each source becomes a MANO hand sequence with at most one contact force anchor C_f=(c_f,\hat d_f,F_f) per hand region and frame, attached to the hand rather than the object.

MANO hand with colored hand regions and one anchor arrow per region; the thumb anchor is labelled with its position c_f, push direction d_f and force F_fregions and anchors
Close-up of the thumb tip with its contact force anchor: the center of pressure c_f, the push direction d_f and the force F_f as the arrow lengththumb-tip anchor
One fingertip touching an object: the contacting pad vertices shaded by pressure, with outward normals; their pressure-weighted centroid c_f on the skin, their weighted mean normal d_f drawn as an arrow into the object, and the total pressure F_f as the arrow length

Closed-form geometric initialization

Finger flexion and abduction are transferred from the human skeleton in closed form, the thumb through a fixed affine map of its bone directions and pad normal, and the wrist pose is set so the robot fingertips align with the human’s. No optimization is involved.

Geometric initialization of the robot hand on a cup
CARD solution on the same cup
Geometric initializationAfter optimization

Sampling-based optimization

For every frame, CARD solves for the wrist pose and joint angles q:

q^{\star}=\arg\min_{q\in\mathcal Q}\;E_{\mathrm{contact}}(q)+E_{\mathrm{style}}(q)+E_{\mathrm{smooth}}(q)+E_{\mathrm{collision}}(q)

E_{\mathrm{contact}} puts each anchored robot pad on its contact force target, facing the push; the other terms handle free fingers, smoothness and collisions. CARD minimizes it with an annealed CEM sampler on the GPU.

Tactile tracking replay

Replay follows the retargeted joint targets, while each finger in contact closes a little further until its fingertip sensor reads the desired force; a feedforward torque adds that force directly.

Three sources, one representation

Each source outputs the same anchored MANO sequence, so one solver serves all three.

Phone video

HaWoR hand pose, Vid-HACO contact

Tactile glove

744-taxel pressure, wrist tracker

Motion capture + glove

OptiTrack markers over the tactile glove

Contact from video: Vid-HACO

Vid-HACO predicts per-vertex hand contact from 8-frame video clips with a V-JEPA 2.1 encoder, trained on 1.55M frames from 12 datasets.

HACO
Vid-HACO
Validation split
HRDexDB, held out

CARD keeps 43–78% of demonstrated contacts, kinematic baselines under 10%

186 demonstrations retargeted to the SHARPA Wave: our 86 clips (20 phone videos, 18 tactile-glove clips, 48 motion-capture-with-glove clips) and 100 GRAB clips. A contact is established when the anchor lies within 2 mm of the assigned link and is reached with its palmar face.

Our data
GRAB, 100 clips
Full table
CARDDexPilotAnyTeleop-KAnyTeleop-PReGrind (w/ object)CARD (init only)
Our data: monocular video (20 clips)
Contact rate (%)61.49.78.37.6–20.5
Correct finger (%)60.19.58.07.5–19.2
In cone (%)35.24.24.75.0––
Gap, median (mm)2.409.9910.8013.19–5.76
Our data: tactile glove + motion capture (66 clips)
Contact rate (%)77.73.83.43.5–15.9
Correct finger (%)76.93.73.33.2–15.8
In cone (%)46.01.11.02.3––
Gap, median (mm)1.3817.1716.6723.22–9.32
GRAB (100 clips)
Contact rate (%)43.14.94.94.622.519.6
Correct finger (%)40.64.84.94.520.717.8
In cone (%)16.21.71.71.75.23.8
Gap, median (mm)3.5620.9020.4520.667.8613.78
Penetration, median (mm)7.9612.7112.7913.029.8011.15

ReGrind takes the object mesh as input, so it runs on GRAB only. CARD uses a contact hull as the collision object there, and the GRAB mesh only to score penetration.

Open-loop replay on real hands

SHARPA Wave
0:00 / 0:00
Human demonstration

Contact force targets during a card flip

Same trajectory, with and without the tactile tracker

With the tactile tracker: hammer held
Tactile tracker on
Held · 8/10 trials
Position replay only: hammer slips
Tactile tracker off
Slipped · 1/10 trials

Holding contact on a moving, curved surface

Barrel moved by hand
Arm moves along the barrel
The arm carries the hand down the barrel’s curved side and turns the wrist by about 45°; the force loop keeps the block pressed against the surface.

A new hand needs only a semantic map

Each human hand region maps to a robot link and its grasping face; the anchors, costs and weights stay the same.

HammerGRAB demonstration
Human source grasp on a hammer
Human
SHARPA Wave
CubeGRAB demonstration
Human source grasp on a cube
Human
SHARPA Wave

Citation

@misc{dai2026card, title = {{CARD}: Contact-Anchored Retargeting for Dexterous Manipulation}, author = {Cunxi Dai and Botao He and Francois R. Hogan and Wei Pan and Rao Fu and Wen-Han Hsieh and Ruoshi Liu and Jitendra Malik and Guanya Shi and Akash Sharma}, year = {2026}, eprint = {2610.36539}, archivePrefix = {arXiv}, primaryClass = {cs.RO}, url = {https://arxiv.org/abs/2610.36539} }