More details and results in the paper, and stay tuned for the 2HANDS dataset & code release!
📄Paper: arxiv.org/abs/2503.09320
🌐 Website: sites.google.com/view/2handedafforder
Work done with Marin Heidinger, Vignesh Prasad & @georgiachal.bsky.social
See you in Hawaii at #ICCV2025! 🌴
We can then use our high-quality dataset to train or fine-tune a VLM that takes in the activity/task text prompt as input and predicts bimanual affordance masks (for a left and right robot hand)
4/5
We extract bimanual affordance masks from egocentric RGB video datasets using video-based hand inpainting and object reconstruction.
No manual labeling is required. The narrations from egocentric datasets also provide free-form text supervision! (Eg. "pour milk into bowl")
3/5
The Problem:
Most affordance detection methods just segment object parts & do not predict actionable regions for robots!
Our solution?
Use egocentric bimanual human videos to extract precise affordance regions considering object relationships, context, & hand coordination!
2/5
📢 PSA for the robotics community:
Stop labeling affordances or distilling them from VLMs.
Extract affordances from bimanual human videos instead!
Excited to share 2HandedAfforder: Learning Precise Actionable Bimanual Affordances from Human Videos, accepted at #ICCV2025! 🎉
🧵1/5
Thank you to all the speakers & attendees for making the EgoAct workshop a great success!
Congratulations to the winners of the Best Paper Awards: EgoDex & DexWild!
The full recording is available at: youtu.be/64yLApbBZ7I
Some highlights:
Learn more at the workshop website: egoact.github.io/rss2025
Happy to be organizing this with @georgiachal.bsky.social, Yu Xiang, @danfei.bsky.social and @galasso.bsky.social!
Call for Contributions:
We’re inviting contributions in the form of:
📝 Full papers OR
📝 4-page extended abstracts
🗓️ Submission Deadline: April 30, 2025
🏆 Best Paper Award, sponsored by Meta!
Core workshop topics include:
🥽 Egocentric interfaces for robot learning
🧠 High-level action & scene understanding
🤝 Human-to-robot transfer
🧱 Foundation models from human activity datasets
🛠️ Egocentric world models for high-level planning & low-level manipulation
📢 Excited to announce EgoAct 🥽🤖: the 1st Workshop on Egocentric Perception and Action for Robot Learning at #RSS2025 in LA!
We’re bringing together researchers exploring how egocentric perception can drive next-gen robot learning!
🔗 Full info: egoact.github.io/rss2025
@roboticsscisys.bsky.social
I'm working on Robot learning and perception : )
