Yixuan Wang

Fun fact – this work is also recognized as the best embodied AI poster at Michigan AI symposium - an amazing and fun event happened at my alma mater 💙💛

Thanks to my amazing collaborators - Leonor, Tarik, Jiuguang, and Yunzhu!! This project is impossible without their support!! I also want to thank a lot of amazing folks from Boston Dynamics AI Institute and it has been an amazing experience intern experience! (9/9)

⬇️ Links to our project. Stay tuned for the code release!

🔗 Website: curiousbot.theaiinstitute.com
📷 Video: youtu.be/1fK9-OrSwpQ
📄 Paper: arxiv.org/abs/2501.13338

(8/9)

How well does our system work? We conduct failure analysis and breakdown failure reasons. We found that perception, decision, and action execution are still major failure reasons, which we want to address in the future. (7/9)

What does the system look like? We build a perception module upon visual foundation models and SLAM to build the actionable 3D relational object graph. Then we serialize graphs and input into foundation models to make decision and execute low-level robot skills. (6/9)

We show that our system can explore diverse environments, such as house-like environments and deformable object, and deploy various robot skills, including checking bottom, opening, lifting, pushing, and flipping. (5/9)

Why bothering to build the actionable 3D relational object graph?
Imagine you want your robot to collect toys spreading and being hidden in the house. This representation can not only guide robot to find all toys but also be used to collect all toys into its blanket. (4/9)

Inspired by human example, we build an **actionable 3D relational object graph** to (1) reason object relations and (2) decide actions for exploration. This clip shows how robot (1) localize unknown spaces and (2) execute skills such as opening, lifting, and pushing. (3/9)

How does human interactively explore the environment?
Human see – we often understand object relations first, such as the space **inside** the cabinet or **behind** the chair.
Human do – then we apply actions to reveal the unknown space, such as opening or pushing. (2/9)

🤔Active robot exploration is critical but hard – long-horizon, large space, and complex occlusions. How can robot explore like human?
🤖Introducing CuriousBot, which interactively explores and builds actionable 3D relational object graph.
🔗https://curiousbot.theaiinstitute.com/
👇Threads(1/9)

I am trying to create a robotics and ai starter pack on bluesky: go.bsky.app/DfAoaJ1

Very incomplete please comment with suggestions (or just if you're missing and want to be added!)

Thanks!

Thank you Chris for the great list! I am a PhD student at Columbia working on robotics. Could you please add me to the list? Thanks!

A starter pack of starter packs:

Robotics and AI go.bsky.app/DfAoaJ1
Computer Vision go.bsky.app/PkAKJu5
Computer Graphics Research go.bsky.app/ckQ1u9
Grumpy Machine Learners go.bsky.app/6ddpivr
Reinforcement Learning go.bsky.app/3WPHcHg