Published on IEEE Transactions on Robotics.
We are excited to announce the first scientific publication from the TORNADO project, a key milestone since the project’s launch in January 2025.
In this study, our partner Istituto Italiano di Tecnologia (IIT) explores Shannon’s information theory to describe hand-objects information flow in video human demonstrations to retrieve automatically an executable plan for the robot replica.
Key Insights from the Research
The paper explores for the first time Shannon’s #Information #Theory to identify active scene elements and quantify the information shared between hands and objects. By exploiting scene graph properties, we encoded the extracted interaction features into a compact structure and segmented the demonstration into discrete blocks. Our framework then automatically generates #Behavior #Trees, that enables robots to replicate human actions efficiently.
Impact on the TORNADO Project
This work put the basis for some of TORNADO’s goals:
- More natural human-robot interaction: Robots learn from us the way we learn from each other, by watching, not programming.
- Smarter, more adaptable robots: Understanding why we do things helps robots adapt to new environments and tasks.
- Simpler teaching: A single video of a person doing a task becomes a complete robot instruction, no manuals, no code.
- Fast, flexible skill transfer: Speeds up how quickly robots can learn new jobs, helping in homes, hospitals, and workplaces.
- Accessible to everyone: Empowers non-experts to train robots, making automation more inclusive and user-friendly.
Thanks to Elena Merlo, Marta Lagomarsino, Edoardo Lamon, Arash Ajoudani.
Read More & Watch the Video
📄 Full Paper on IEEE Xplore: IEEE Xplore
📽️ Video Presentation: YouTube
