Everyday Activity Robot Manipulation in an Interactive Learning Environment

Cognitive robotics is a driving force for the development of AI systems that can master complex tasks autonomously. These knowledge-based robotic systems are able to interpret their environment, thus allowing them to interpret the context and to understand vaguely formulated instructions. The skills to understand and develop such systems can best be taught using the robot control system as a structure that is then deconstructed into its components, enabling students to understand all relevant components as well as how they interact with each other.

One of the milestones of the European Robotics and AI Network (euROBIN) is for a robot to receive a parcel from a human, open, and empty it. Based on this scenario, we present a best practice tutorial on how to implement a solution for this task that integrates all necessary software components in the framework of the robot control process. In the context of this tutorial, we focus on knowledge representation and reasoning, planning, and the simulation framework (Mujoco), bringing these components together into a learning environment that – in the extended version – introduces the whole control process of Cognitive Robotics. The learning environment follows an immersive approach, using a physics-based simulation environment for visualization purposes that helps to illustrate the concepts taught in the tutorial.

Using Jupyter Notebooks in a Docker environment, our learning environment is easily accessible without having to install different software packages and is independent of the learners’ technical setup. The tutorial is a shortened version of the full one-week tutorial presented at the EASE Fall School 2023 and aims to introduce the teaching approach as a best practice example.