Everyday Activity Robot Manipulation in an Interactive Learning Environment

Cognitive robotics is a driving force for the development of AI systems that can master complex tasks autonomously. These knowledge-based robotic systems are able to interpret their environment, thus allowing them to interpret the context and to understand vaguely formulated instructions. The skills to understand and develop such systems can best be taught using the robot control system as a structure that is then deconstructed into its components, enabling students to understand all relevant components as well as how they interact with each other.

One of the milestones of the European Robotics and AI Network (euROBIN) is for a robot to receive a parcel from a human, open, and empty it. Based on this scenario, we present a best practice tutorial on how to implement a solution for this task that integrates all necessary software components in the framework of the robot control process. In the context of this tutorial, we focus on knowledge representation and reasoning, planning, and the simulation framework (Mujoco), bringing these components together into a learning environment that – in the extended version – introduces the whole control process of Cognitive Robotics. The learning environment follows an immersive approach, using a physics-based simulation environment for visualization purposes that helps to illustrate the concepts taught in the tutorial.

Using Jupyter Notebooks in a Docker environment, our learning environment is easily accessible without having to install different software packages and is independent of the learners’ technical setup. The tutorial is a shortened version of the full one-week tutorial presented at the EASE Fall School 2023 and aims to introduce the teaching approach as a best practice example.





time October 5th, 2023
08:30 – 08:45 Opening Session – Michael Beetz, Jörn Syrbe  
08:45 – 10:00 Introduction – Michael Beetz  
10:00 – 11:00 Coffee break  
11:00 – 12:30 Hands-on Robot Control in cram – Arthur Niedzwiecki  
12:30 – 13:30 Lunch  
13:30 – 15:00 Hands-On robotics Simulation in multiverse – Giang Nguyen  
15:00 – 16:00 Coffee break  
16:00 – 17:30 Hands-On Knowledge openEASE – SaschA Jongebloed  
17:30 End


Intended audience

The tutorial introduces a teaching and learning approach that requires minimal technical prerequisites on the learners’ side while at the same time considering Cognitive Robotics on the system level. The learning modules are based on deconstructing and reconstructing all necessary system components and their required knowledge in order to foster a holistic understanding of how to develop cognitive-enabled robotic systems. Attendees of the tutorial will gain insights on how to develop cognitive-enabled robotics systems on a system level while learning how the required components work as a stand-alone as well as how they interact with each other. For this purpose, the tutorial provides hands-on exercises on knowledge representation and planning as well as an overview on how these modules can be combined in a simulation framework.
The workshop will be of interest to researchers, ECRs, learners and practitioners in the field of cognitive robotics who are looking to expand their knowledge and skills and to gain a deeper understanding of how to develop cognition-enabled robotic systems. The sessions will cover various aspects of cognitive robotics, including knowledge representation and reasoning, planning, and simulation, providing a comprehensive overview of the robot control process.

Prerequisite knowledge

The tutorial is independent of the operating system and requires only an installation of Docker.

Supplementary materials

Past tutorial content from the EASE Fall Schools is available at: https://github.com/IntEL4CoRo/ease_fall_school_2022


  • Code and tutorial available on Github
  • Video recording


  • Collaborative Research Center "Everyday Activity Science and Engineering (EASE)", funded by the German Research Foundation (DFG)
  • "Integrated e-Learning for Cognitive Robotics (IntEL4CoRo)", funded by the German Federal Ministry of Education and Research (BMBF)

Leave a Reply