Description
This virtual lab explores the idea of knowledge-based servoing, a paradigm in robotic control where reactive motion is guided by symbolic reasoning directly within the motion control loop. This enables robots to make decisions based on semantic understanding rather than relying solely on raw sensor data. Inspired by the principles of visual servoing — which uses image features to guide robot movements — knowledge-based servoing instead operates on semantic features, enabling real-time, interpretable, and adaptable robot behavior even in dynamic and partially observable environments.
In the paper "A Modular Framework for Knowledge-Based Servoing: Plugging Symbolic Theories into Robotic Controllers", we introduced a modular control framework that allows symbolic theories to be "plugged into" robot controllers. The framework continuously reasons about task-specific semantic features, inferring and adjusting motion commands based on qualitative models derived from concepts like image schemata. Image schemata provide a powerful abstraction that captures the essential functional and spatial relationships in tasks, making the reasoning process both efficient and robust across variations in the environment. We demonstrated this approach by developing a symbolic, image-schema-based reasoner for the task of pouring liquids, which adapts flexibly to different situations such as standard pouring, draining mixtures, and scraping sticky materials — all while maintaining transparent and human-understandable decision processes. Figuratively speaking, this enables the robot to think about what movement it is executing and why, and to deliberately change its movement if the environment does not behave as expected.
The pouring experiment, demonstrated with a Toyota HSR robot, can be experienced and repeated in this virtual lab environment. You can observe how the symbolic reasoner monitors the pouring action, infers motion primitives like tilting or aligning containers, and reacts intelligently to spillage or the completion of the task — showcasing a real-time, knowledge-driven control system in action.
Videos
Software Components
- GISKARD: A framework for constraint- and optimization-based robot motion and planning control.
- Symbolic Rules and Semantic Interpreter.
- Defeasible Reasoner.
Authors
- Malte Huerkamp
Tel: +49 421 218 64020
Email: huerkamp@uni-bremen.de
Profile: https://ai.uni-bremen.de/team/malte_huerkamp/ - Kaviya Dhanabalachandran
Tel: +49 421 218 64000
Email: kaviya@cs.uni-bremen.de
Profile: https://ai.uni-bremen.de/team/kaviya_dhanabalachandran - Mihai Pomarlan
Tel: +49 421 218 64421
Email: mpomarlan@yahoo.co.uk - Simon Stelter
Tel: +49 421 218 64014
Email: stelter@cs.uni-bremen.de
Profile: https://ai.uni-bremen.de/team/simon_stelter - Prof. Michael Beetz PhD
Tel: +49 421 218 64001
Email: beetz@cs.uni-bremen.de
Profile: https://ai.uni-bremen.de/team/michael_beetz