Chapter 03

Querying Knowledge

Knowledge Lab Back to overview

Welcome to the third Chapter of our hands-on course!

  • Knowledge Queries with KRROOD (KnowRob 3.0)

    Today, you will focus on understanding how a knowledge base supports robotic decision-making. You'll learn to query the robot's knowledge base using the Entity Query Language (EQL) to identify the actions required for a task, such as fetching the milk from the fridge.

    Goal: By the end of the session, you will have successfully made queries against the knowledge base to retrieve facts about the world, generate answers to underspecified questions, and query the robot's long-term memory, enabling the robot to determine the steps required to complete its tasks.

    Theoretical Background

    • We will provide an overview of how knowledge bases operate in the context of robotics, and how KRROOD reimagines them through object-oriented design.

    • You'll learn how the robot can work out where the milk is stored, where it should stand to reach the fridge handle, and how it remembers the whole episode afterwards.

    Knowledge Base:

    In this example, we use KRROOD (Knowledge Representation and Reasoning through Object-Oriented Design), the implementation of KnowRob 3.0 and the successor to the earlier, Prolog-based KnowRob system. KRROOD is a Python framework that turns a Python package into a knowledge base: entities and their relationships are represented as Python classes and objects rather than abstract symbols, queries are written in Python itself, and inference rules are first-class citizens that integrate directly with the data model. The approach draws from the classical knowledge representation paradigms, Prolog, first-order logic, SQL, and description logics, while packaging them in the object-oriented style that the vast majority of today's developers already use. KRROOD is developed as part of the Cognitive Robot Abstract Machine (CRAM) monorepo, which integrates it with the semantic digital twin (the robot's world model), plan execution, perception, and probabilistic modeling. For more information, check out the KRROOD Documentation and the CRAM repository.

    From Ontologies to Object-Oriented Design

    Classical knowledge representation splits knowledge into two boxes: the TBox, general terminological knowledge ("a Door is opened via its Handle"), and the ABox, specific assertional facts ("Door3's handle is Handle7; Milk1 is inside Fridge1"). Earlier versions of KnowRob encoded this split in OWL ontologies derived from the DUL foundational ontology, complemented by domain ontologies such as SOMA. KRROOD keeps the split but changes the encoding: the TBox becomes Python class definitions, and the ABox becomes Python object instances. The same fact that Prolog states as inside(milk1, fridge1), OWL as an individual with an inside_of property, and SQL as a table row, KRROOD states as InsideOf(body1="milk1", body2="fridge1").

    Concretely, the robot's environment lives in the semantic digital twin: a world model parsed from URDF whose bodies carry semantic annotations,  Python classes such as Fridge, Milk, ShelfLayer, Door, Handle, or the mixin IsStorageSpace. These annotations are not all asserted by hand. The WorldReasoner runs rule trees (Ripple Down Rules, themselves written as EQL queries) over the world's kinematic structure and infers annotations recursively: bodies become handles, handles fixed to prismatic bodies become drawers, drawers grouped by parent become a wardrobe,  and the same chain infers doors and fridges. Wrong or missing answers become prompts to an expert, whose answers are stored as new, versioned rules, so the reasoner grows with every correction.

    The Entity Query Language (EQL)

    EQL is KRROOD's pythonic relational query language. A query is built from a handful of pieces: variable(...) declares what you are looking for and over which domain, entity(...) marks it for selection, .where(...) adds constraints, and quantifiers such as the(...), an(...), and a(...) wrap the whole into an evaluable query. Plain Python functions can join in as symbolic functions and predicates: called with concrete values, they behave normally, but called with a variable inside a query, they defer execution and are evaluated per candidate binding. The same query runs unchanged against different backends,  walking live Python objects in memory, compiled to SQL via ORMatic, or sampling a learned probabilistic model,  and every query can verbalize itself, rendering its meaning in plain English so you can check that the robot is asking what you think it is asking.

    Underspecified Queries: Generating Instead of Filtering

    Some of the questions a robot must answer have no stored answer at all. "Which storage space contains the milk?" can be answered by filtering existing objects,  but "where should the robot stand to open the fridge?" has infinitely many candidate answers, none of which exist in the world model. For these underspecified questions, EQL offers generative queries: a(Point3)(x=..., y=..., z=...) does not filter existing candidates; it samples new values for a type's fields from a probabilistic backend, with any fields you pin down (say, the height of the handle) held fixed and the rest (x and y) left open. Left completely unconstrained, most samples are useless,  points inside walls or across the room. The fix is to constrain the distribution: a navigation map derived from the world's geometry yields the free space around the target, which can be used either to filter samples after the fact (rejection sampling) or, far more efficiently, translated into a where condition in disjunctive normal form that is pushed directly into the generating distribution, so that only feasible standing positions are ever produced.

    NEEMs: Long-Term Memory

    Everything the robot does may be worth remembering, so it can later learn from it. In earlier KnowRob versions, episodic memories (NEEMs,  Narrative-Enabled Episodic Memories) lived in a separate representation from the world model. In KRROOD they reuse the very same classes: arbitrary objects, such as the plan the robot just executed, are persisted into a relational database through ORMatic's data access objects (to_dao), and the resulting long-term memory is just another EQL backend. You query past episodes with the same the(...)/a(...) syntax you use against the live world, and a stored plan is not merely inspectable data,  from_dao() reconstructs the full plan object, which can be treated like working memory again and resumed. Because episodes are ordinary queryable objects, hundreds of recorded episodes can also become training data for the probabilistic models used in generative queries: the robot learns from its own history, with no translation step in between.

    Step-by-Step Hands-On Exercises

    1. Build the world: Load the kitchen environment from URDF into the semantic digital twin, spawn the shelf layer and the milk inside the fridge, and run the WorldReasoner to infer the semantic annotations of the scene.

    2. Query the Knowledge Base: Implement contains(container, body) as a symbolic function on top of the InsideOf spatial predicate, then ask the knowledge base which IsStorageSpace annotation contains the milk. The answer,  the fridge,  hands you the handle of its door: the target the robot must reach.

    3. Underspecified Queries: Ask "where should the robot stand?" as a generative query, sampling Point3 positions at the handle's height. Implement the IsFree predicate against the navigation map's free space, observe rejection sampling, then push the free-space condition directly into the distribution and inspect the verbalized query.

    4. Plan Execution: Based on the query results, park the robot's arms and navigate it to the sampled standing position in front of the fridge.

    5. Long-Term Memory: Persist the executed plan into an in-memory database, query the long-term memory for the stored Plan and for every Body encountered during the experience, and finally reconstruct the plan with from_dao() so it could be resumed.

    Throughout the exercises, code examples demonstrate how to perform the queries in Python; several cells are left as tasks for you to fill in before revealing the solution.

    Interactive Actions and/or Examples

    For the Hands-On Exercises, please use the accompanying notebook (ijcai_demo.ipynb).

    Summary

    By the end of the session, you will have a robot that works out for itself where the milk is stored, generates a feasible place to stand, navigates there to open the fridge,  and remembers the whole episode in a queryable long-term memory it can learn from and resume. You will have learned how to query the knowledge base to determine the necessary actions for the robot to complete its tasks.

    Further Reading/Exercises

Authors and Contact Details

Back to top