Chapter 04

Completing the Full Transportation Task

To The Tutorial Back to overview

Welcome to the fourth day of our hands-on course!

Chapter 4: Completing the Full Transportation Task

Welcome to the fourth day of our hands-on course!

Today you will execute one complete task from end to end: transporting the milk from the fridge to
the table. This session integrates all the knowledge and skills you have developed so far — the
environment you modelled, the objects you perceived, and the knowledge you can query about them —
and adds the layer that turns them into robot behaviour: coraplex the high-level planning
framework.

Goal

By the end of the session you will have completed the full milk delivery task, and you will
understand how a single symbolic task description turns into a sequence of motions that a real or
simulated robot performs.

Prerequisites

  • Chapters 1–3 completed
  • A working understanding of URDF, perception, and the semantic digital twin
  • A running installation of the course workspace

The task

The whole chapter follows one task, the same one the presentation is built around:

> Bring me the milk.

The milk is inside the closed fridge, and it has to end up on the table. In coraplex this can be achieved by building a plan using the following Action designators: 

  • Open 
  • Navigate 
  • PickUp
  • Place 
  • Close

Everything else is left open on purpose, and each open question is answered by one part of the
framework:

  •  Where should the robot stand so that it can reach the milk? → location designators
  • Which of the possible stand poses should it take? → underspecification and grounding
  • Does the fridge have to be opened first? → the plan adapts to the current world state
  • Does this run in simulation or on the real robot? → the execution environment

Keep this list in mind: the rest of the chapter works through it from top to bottom.

What coraplex gives you

coraplex is the plan and action layer of the cognitive robot abstract machine. It separates
describing what a robot should do from grounding that description against the live world state
and executing it. In practice that gives you:

  • Task and action definition — actions are dataclasses that describe what should happen
  • Parameterisation of tasks and actions — the parameters are ordinary dataclass fields
  • Composition and structured execution — actions build plans from other actions, sequenced by
      the plan language
  • Pose inference — locations compute where the robot has to stand instead of you hardcoding it
  • Handling uncertainty and underspecified tasks — one description can stand for many concrete   actions, resolved lazily and retried on failure
  • The same plan in simulation and on hardware — the execution environment is the only thing that
      changes

Learning objectives

After this chapter you can explain what a designator is and how it differs from a command, read the
plan an action expands into, describe how a robot base pose is inferred, say what happens when a
grounded action fails, and run the same plan against both a simulated and a real robot.

Core concepts

The sections below follow the presentation in order.

Designators describe, they do not command

A designator describes an action or a location symbolically, without committing to concrete values.
`TransportAction` is a plain dataclass with four fields — the object, the target pose, the arm, and
the grasp description — plus one method that says what plan it expands into. It says the milk goes
on the table; it never says where to stand, or in which order to move.

Two families of actions

Actions live at three levels of abstraction:

  • Composite actions compose other actions: `TransportAction`, `PickAndPlaceAction`
  • Core actions bottom out in motions: `NavigateAction`, `PickUpAction`, `PlaceAction`,
      `OpenAction`, `CloseAction`, `ParkArmsAction`
  • Motions are the leaves that drive the robot: `MoveMotion`, `ReachMotion`, `MoveGripperMotion`

Your one `TransportAction` reaches through all three levels.

Building a plan

`TransportAction` expands into a sequence: open the fridge, park the arms, navigate to a pose from
which the milk is reachable, pick the milk up, park and raise the torso, navigate to a pose that
reaches the table, place the milk, park the arms. None of the poses in that plan are hardcoded —
every navigation target is itself a designator that is resolved later.

Adapting to the world

The first step exists only because of a question the action asks the world before it builds its
plan: is the milk inside something? If it is, navigation and opening are prepended; if the milk
stands on the counter instead, those steps simply are not there. There is no separate belief store —
the semantic digital twin is the belief state, so the plan that gets built already reflects what
is true right now.

Locations infer where the robot must stand

A location describes where the same way an action describes what. To reach the milk, an
occupancy costmap is combined with a ring around the target at the arm's reach distance, and each
candidate pose drawn from that map is validated for reachability in a copy of the world before it is
ever used. The result is not one pose but a lazy stream of candidate stand poses.

Underspecification: describing many actions at once

Instead of a single concrete action you can build a template whose fields range over a domain —
and the domain can be the location itself, not a hand-written list of poses. One template stands for
as many concrete actions as the combinations of its fields allow, so leaving the arm unspecified
multiplies the candidates by the number of arms.

Lazy grounding and failure handling

Nothing is grounded while the plan is being built. Only at execution time is a candidate pulled from
the stream and tried; if it fails, the next one is pulled and tried instead. This matters for our
task: a stand pose only counts as reachable once the fridge is actually open, which is a state an
earlier step of the very same plan produced.

The plan graph

All of this produces one tree that mixes node kinds: language nodes sequence their children, action
nodes wrap a designator and expand into their own sub-plan, an underspecified node holds a stream of
candidates, and motion nodes are the leaves. Parts of the tree do not exist until execution reaches
them.

Notify, parse, execute

Performing a plan runs in three stages: notify expands and grounds the subtree, parse
compiles the now-concrete subtree into an executable, and execute actually runs it. Building a
plan is cheap and side-effect free; only execution moves the robot or changes the world.

Simulation or the real robot

The same plan object runs in both. Entering `with simulated_robot:` or `with real_robot:` switches
the execution environment, and only the lowest-level call branches on it — everything to the left of
that branch is identical.

Following the presentation

Slide  Topic Section Above
2 The task: bring me the milk  The task
3 What is a designator?  Designators describe, they do not command 
4 Two families of actions  Two families of actions 
5 Designators in use: building a plan  Building a plan 
6 Adapting to the world  Adapting to the world 
7 Locations are designators too  Locations infer where the robot must stand 
8 Underspecification  Underspecification: describing many actions at once 

9

 Resolving it: lazy, one candidate at a time Lazy grounding and failure handling 

10

 The plan graph  The plan graph 

11

 Plans: parsing vs. execution  Notify, parse, execute 

12

 Same plan, simulation or real robot Simulation or the real robot 

13

 Recap  Summary 

 

The environment

You already got to know the semantic digital twin in the previous chapters and built a small
environment there. In this chapter we use a more complex environment: the kitchen we have in our
lab. The general setting and the task stay the same. There is a fridge which contains the milk,
which the robot has to fetch and bring to the table. The robot has to open the fridge, take the
milk, and close the fridge again.

Before you start, make sure you can answer three questions about that world: which body is the milk,
which body is the fridge door handle the robot has to grasp, and where is the table pose the milk
should end up on.

Step by step

  1. Inspect the world Load the kitchen, find the milk and the fridge, and confirm the milk is
       inside the fridge.
  2.  Describe the task Build a `TransportAction` for the milk and the table pose. Do not
       execute it yet.
  3.  Read the plan Look at the plan the action expands into and identify the eight steps.
       Find the step that opens the fridge, and convince yourself why it is there.
  4.  Close the fridge Extend the plan so the robot closes the fridge again after taking the
       milk out, using `CloseAction` on the fridge door handle.
  5. Execute Run the plan under `simulated_robot` and watch the robot work through it.
  6. Look at what happened Inspect the plan graph after execution: which nodes only appeared at
       execution time, and which stand pose was actually chosen.

Going further

  • Leave the arm underspecified and count how many concrete plans the description now stands for.
  • Move the milk out of the fridge onto the counter and re-run. The opening steps should disappear
      from the plan without you changing a single line of your task description.
  • Block the first stand pose and watch the plan fall back to the next candidate instead of failing.

Summary

By the end of this session you know how coraplex describes tasks symbolically, how it structures
their execution, how it infers the poses the robot needs, and how it deals with uncertainty,
underspecified task descriptions, and failures. You will have completed the full milk delivery
task, demonstrating your ability to integrate perception, the semantic digital twin, and knowledge
retrieval in a practical scenario.

Further reading

Authors and Contact Details

Back to top