Explore Labs

Retail Robotics Lab

Explore Labs

Enter Lab Back to overview

Description

The Dynamic Retail Robotics Laboratory addresses the complex challenges robots face within retail settings. Robots in this lab autonomously deploy in retail stores, adapting to changing environments, including shelf layouts and product placements. They manage inventory, guide customers, and integrate real-time product information from various sources into actionable knowledge. The goal is to develop robots that support shopping and inventory tasks while seamlessly adjusting to new products and store layouts, enhancing customer service and operational efficiency in the retail ecosystem.

ProductKG: A Product Knowledge Graph for User Assistance in Daily Activities

Abstract: The Web offers plenty of product information that is valuable for supporting decision processes. Research on Web knowledge acquisition and the Semantic Web has led to the creation of many domain ontologies and Web applications. What still is lacking is a connection of such knowledge to the real world. If object information is linked to environment information, users can get better, more personalised support in their daily activities like shopping or cooking since this enables them to link information about leftover products in the fridge to recipe information or a health profile to products the user is looking at in the store.

It has been shown that semantic Digital Twins can successfully link object to environment information that can be used by agents like smartphone or service robot. Such semantic Digital Twins can offer even more services to users if they are connected to product information from the Web. This work introduces ProductKG, an open-source product knowledge graph integrating modular product information from the Web as well as accurate environment information from a semantic Digital Twin that can be customised for different applications and used devices as an example knowledge graph for assisting users in daily activities. We describe the design process and modularity of the knowledge graph as well as example applications of it, including an Augmented Reality shopping assistant, a dietary recommender and a hands-free recipe application. The modular ontologies enable personalisation of applications as well as accessing object information in relation to the current environment. We evaluate the acceptance of one example application through a user study. ProductKG is publicly available and will be maintained and extended over time in order to facilitate various applications such as in the retail and household domain.
Resource Website: https://michaelakuempel.github.io/ProductKG/
Application Website: http://productkg.informatik.uni-bremen.de/
and https://ai.uni-bremen.de/productkg/

Publication: Kümpel, M. and Beetz, M. (2023). ProductKG: A Product Knowledge Graph for User Assistance in Daily Activities. In: Ontology Showcase and Demonstrations Track, 9th Joint Ontology Workshops (JOWO 2023), co-located with FOIS 2023. 

Robots Collecting Data: Modelling Stores

Abstract: Retail stores are a promising application domain for autonomous robotics. Unlike other domains, such as households, the environments are more structured, products are designed to be easily recognizable, and items are consciously placed to facilitate their detection and manipulation. In this book chapter we exploit these properties and propose a mobile robot systems that can be deployed in drugstores and autonomously acquire a semantic digital twin model of the store. This facilitates autonomous robot fetch and place and shopping in a virtual replica of the store. The potential commercial impact is substantial because in the retail business stores are an information blackbox and being able to automate inventory on a regular basis could improve the knowledge of retailers about their business drastically.

In this video the Kuka KMR IIWA performs a stock tacking action in the retail lab.In this video the Kuka KMR IIWA performs a stock tacking action in the retail lab.

 Publication: Beetz M, Stelter S, et al. (2022). *Robotics for Intralogistics in Supermarkets and Retail Stores*. Cham: Springer. 

Manipulation Planning and Control for Shelf Replenishment

Abstract: Manipulation planning and control are relevant building blocks of a robotic system and their tight integration is a key factor to improve robot autonomy and allows robots to perform manipulation tasks of increasing complexity, such as those needed in the in-store logistics domain. Supermarkets contain a large variety of objects to be placed on the shelf layers with specific constraints, doing this with a robot is a challenge and requires a high dexterity. However, an integration of reactive grasping control and motion planning can allow robots to perform such tasks even with grippers with limited dexterity. The main contribution of the letter is a novel method for planning manipulation tasks to be executed using a reactive control layer that provides more control modalities, i.e., slipping avoidance and controlled sliding. Experiments with a new force/tactile sensor equipping the gripper of a mobile manipulator show that the approach allows the robot to successfully perform manipulation tasks unfeasible with a standard fixed grasp.

 Publication: Costanzo M, Stelter S, et al. (2020). IEEE Robotics and Automation Letters. 

Robotic Clerks: Autonomous Shelf Refilling

Abstract: Nowadays, robots are used in the retail market mostly for warehousing, while they could be of great help in different in-store logistics processes as discussed in previous chapters. The present chapter deals with the shelf replenishment task; its execution by a robot requires overcoming of technological and methodological barriers in the handling of single products rather than the boxes containing them. The challenges a robot has to face to replenish a supermarket shelf are all related to manipulation in narrow spaces of products with a large variety of size, shape, weight, and fragility. The solution proposed by REFILLS is based on a robotic system where perception is used at all hierarchical levels of the control architecture, from high-level task planning algorithms and motion planning to reactive control layers based on physics models, where tactile and visual perception are combined to achieve highly reliable manipulation of items. Experiments in an emulated supermarket shelf are carried out to demonstrate the effectiveness of the approach.

 Publication: Cavallo A, Costanzo M, et al. (2021). *Robotics in Retail Applications*. 

Shelf replenishment in Simulation

Software Components

  • CRAM: A software toolbox for implementing autonomous robots. Source Code
  • KnowRob: A knowledge processing system for robots. Source Code
  • OpenEASE: A web-based knowledge service providing robot and human activity data. Source Code
  • GISKARD: A framework for constraint- and optimization-based robot motion and planning control. Source Code
  • ROBOKUDO: A perception framework targeted for robot manipulation tasks. Source Code
  • PyCRAM: The Python 3 re-implementation of CRAM, serving as a toolbox for designing, implementing, and deploying software on autonomous robots. Source Code

Contact Information

Back to top