Explore Labs

Human Biosignal Lab

Explore Labs

Run Locally Back to overview

Description

To fill the robot's knowledge base and to enable adaptive human-robot interaction, we created a comprehensive pipeline to record, annotate, process, model, and query rich multimodal human data in the context of robotic applications.

 

Data collection and annotation: We collected 600 episodes of humans setting the table in the EASE TSD data set, containing rich biosignals, capturing brain and muscle activity, motion, speech, and other processes. In a human-in-the-loop process, we annotated these data with speech transcriptions, object trajectories, and action annotations. For this purpose, we developed the EASElan software.

 

NEEM generation

We created an automatic pipeline to allow seamless upload of annotated episodes of human everyday activity to the NEEM storage.

 

Model building

Based on individual semantic queries, we can retrieve specific data for adapting machine learning models to new conditions.

Real-time applications

For human-robot interaction and other purposes, we created the LiveNodes framework which allows the flexible composition of data streaming pipelines from components for signal preprocessing, visualization, and machine learning.

 

Authors and Contact Details

Back to top