filename     : Ser23a.pdf
entry        : article
conference   :
pages        : 3748 - 3755
year         : 2023
month        : June
title        : Transformer-based Neural Augmentation of Robot Simulation Representations
subtitle     :
author       : Agon Serifi, Espen Knoop, Christian Schumacher, Naveen Kumar, Markus Gross, Moritz B{\"a}cher
booktitle    : IEEE Robotics and Automation Letters
ISSN/ISBN    : 2377-3766
editor       : Cecilia Laschi
publisher    : IEEE
publ.place   :
volume       : 8
issue        : 6
language     : English
keywords     : deep learning methods, simulation and animation, neural augmentation, robotics, dynamics
abstract     : Simulation representations of robots have advanced in recent years. Yet, there remain significant sim-to-real gaps because of modeling assumptions and hard-to-model behaviors such as friction. In this letter, we propose to augment common simulation representations with a transformer-inspired architecture, by training a network to predict the true state of robot building blocks given their simulation state. Because we augment building blocks, rather than the full simulation state, we make our approach modular which improves generalizability and robustness. We use our neural network to augment the state of robot actuators, and also of rigid body states. Our actuator augmentation generalizes well across robots, and our rigid body augmentation results in improvements even under high uncertainty in model parameters.