Simulation of Complex Actuator of a Large Agricultural Robot for Deep Reinforcement Learning

Abstract

Testing with robots and especially with large agricultural robots is a task that requires high costs, risks and depends on weather conditions. Although there is a gap between simulation environments and real environments, simulation environments offer the advantages of being totally safe, making it possible to verify the performance of the control algorithm and even have the possibility of simulating sensors and actuators that have not yet been physically implemented. In addition, simulation enable risk-free assessment and adjustment of new control methods before implementation in real systems. In this work, we propose an approach to run experiments with Deep Reinforcement Learning (DRL) algorithms using Robot Operating System (ROS) and Gazebo robotics simulator. For this purpose, we use the robo-gym framework to interface between the DRL algorithm and the simulation using an OpenAI Gym environment, and augmented the capacity of Gazebo using the gazebo-fmi-actuator plugin that allows co- simulation with Functional Mock-up Unit (FMU). It is also presented an application of the simulation and control of the hydraulic steering system of a large agricultural robot using a Deep Deterministic Policy Gradient (DDPG), Soft Actor Critic (SAC), and Twin Delayed DDPG (TD3) DRL algorithms and comparing they with a PID controller.