The spills of petroleum products can cause environmental degradation, socioeconomic problems, and damage to human health. Due to the difficulties related to the monitoring of these events over large areas, the utilization of Unmanned Surface Vehicles (USVs) has become essential for this task. The use of these devices requires instrumentation with sensors that measure water quality and a high-performance Guidance, Navigation and Control (GNC) system. Disturbances related to USV maneuverability, such as waves, winds and ocean currents, require techniques in the control system design to reject uncertainties and parametric variations in the plant. In this work, an approach is proposed for the online design of an optimal control system, based on Adaptive Dynamic Programming (ADP) and reinforcement learning, for the maneuverability of a fully actuated USV. The control system developed is based on the Discrete Linear Quadratic Tracking with integral action (DLQT-I) and the Action Dependent Heuristic Dynamic Programming (ADHDP) methods, where the optimal control law is computed online by measuring the plant input and output.
