Learning Delta Policies for Automated Driving via Reinforcement Learning

Authors

M. Templer, J. Kaste, P. Hochrein, B. Mennenga, Volkswagen AG

Summary

Safe and robust control of automated vehicles is a difficult task, especially in driving maneuvers, where vehicle reactions get highly nonlinear and are therefore hard to model. Although state of the art control approaches show good performance, they mostly rely on an accurate vehicle model. Building such a model can be extremely complex in high-dimensional continuous-control tasks and comes with a tedious process of parameter tuning. Inspired by the success of Reinforcement Learning (RL) in the robotics domain, we present a hybrid control approach which combines classic control strategies with model-free reinforcement learning. To overcome the sample inefficiency of model-free learning, a digital twin of the car is combined with a physics simulator to reduce training time on the vehicle. To minimize the state distribution shift between simulation and reality, we identify the system dynamics based on various real world driving scenarios, develop an actuator model and emulate system latency. Furthermore we model a compact observation state, relying solely on 1-D sensor data for vehicle state and trajectory information...