Selection of Test Cases for the Verification of Automated Vehicles

Authors

L. Vater, L. Eckstein – ika, RWTH Aachen University, A. Pütz - Ford-Werke GmbH, L. Tellis, S. Kassar - Ford Motor Company

Summary

Recent projects have already shown that highly automated driving systems (ADS) are technically ready to be released to the streets. However, the systems are responsible for behaving safe even under challenging driving conditions. Therefore, it has to be proven that they are at least as safe as (or even safer than) human drivers, before they can be released. Thus, the verification and validation of ADS is currently in the focus of the research around automated driving. A scenario-based approach where the ADS is tested only in relevant driving scenarios in controlled environment seems promising. However, new challenges arise with this approach. Namely, the selection of scenarios to be tested to verify an ADS efficiently and robustly. Most literature generates test cases from driving scenarios based on knowledge gained from naturalistic driving data or it focuses purely on safety-relevant scenarios. This paper proposes a scenario-based method for generating test cases for the verification and validation of ADS in an efficient way while not neglecting relevant scenarios. The idea is to generate test cases from driving scenarios based on knowledge gained from naturalistic driving data and the operational design domain (ODD) of the ADS to build a list of possible concrete driving scenarios. Impossible and irrelevant scenarios are then dropped and the remaining scenarios rated according to their relevance to the safety assessment of the ADS. Based on the rating, a group-individual selection procedure is used to focus on highly relevant test scenarios while not completely neglecting less relevant ones. The method was developed in a joint project with Ford and ika. During the course of the project, a proof of concept has been implemented, the proposed method was applied in a case study and the resulting test scenarios were simulated with an exemplary ADS. The results indicate that the proposed method has potential to increase testing efficiency compared to just testing all available test scenarios.