Gabriel Behrendt

“Combinatorial and Adaptive Stress Testing for Autonomous Spacecraft Control Systems”

Session Materials: Link

Session Recording: Link

Speaker Bio: 

Abstract: 

This paper addresses the growing challenge of evaluating increasingly complex autonomous control systems, particularly those developed using machine learning techniques like reinforcement learning, by introducing novel testing methods that overcome the limitations of traditional analytical and Monte Carlo simulation techniques. Specifically, we propose a dual approach that leverages combinatorial (t-way) testing to efficiently explore critical parameter interactions and Adaptive Stress Testing (AST) to dynamically identify edge-case failure modes, enhancing test efficiency and failure case identification in simulation environments. This paper applies this combination test case generation approach to a runtime assured autonomous control system for spacecraft attitude, where a neural network control system trained via reinforcement learning
is bounded by a control barrier function-based assurance module and compares the results of these testing methods against conventional Monte Carlo approaches. The contributions include development of a combined t-way testing and AST test case generation approach, application to a runtime assured neural network control system, and evaluation of results compared to a traditional Monte Carlo Simulation.