A FRAMEWORK FOR TEST PLANNING OF MACHINE LEARNING ENABLED SYSTEMS IN AVIATION

As the aviation community explores integrating advanced sensors that rely on machine learning algorithms to provide new capabilities to both crewed and uncrewed aircraft, the current test and evaluation frameworks require adaptations to test planning to ensure a robust evaluation. Research must be completed to address limitations to the current system to include lack of definitions, methods, standards, and policy will limit the effectiveness and efficiency of test programs. The goal of this research was to review emerging tools and methodologies, compare current test and evaluation frameworks, and complete a developmental research program to iterate through adaptations to address many of the limitations with the current frameworks. The use case of an uncrewed aircraft using a computer vision system to provide terminal navigation for the automated air-to-air refueling task was selected and a tailored test plan using the adaptations proposed was developed to evaluate four different neural networks. The results of this research included definitions, tables for documenting capabilities of the system under test, discussion on resources and tools found during research, and examples of the use of adaptations via a tailored test plan for the use case. The primary findings included that a significant quantity of information exists today to both develop initial adaptations, as well as continue to iterate and expand the scope of this research effort and that immediate benefit could be realized by integrating the findings of this research into future test programs.