VERIFICATION AND VALIDATION OF AN AI-ENABLED SYSTEM

Recent advancements in Machine Learning (ML) algorithms and increasing computational power have driven significant progress in Artificial Intelligence (AI) systems, especially those that leverage ML techniques. These AI-enabled systems incorporate components and data designed to simulate learning and problem-solving, distinguishing them from traditional systems. Despite their widespread application across various industries, certifying AI systems through verification and validation remains a formidable challenge. This difficulty primarily arises from the probabilistic nature of AI and ML components, which leads to unpredictable behaviors.

This dissertation investigates the verification and validation aspects within the Systems Engineering (SE) lifecycle, utilizing established frameworks and methodologies that support system realization from inception to retirement. It is comprised of three studies focused on applying formal methods, particularly model checking, to enhance the accuracy, value, and trustworthiness of models of engineered systems that use digital twins for modeling the system. The research analyzes digital twin data to understand physical asset behavior more thoroughly by applying both an exploratory method, system identification, and a confirmatory technique, machine learning. This dual approach not only aids in uncovering unknown system dynamics but also enhances the validation process, contributing to a more robust modeling framework.

The findings provide significant insights into the model-based design of AI-enabled digital twins, equipping systems engineers, and researchers with methods for effectively designing, simulating and modeling complex systems. Ultimately, this work aims to bridge the certification gap in AI-enabled technologies, thereby increasing public trust and facilitating the broader adoption of these innovative systems.