Heuristics for Engineering Design Processes: A Reinforcement Learning Approach

<p dir="ltr">Engineering design problems are inherently complex, ill-structured, and characterized by significant uncertainty and resource constraints. To navigate these intricate landscapes and achieve satisfactory solutions, designers commonly employ heuristics, which are context-dependent directives derived from intuition, tacit knowledge, or experiential understanding. However, the prevailing informal and implicit methods for extracting and applying these valuable heuristics severely limit their systematic generalization and broader applicability across diverse design scenarios. This represents a fundamental impediment to effectively leveraging and transferring accumulated experiential knowledge, thereby constraining the cumulative advancement and efficiency of design practice. This dissertation presents a computational framework for systematically extracting generalizable heuristics in engineering design. The framework represents design processes through explicit problem decomposition and solver assignment decisions, broadly applicable across domains such as parametric design optimization, general engineering design, and systems engineering. Problems are characterized through attributes including complexity and coupling in the problem space, expertise and cost considerations for solver capabilities, and resource constraints within preference functions. To capture the inherently sequential nature of design decision-making, the framework formulates engineering design processes within a Markov Decision Process (MDP) structure, precisely defining state spaces that track solver-module assignments and historical outcomes, action spaces representing solver-module combinations, and reward functions that quantify utility and cost through explicit preference functions. Reinforcement Learning (RL) algorithms, including Q-learning and Multi-Armed Bandits, systematically explore the design process space, evaluate heuristic policies, and learn optimal decision policies that maximize cumulative rewards. A structured approach utilizing Gaussian Mixture Models (GMMs) and confidence thresholds is employed to extract actionable inclusionary heuristics that specify which solvers to include and exclusionary heuristics that identify which solvers to avoid from these learned policies. The framework's efficacy and broad applicability are empirically validated through four distinct engineering design settings: Sequential Information Acquisition and Decision-Making, Solver-Aware Systems Architecting, hierarchical resource allocation in complex systems design, and Human-AI collaborative decision-making. In Sequential Information Acquisition and Decision-Making (SIADM), optimal heuristics were extracted for function learning, information acquisition, and stopping criteria, demonstrating adaptation to varying problem complexity and performance-cost preferences. Simpler heuristics (e.g., Linear regression, Random Selection, Fixed Remaining Budget) proved effective when computational cost was prioritized, while expected utility-based heuristics (e.g., Gaussian Process, Expected Improvement, Relative Expected Improvement) were preferred for accuracy-driven scenarios. For Solver-Aware Systems Architecting (SASA), the research generated inclusionary and exclusionary heuristics for optimally assigning solver types (e.g., professional experts, specialists, crowdsourced amateurs) to design modules, as illustrated by the Golf model and robotic arm design problems. The framework's capacity for hierarchical decision-making was demonstrated in complex systems design through race car optimization in the TORCS environment, where heuristics for system-level budget allocation and subsystem-level design optimization were learned. Finally, in collaborative decision-making scenarios involving human reviewers and AI systems for terrain analysis and mine detection tasks, the framework extracted heuristics for optimal task delegation between human reviewers and AI systems in the effectively balancing accuracy, cost, and trust in uncertain operational environments. The RL-based hybrid architecture consistently outperformed traditional human-in-the-loop and automation-in-the-loop approaches in balanced trade-off scenarios. The intellectual contributions of this dissertation include a formal MDP-based representation of engineering design processes, methodological advancements through systematic RL-based heuristic extraction, and practical contributions by providing actionable guidance for designers. The cumulative impact of this formalization, computational extraction, and empirically demonstrated generalizability establishes a data-driven methodology for understanding and generating design heuristics, elevating heuristic development from an ad hoc process to a verifiable science.</p>