Integrating Multi-Modal Remote Sensing, Deep Learning, and Attention Mechanisms for Yield Prediction in Plant Breeding Experiments and Management Practices Experiments

To address the challenges of increasing global food demand, climate change, and resource constraints, significant advances are required in plant breeding, sustainable agricultural practices, and technological solutions. This dissertation examines the use of remotely sensed data from unmanned aerial vehicles (UAVs) integrated with deep learning models that incorporate temporal attention mechanisms to improve the accuracy and explainability of yield prediction in plant breeding and management trials. This study leverages a multimodal remote sensing dataset, including hyperspectral, LiDAR, and environmental data, to mitigate challenges related to early-season prediction, model explainability, and broad applicability.

The study consisted of three themes: identification of relevant features within hyperspectral and LiDAR datasets for models, exploration of temporal attention mechanisms to improve model interpretability, and achievement of robust yield prediction generalization across varied temporal periods and geographic areas. The research investigates the utility of Shapley Additive Explanations (SHAP) for feature selection, isolating key features derived from RS data that improve model performance without sacrificing interpretability. Attention-based DL architectures, including stacked Long Short-Term Memory (LSTM) networks, are implemented to capture temporal dynamics and align model predictions with biologically significant growth stages. Transfer learning and domain adaptation are investigated to improve the generalization of yield prediction models under diverse growing conditions and with limited training data.

The SHAP-based feature selection successfully decreased input dimensionality without sacrificing LSTM model accuracy; concurrently, attention mechanisms highlighted the temporal significance of features, correlating with physiological phases of maize growth. Supervised approaches and semi-supervised/unsupervised generative methods for domain adaptation demonstrated potential for robust cross-environment prediction, enhancing scalability and practical utility. This research contributes to the understanding of how multi-modal remote sensing data and deep learning techniques can be utilized to address crop yield prediction. This research suggests improvements to sustainable agricultural practices are possible, specifically within plant breeding and crop production management.