Automatic grain unloading for crop harvest machine

The world is facing a higher demand for food as the population is expected to grow to 9.1 billion by 2050, but the expected growth of arable land is much slower. In the meantime, the US has seen farm labor shortages for many years. These trends indicate the need for improving agricultural productivity while lowering the demand for skilled labor for farm operations. Automation of agricultural operation is one approach to achieve these goals. An automated unloading system is desirable as it can improve productivity and reduce the requirement for high-skill labor by lowering the complexity of the unloading on the go operation.

Agricultural machinery companies have developed various products to automate or assist parts of the unloading operations. Some researchers built unloading automation systems, but the limited performance, strict constraints, and the high cost curb their impact on productivity improvement or adaption for commercialization. Additionally, several companies have released product to automate the forage harvester unloading. However, no existing system can fully automate the combine harvester unloading on the go. Therefore, a system was proposed to automate combine harvester unloading on the go by automatically monitoring grain fill status, determining preferred auger location to fulfill prescribed fill strategy, and controlling the auger operation and location to achieve the desired fill.

An automatic unloading strategy for grain unloading automation was developed. The automatic unloading system is built by integrating a controller and a perception system to the combine harvester with an existing vehicle guidance technology, Machine Sync. Machine Sync is used to control the combine-tractor relative position by automatically changing the speed and moving direction of the tractor.

To develop the automatic unloading system, simulation tools were built to model the unloading on the go process and validate the model accuracy with in-field testing. The tools include:

A grain fill model to simulate how grain pile up in a container such as grain cart or wagon given the grain unloading location and unloading rate. A grain fill model benchmark system was built with LiDAR and validated that the grain fill model can achieve an accuracy of 0.2 m during a static grain cart unloading.

A vehicle dynamics model to simulate the dynamics of the relative position between the tractor and the combine harvester. The relative motion between the combine and the tractor controlled by Machine Sync was treated as an aggregated system. To characterize the dynamics of the aggregated system, the instrumental variable approach was used to identify the model parameter based on black-box model simulation results. After that, a testing pipeline was developed to validate and refine the model parameters with in-field testing.

A perception model to simulate the raw data of the perception sensors (i.e., stereo camera) during unloading with different lighting conditions, vehicle configurations, and sensor properties. To validate the perception model, stereo camera data were collected during automatic unloading in some typical conditions and compared them with the simulation results.

The simulation tools together build a virtual environment to simulate the unloading process. Based on these tools, the automatic unloading controller was developed. The controller automatically determines the desired auger location to fill the grain cart based on the current filling status and prescribed fill strategy. The controller also includes a closed-loop movement controller synthesized with H infinity mixed sensitivity loop shaping that closes the loop around Machine Sync to enhance its tracking performance and robustness. After that, the automatic unloading system was validated in the virtual environment.

After validating the automatic unloading system in simulation, the automatic unloading system was implemented in hardware and a camera-based perception system (IPM) was integrated to monitor the unloading status. In-field testing demonstrated that the automatic offloading system can effectively automate the unloading-on-the-go of a combine harvester to fill a grain cart to the desired level under nominal harvesting conditions. The achievable fill level for a 1000-bushel grain cart without spillage ranges from -0.7 m to -0.2 m for the near-edge grain height relative to cart edge.

The in-field testing shows that the camera-based perception system (IPM), which is susceptible to environmental changes, can lose track of the grain cart and cause the automatic unloading to stop. To make the grain perception more robust to different lighting conditions, a fusion algorithm was developed by leveraging both the IPM and grain fill model. In-field testing data demonstrate that the fusion result can achieve higher accuracy, greater coverage, and better robustness than either the IPM or the grain fill model alone.