Information Extraction from Pilot Weather Reports (PIREPs) using a Structured Two-Level Named Entity Recognition (NER) Approach

Weather conditions such as thunderstorms, wind shear, snowstorms, turbulence, icing, and fog can create potentially hazardous flying conditions in the National Airspace System (NAS) (FAA, 2021). In general aviation (GA), hazardous weather conditions are most likely to cause accidents with fatalities (FAA, 2013). Therefore, it is critical to communicate weather conditions to pilots and controllers to increase awareness of such conditions, help pilots avoid weather hazards, and improve aviation safety (NTSB, 2017b). Pilot Reports (PIREPs) are one way to communicate pertinent weather conditions encountered by pilots (FAA, 2017a). However, in a hazardous weather situation, communication adds to pilot workload and GA pilots may need to aviate and navigate to another area before feeling safe enough to communicate the weather conditions. The delay in communication may result in PIREPs that are both inaccurate and untimely, potentially misleading other pilots in the area with incorrect weather information (NTSB, 2017a). Therefore, it is crucial to enhance the PIREP submission process to improve the accuracy, timeliness, and usefulness of PIREPs, while simultaneously reducing the need for hands-on communication.

In this study, a potential method to incrementally improve the performance of an automated spoken-to-coded-PIREP system is explored. This research aims at improving the information extraction model within the spoken-to-coded-PIREP system by using underlying structures and patterns in the pilot spoken phrases. The first part of this research is focused on exploring the structural elements, patterns, and sub-level variability in the Location, Turbulence, and Icing pilot phrases. The second part of the research is focused on developing and demonstrating a structured two-level Named Entity Recognition (NER) model that utilizes the underlying structures within pilot phrases. A structured two-level NER model is designed, developed, tested, and compared with the initial single level NER model in the spoken-to-coded-PIREP system. The model follows a structured approach to extract information at two levels within three PIREP information categories – Location, Turbulence, and Icing. The two-level NER model is trained and tested using a total of 126 PIREPs containing Turbulence and Icing weather conditions. The performance of the structured two-level NER model is compared to the performance of a comparable single level initial NER model using three metrics – precision, recall, and F1-Score. The overall F1-Score of the initial single level NER model was in the range of 68% – 77%, while the two-level NER model was able to achieve an overall F1-Score in the range of 89% – 92%. The two-level NER model was successful in recognizing and labelling specific phrases into broader entity labels such as Location, Turbulence, and Icing, and then processing those phrases to segregate their structural elements such as Distance, Location Name, Turbulence Intensity, and Icing Type. With improvements to the information extraction model, the performance of the overall spoken-to-coded-PIREP system may be increased and the system may be better equipped to handle the variations in pilot phrases and weather situations. Automating the PIREP submission process may reduce the pilot’s hands-on task-requirement in submitting a PIREP during hazardous weather situations, potentially increase the quality and quantity of PIREPs, and share accurate weather-related information in a timely manner, ultimately making GA flying safter.