Moslem Kazemi
01
Weather navigation is a critical capability for autonomous flight systems, enabling safe operation under diverse weather conditions. At Merlin, we recently conducted research that addresses the challenges of developing these skills by introducing a framework that learns from human pilots. A learning-based approach not only eliminates the need for exhaustive programming but also captures the nuanced decision-making pilots use in scenarios where aviation guidelines fall short. By learning directly from pilot demonstrations, the system acquires more human-like, adaptable navigation skills, which are difficult to achieve through traditional programming methods.
The Challenge of Weather Navigation
Weather navigation is one of the most demanding and complex aspects of aviation and plays a critical role in today’s increasingly connected world. Pilots must interpret radar data, synthesize real-time inputs, and make rapid decisions to ensure passenger safety and flight efficiency. Adverse weather conditions, such as thunderstorms or turbulence, pose significant risks, requiring pilots to make timely and accurate judgment calls in order to maneuver the aircraft and avoid danger zones.
Traditionally, aviation guidelines provide a framework for safe navigation but they often leave finer details to pilot discretion. This is particularly true in dynamic and unstructured weather conditions where specific rules are unavailable. Programming autonomous systems to handle every possible weather scenario would require exhaustive manual coding and tuning, an impractical and inefficient approach. Moreover, such systems often lack the adaptability and human-like flexibility required to navigate the real-world complexity of the skies.
Fig. 1: An example of weather navigation using airborne radar intensity images: (a) an example radar intensity image (source: Adobe Stock) with the current and goal locations of the airframe depicted in green and red, respectively, (b) a viable planned route (cyan) from the current to the goal location
Learning from Pilots: A Smarter Approach
To overcome these challenges, we incorporated imitation learning, an advanced technique that allows autonomous systems to learn from human demonstrations. Instead of programming every decision explicitly, imitation learning (implicitly) captures the strategies pilots use to navigate complex weather conditions.
The approach frames weather navigation as an inverse optimal control problem. By analyzing pilot-drawn routes on radar images, the system learns a cost function—a mathematical representation of pilot preferences for safe, efficient paths. This cost function maps weather features, such as radar intensity levels, to scalar values that guide route planning. The result is a system capable of producing costmaps, which a deterministic planner uses to identify optimal routes around adverse weather.
Key Advantages of a Learning-Based Approach
- Human-Centered Design: By learning directly from pilot demonstrations, the Merlin system implicitly captures the human pilot expertise and intuition, including a deep understanding of FAA rules and guidance, that traditional programming can hardly capture and replicate.
- Scalability and Adaptability: The framework is designed to integrate additional data sources, such as NEXRAD weather data, ATC guidance, and pilot reports, making it highly adaptable to real-world scenarios.
- Dynamic Decision-Making: The system’s ability to update navigation plans in real-time ensures it can respond effectively to changing weather conditions.
- Enhanced Safety: By mimicking the decision-making of skilled pilots, the system generates routes that are not only efficient but also inherently safer.
Proof of Concept: A Promising Step Forward
To test this approach, we developed a system to demonstrate the effectiveness through rigorous testing with airborne radar data. Pilots annotated radar images with feasible routes, which were then used to train the system. The resulting navigation models produced routes comparable to those planned by human pilots, successfully navigating diverse weather scenarios.
Fig. 2: Validation results on the test dataset: pilot annotated routes (in cyan), the routes generated using learned weights (in red), and the shortcutted version of the learned routes (in orange) overlaid on the learned costmaps shown in grayscale
Validation tests showed that the system could generate routes comparable to those of human pilots, effectively navigating various weather scenarios. While some challenges remain—such as refining heading changes and ensuring adequate safety buffers—the results highlight the system’s potential to handle real-world conditions effectively.
Overcoming Challenges and Expanding Capabilities for the Future
We are committed to further refining and expanding the weather navigation system. This proof-of-concept has yielded valuable observations and feedback, offering potential directions for extending and improving this work to develop a real-world weather navigation component for autonomous flight systems.
- Integrating Diverse Data Sources: Incorporating additional inputs like NEXRAD, ATC updates, and visual observations to enhance the system’s decision-making.
- 3-D Routing: Extending navigation capabilities to three dimensions to handle complex weather systems across altitudes.
- Dynamic Planning: Introducing real-time planning and re-planning to adapt to rapidly changing weather conditions.
- Addressing Radar Shadows: Developing solutions to detect and avoid radar shadow areas, further improving route safety.
Redefining the Future of Autonomous Flight
This work in autonomous weather navigation represents an exciting step forward for Merlin as we advance our technology, while prioritizing safety and building trust within the aviation industry. By integrating pilot expertise each step of the way, we are bridging the gap between human intuition and machine precision. These innovations not only make autonomous flight more reliable, but also establish a new benchmark for standard for safety and efficiency for autonomous flight systems.
To read the full version of the paper that was originally published in and presented at the 43rd Digital Avionics Systems Conference, San Diego, September 2024, click here.