What Are the Implications of Artificial Intelligence on Flight Training in the United States?

My name is Kevin. I’m a backend engineer by trade, but also a commercial pilot. I am fortunate enough to have a background in both computer science and professional aviation, having studied at the Pennsylvania State University (B.S. Computer Science) and Embry-Riddle Aeronautical University (B.S. Aeronautical Science).

Anyone who’s gone through training for an FAA pilot certificate knows the stress of a checkride, expensive and high-stakes. You sit across from a Designated Pilot Examiner (DPE) who probes your knowledge of regulations, aircraft systems, and weather for several hours. I’ve seen DPEs charge anywhere from $600 to $2,500, plus a failure stays on your FAA record forever.

When I was prepping for my own checkrides, the available tools felt dated and not realistic enough to accurately replicate the exam. Everyday AI tools (and even some aviation-specific ones) don’t truly understand aviation content, and the other options are totally non-immersive, such as videos and flashcards. Learning from my own training, I built MockDPE to give students a way to practice their checkride before it counts for real.

MockDPE runs full, voice-based oral exams against the official FAA Airman Certification Standards (ACS). You pick your aircraft, customize the route of flight, and upload an optional study guide or gouge. MockDPE then fetches live aviation weather, weaves real world conditions into an immersive scenario across 16,000+ US airports, and grades you task-by-task against the ACS, just as you will be when you sit across from your real DPE.

Getting a small LLM to evaluate a free-response aviation answer accurately and consistently is harder than it sounds: my initial approach of using plain RAG fell far short. Looking back at the git history, it took just under 200 iterations of prompt and system refinements to land on the current architecture.

I chose to use several smaller models, each given a narrow, rigid task so none of them get overwhelmed by too many rules or instructions. A dedicated grader model scores each user claim against injected reference cards (FAR, AIM, IFH, and other aviation content) and is forced to pull from official sources via tool calls to provide a regulatory citation of why the response is right or wrong. The result of this architecture and many training iterations is 96.3% grading consistency across runs, plus validation from instrument flight instructors.

Anyone can run a full mock checkride for free, which is enough for a few hours of focused instrument practice. At the end, you’ll receive an area-by-area breakdown of exactly where you should focus your study efforts before the real exam. If you want to dive deeper, unlock unlimited exam sessions, focused practice, and a personalized learning mode for the cost of 10 minutes in your school’s C172. Get started for free today at https://mockdpe.org.

The question I keep coming back to is, at what point trusting an LLM evaluator makes sense. My answer to this was when it feels like a real evaluation, with the model having access to official resources, using mandatory regulatory citations, and earning validation from people who do this for a living. If anyone here has differing opinions, especially those who have worked with LLM evaluators, I’d love to hear your thoughts on my approach.