FlightSafety International has formally launched an effort called FlightSmart that uses artificial intelligence (AI) to elevate aviation training quality. What the FlightSmart team has done is figure out a way to use modern technology, including IBM’s Watson AI tool, to help its instructors and training centers improve a student’s experience.
“The product, through the collaboration with IBM, is utilizing advanced algorithms, machine learning, artificial intelligence—all of those cognitive technologies—to provide the objective evidence or objective evaluation of the student's performance,” said Matt Littrell, FlightSafety product director of AI and adaptive learning.
Ultimately, FlightSafety sees FlightSmart as helping students learn faster by mastering tasks more quickly, while at the same time giving instructors better information about the students’ performance so they can act as “learning managers” and provide better feedback to improve the training process. The goal of FlightSmart is to help pilots master their skills and become more proficient. An ancillary benefit is that the system also will increase training efficiency, thus bringing more pilots into the workforce and lowering the burden on instructors, for which there is also a shortage.
“The primary focus is to help the student pilots become better and faster and master tasks that are challenging,” said senior product manager Chris Starr.
FlightSmart isn’t just for simulator training, but will also be helpful for task training on avionics and operating flight management systems; use of automation; standard operating procedures; crew resource management; and other areas besides flying—such as maintenance and operating unmanned systems. “It has tremendous potential throughout many avenues and markets,” Littrell said.
The first formal FlightSmart implementation is with the U.S. Air Force for the T-6A pilot training program. For FlightSafety’s business aviation customers, Littrell said, “We have implemented FlightSmart in a limited capacity on one of our business jet programs for developmental and evaluation purposes only. We are currently evaluating the best strategy for a broader implementation within our learning centers.”
When and Why
There are three use cases that FlightSafety customers have flagged and that FlightSmart is designed to satisfy, according to Littrell.
The first is to identify problems and address them much earlier in the training process, which improves efficiency because this can eliminate the need for additional remedial training. An extra day of full-flight simulator training adds thousands of dollars to the training cost, takes more valuable time, and adds complexity to the logistics of running an extremely complex training process.
To address this first use case, FlightSmart can automatically identify training tasks, instead of the instructor having to look away from observing the student and selecting the tasks. An example might be a steep turn, and FlightSmart is programmed with start-stop times, maneuver criteria, and other elements that identify and record the task so the instructor doesn’t have to make notes or try to remember the task and the student’s performance.
This automation helps the instructor give the student a more comprehensive evaluation because now the simulator is recording everything the student does and, more importantly, records parameters that the instructor cannot perceive.
“The problem is the instructor can’t see everything the pilot is doing,” said Starr. “He doesn’t know how much force [the student] is putting on the column or how much he's moving the throttles. FlightSmart takes the data out of the simulator, all the forces that are being applied, all the movements that are happening, how soon a student is selecting a switch or pushing a button.”
The instructor usually can’t see all of these elements because his or her workload is already high, but with the information from FlightSmart, the instructor can help the student improve. He explained, “The instructor has objective data to say, ‘You had too much force on the pedals during that rejected takeoff. [Or] this showed that you shouldn't have been on the toe brakes. Or, you did it perfectly. You're within the top 5 percent of everyone that does this maneuver.”
Starr gave two examples of where this information can help the student. In training for a particular aircraft, FlightSmart identified that pilots were riding the brakes during takeoffs and landings, and this not only lowers brake life but could cause directional control and performance problems. But those problems weren’t the sole benefit of sussing out that information. The information lends itself to root-cause analysis: is the design of the rudder pedals and brakes less than optimal?, for example. “It opens the door for that investigation and re-education,” he said.
The second example was a pilot who struggled with V1 cuts (engine failures at a critical time during takeoff), then suddenly improved toward the end of training and passed the checkride. A review of the data later showed that the pilot-not-flying began stepping on the correct rudder pedal during the V1 cut. “The copilot, in essence, was helping him push on that rudder pedal,” said Starr, “which is what he had been struggling with all along. We passed that on to the examiner, and he commented, ‘I was wondering why all of a sudden he did so much better.’ Those are the types of things that the instructor physically cannot see, but through the data, we can gain visibility into those and provide that insight to the instructor.”
Another use case is to help an operator adapt the training style to the characteristics of a pilot job applicant. An airline looking to hire pilots typically sees two types of applicants: one is fresh out of school with strong theoretical and avionics knowledge, but weak stick-and-rudder skills. The other hasn’t been flying for a while, perhaps because they gave up on aviation during the 2008 Great Recession, and their flying skills are strong but they are way behind on current technology and processes.
“The challenge they face is those two different types of applicants require two different styles of training,” Littrell said. “They're looking to FlightSmart to identify what's the best method of training these two different types and tailor the training to those needs so they're not forced to run everybody through the singular canned training profile.”
While the canned training profile serves some applicants, others end up needing significant amounts of additional training, which is expensive and time-consuming.
For the third use case, FlightSmart can help an operator screen pilot candidates. “It knows, in essence, what makes a good pilot, or the traits and the style of flying that have the greatest chance of success in the training, as well as beyond,” he said.
For military flying, although military forces are very good at weeding out pilots who aren’t going to succeed, there is still an enormous cost associated with putting a pilot through a lot of initial training then finding out toward the end that it isn’t going to work. If it costs, say, $1 million to train a pilot and the attrition rate is 10 to 15 percent, that’s a lot of money. “That's where we have shown them with FlightSmart, we can advance them earlier, we can [help identify failing candidates], and that saves them a lot of money when you look at their attrition rate,” said Starr.
FlightSafety has discussed with customers the issue of what happens when FlightSmart identifies a student who shouldn’t be flying, according to Littrell, but that isn’t FlightSafety’s role. “We're not the deciding factor of whether a pilot should continue on or not,” he explained. FlightSmart uses computers and AI tools to provide information to the customer. “That's where FlightSmart comes in. We’re providing the objective evidence, then it's up to them to decide the criteria they ultimately use to determine when a pilot should continue flying or not.”
How FlightSmart Works
The goal of the FlightSmart program, which got underway two years ago but was formally launched a year ago, was to figure out how to provide a more objective evaluation of student performance. As Littrell put it, “We consider FlightSmart a revolutionary tool that is designed to turn the aviation professional training experience on its head. It's focusing on the individual themselves, but it also can focus on the population as a whole or a subset of a population. We’re tailoring the training to the specific needs, strengths, weaknesses, and focus areas through objective evidence and machine learning.”
The collaboration with IBM’s Watson service, he added, “is utilizing advanced algorithms, machine learning, artificial intelligence, all of those cognitive technologies to provide the objective evidence or objective evaluation of the student's performance as they’re in the entire training ecosystem. Our initial development started focusing solely on the simulator side, capturing the data out of the simulators and applying the algorithms to first automatically identify what training tasks were accomplished during that training session.”
The important factor here, however, is what standards apply for measuring the student’s performance. Should it be the FAA Airman Certification Standards, as flown perfectly by the simulator?
According to Littrell, “We go more granular than that rather than just an A/B-, pass/fail-type evaluation.” There is scoring against the overt regulator-derived standard, but FlightSmart also uses a “gold-standard baseline,” which is based on captured human performance.
With skilled pilots at the controls, FlightSmart “captured their sessions flying that particular training task and then worked up a baseline based on all that data,” he said. “We could have the simulator fly a perfect steep turn or whatever the case was. But, but that's not realistic, that doesn't take into account the human element. So we opted to use actual humans to create that baseline. The way machine learning and analytics work, the more data you have, the smarter it gets. So over time, the baseline will continue to improve and be more reflective of the pilot population and that gold standard.”
The next step is evaluating the student’s performance against the new standard. This isn’t just a matter of plotting the physical measurement of a student’s flying to the baseline standard, but more about assessing the intangible qualities of skillful flying. “It's also evaluating the—for lack of a better term—the smoothness of their flying or their style of flying,” Littrell explained. “Looking at the frequency and amplitude of their plotline; are they manhandling the controls? Are they smooth and precise? Are they behind the power curve, slow to respond to deviations, and so forth?”
The benefits don’t just accrue to the students, who not only get scored on their performance but also get an objective evaluation of their performance, with information that can help drive improvement. The other benefits are that FlightSmart helps instructors shift from mundane tasks to truly being able to observe and help their students. It also promotes standardization of instructors, Littrell said.
“We've all seen those examiners and instructors—some are more stringent; and others are more lax. So we worked towards standardizing them. There are reports that can be run to look at the scores of students that a particular instructor has, to look for variations. But one of our key drivers from an instructor standpoint is reducing the burden on them.”
The burden takes the form of the rapidly growing amount of content that must be taught in the same limited period of time. “There are a lot of times they're very task-burdened,” he said. “Our mission from the outset was to reduce the burden as much as possible for those instructors. Providing the objective evaluation is one element of that. Another element is the automatic identification of the training tasks. We don't want the instructor having to make a selection while they're in the simulator and distracting them from what they're there to be doing, which is the teaching.”
The teaching itself is not just the time in the simulator, but afterward when the instructor can fully debrief the student, again based on objective information that captures exactly what the student was doing.
“[For a] steep turn,” Littrell said, “if they deviate on altitude by 126 feet, a good instructor is going to be able to deduce within reason what may have caused them to deviate from that altitude. But we are using machine learning to drive to the root cause of what caused them to deviate. We have all the parameters available to us, and machine learning looks at those and in essence creates an error chain to determine what was the ultimate root cause of that deviation on altitude. It may have been two minutes and 36 seconds ago you bumped the rudder pedal, which destabilized you, which led to this.”
Once debriefed, students can take that critique and practice on their own to hone their skills, using FlightSafety’s advanced training devices or graphical flight deck simulators, which can also incorporate FlightSmart. This will also help make the actual full-flight simulator training portion more effective.
More important is that FlightSmart will be used to continually hone the training program to fit the student, all with the goal of improving mastery of the subject and tasks. The instructor will become more of a training or learning manager who can help guide the student toward success.
FlightSafety students and instructors can view FlightSmart information on their own dashboards, accessible via browser software and including an overall competency assessment grading view.
Students will be able to see their scores by task; the top three reasons for failure; next suggested training tasks; the top three recommended next actions; master percentage; and progression trend graph.
The dashboard for instructors (training managers) will show ranking of students by scores, mastery level, and training task score; the likelihood of additional training required; and the likelihood of washout.
FlightSmart Security
It goes without saying that secure storage of all the data gathered by FlightSmart is critical, and FlightSafety has taken this into account.
“The way we approach that is, first, we only identify data to the level that we received permission to identify it,” said Littrell. That means it is up to the customer to specify whether the data is recorded anonymously or that the organization is identified or the identify of the person undergoing training is available to the customer. “Every organization has their own unique requirements as it pertains to data privacy,” he added. “We work with each customer to ensure that we meet not only the letter of the law but also their internal requirements.
But FlightSmart’s benefits extend well beyond the simulator and it can be integrated into learning/training management systems, classroom environments, computer-based training, etc., to help pilots from early in their career throughout their professional flying lifecycle.
“We’re seeing interest from all over the industry,” said Littrell, “including large commercial operators to single-pilots, ab initio training, and universities.” The FlightSafety Academy, which teaches new pilots, is also looking to implement FlightSmart.
While some FlightSafety instructors worried that FlightSmart was designed to replace them, and students worry that the computer will try to determine whether they are or aren’t qualified to fly, the company is sending a clear message. “My goal is to provide you with additional resources to make you better,” Littrell concluded. “It’s the same with students. We’re not trying to determine whether you should or should not continue flying, our goal is to make you a better pilot and provide you with the knowledge necessary to hone your skills.”