Redbird Flight has been testing a prototype of its desktop Redbird TD basic aviation training device (BATD) fitted with a mixed-reality headset driven by software that gives users a more realistic and immersive flying experience. Many companies are developing mixed-reality systems, but Redbird’s promises to cost in the tens of thousands of dollars versus hundreds of thousands for systems from traditional simulator manufacturers.
On display at the Redbird exhibit this week at the EAA AirVenture show in Oshkosh, Wisconsin, the Redbird setup consists of a desktop BATD, a Varjo mixed-reality headset, and an image-generation system developed by Quantum 3D. Redbird is encouraging visitors to its exhibit to try the mixed-reality simulator.
Mixed-reality (MR) combines a virtual-reality (VR) view of the outside world with the ability of the user to see and interact with physical hardware. A pilot using the Redbird MR system while wearing the Varjo headset sees a 360-degree view of the outside world simply by looking in any direction. But unlike a VR headset, where users have to manipulate a simulator’s buttons and controls with some kind of cursor-control device, the MR headset has cameras on the outside pointing in the direction the user is looking.
These cameras show the user a portion of the physical world, in this case, the instrument panel of the simulated Cessna 172, so the user can see his or her own hands moving the yoke, twisting a knob, or pushing a button. The result is a far more realistic VR experience and better positive transfer in the training environment.
Running a MR system takes a lot of computing power, and the Redbird system uses two powerful graphics processing units, one for each eye. The visuals of the outside world that the user sees are not just those from the underlying simulator software (Redbird uses Prepar3d, but Quantum3D also works with X-Plane), but Quantum3D’s Mantis module-based image-generation software platform.
Mantis is already being used for some military applications and Virgin Galactic used a Quantum3D system for training pilots for its suborbital flights, according to Quantum3D CEO Murat Kose. An air ambulance operator in Germany uses Quantum3D MR for its Airbus H145 helicopter pilot training, and there is also an application for firefighting flight operations.
The Mantis system delivers high-resolution graphics at 60 frames per second. Some of Mantis’s capabilities include a variety of weather phenomena, ocean effects, rotor wash, global terrain, more than 30,000 airports, and night-vision goggle and infrared sensor replication. “We have a bunch of level-D airports and weather,” Kose said.
During a demonstration of the MR system, I flew the simulated Cessna 172 in a traffic pattern at San Francisco International Airport. Being able to look in any direction and see the airplane as if I were sitting in it was a big improvement on a regular desktop simulator. Yet at the same time, I could look straight at the instrument panel and see my hands on the yoke and throttle and pushing the G1000 avionics buttons and twisting the knobs.
The visuals kept pace when I moved my head rapidly from side to side with no latency. The instrument panel was fairly sharp, although there were some parts of the view that were slightly blurred.
In any case, I was able to fly the simulation immersed in the view of the outside world and easily able to judge when to turn from downwind to the base leg by looking back over my shoulder, something that is difficult to do in a regular desktop simulator. The only thing lacking was any control feel, although Redbird could easily add the MR system to one of its motion-based simulators and add feedback to the yoke.
Redbird hopes to obtain FAA approval for use of the MR simulator as a BATD, which allows for credit for some instrument flying and currency training. The aviation university and high school aviation programs “are most excited about this,” said Josh Harnagel, Redbird v-p of marketing.
Redbird is also developing an advanced aviation training device (AATD) that replicates the Air Force T-1A Jayhawk (Beechjet). The Air Force has ordered 25 of the Redbird AATDs and will use them for jet transition training, replacing the T-1As after they are decommissioned. The AATDs will have Redbird’s current visual display, but Harnagel said they could be equipped with the MR system if the Air Force wanted to add MR capability.
“The new proof-of-concept device is a promising step forward in enhancing the human-computer interface of simulated flight,” said Redbird CEO Todd Willinger. “In our years of research and development of virtual, augmented, and mixed reality systems, we had found that creating a more immersive visual environment came at the expense of impeding a pilot’s ability to interact accurately with the physical elements required in certified training devices. This device allows pilots to see and adjust to everything inside and out of the flight deck—both virtual and physical—demonstrating that it could become an effective solution for professional flight training.”