There is a significant effort underway to replace flight simulator visual systems with virtual- or mixed-reality (VR or MR) headsets, which give the user an unlimited field of view in any direction. Headsets also have the advantage of eliminating simulators’ bulky visual displays, which have a narrower field of view and add significant cost to the price of training devices.
At the same time, not everyone adapts to VR or MR without experiencing mild or even worse motion sickness. A start-up company called Brelyon has developed an alternative, a so-called “immersive monitor” that looks and feels like the view through a headset without having to wear a head-mounted device, and provides true depth and simultaneous views of multiple layers.
The U.S. Air Force has adopted headset-based training in its Pilot Training Next program but found that 24% to 27% of students were unable to tolerate more than seven minutes in the VR/MR world, according to Alok Mehta, COO and co-founder of Brelyon.
Brelyon’s display is designed to eliminate the convergence mismatch or vergence accommodation conflict that can occur with VR/MR headsets and cause discomfort. For normal stereoscopic vision, the eyes and brain work together to provide a sharp image on the retina using vergence and accommodation. Vergence is the independent rotation of the eyes that enables perception of an object as a single object, and if this isn’t correct, double vision of that object can occur. Accommodation focuses images so they are sharp on the retina. If both vergence and accommodation distances don’t match, the result is vergence accommodation conflict, which is common with 3D images.
For this reason, most people are comfortable with legacy 2D displays, but they lack the feeling of immersion in the environment that is replicated, for example, in a flight simulator.
Mehta, who holds a doctorate in optics and optical sciences, was a principal scientist at DARPA and has been working for many years on ways to get more capability and performance from smaller size and weight and lower power-using displays. What is needed to overcome the drawbacks of 2D displays and 3D convergence mismatch, he explained, is optimal visual ergonomics. Brelyon “is bridging that gap, with comfort, performance, and immersion,” he said. We’re focused on practical solutions.”
The goal, according to Brelyon founder and CEO Barmak Heshmat, is: “Can you mimic a headset-like experience by engineering the light around the user instead of forcing them to use a device?”
That solution is Brelyon’s Ultra Reality (UR) and Ultra Reality Extend (URE) immersive monitors, which allow for information to be layered, with “multiple data streams at varying depths,” according to the company, enabling users to “visualize vast amounts of data in an organized and easily accessible way.” That includes arranging the data layers as needed and also adjusting the transparency of each layer.
Looking at a UR or URE monitor, the viewer will see two channels of 8K aggregate resolution, with two depth planes, 122 inches at the deepest layer and 40 inches at the near layer. The actual monitor measures only 30 inches, but has a horizontal field of view of 110 degrees and a vertical field of view of 51 degrees. While looking at the monitor, the user can move around and still perceive the depth between the layers, according to Brelyon. With HDMI or DisplayPort inputs, the Brelyon displays can be used like a typical computer monitor.
The company began as a 3D display manufacturer, according to Mehta, but the market required too much of a customized solution. “We realized…before we raised our first capital, that our core engine, this idea of optical depth modulation, could be used to provide monocular-depths virtual displays that could supersede the displays that we all use every day,” he explained. “We now have the ability to control all of the pixels that are provided through our display solution in very unique ways.”
Flight simulation, and environments such as medical simulation and real estate walkthroughs that Brelyon is also targeting, require “this concept of things that should look close to you, should be optically close to you,” he said. "As you propagate through that environment, those [other] things are further away.”
After joining a technology incubator run by a pilot who encouraged Brelyon to consider the flight simulation market, the company demonstrated the URE monitor at the Interservice/Industry Training, Simulation and Education Conference. “That’s when Lockheed Martin first saw us and said, ‘Hey, this is a big deal for everything that we're doing,’” Mehta said. “They brought in their computer [for] the flight training environment, plugged it in, and that was the first aha moment, because they brought in pilots, and that was the first time that we saw like there really is something [to this technology].”
This led to further investments in Brelyon and work for the U.S. Air Force through Afwerx Small Business Innovation Research funding. “Everything that we're doing in the defense world, we have parallel activities in the commercial domain,” Mehta said. This includes working with an airline that wanted to set up training activities at its bases so it wouldn’t have to send pilots to a distant simulator training company as often. The airline tested Brelyon’s URE technology against the full flight simulator and found it fulfilled 95% of pilot training needs, much better than the previous solution the airline had been using, which was curved 2D displays.
Brelyon is already working with commercial airline and military customers on applications of the URE monitor for certain aspects of pilot training. However, it isn’t yet suited for air combat maneuver training due to the need for more vertical field of view, which Brelyon is already working on improving.
Pilot Kel Thompson had this to say about the Brelyon URE: “I am an airline pilot and instructor pilot on the Boeing 737 Max. Simulator images have been highly refined, and today’s are super realistic. I use the Brelyon Ultra Reality display with my home simulator. The images it produces are more like those produced on the collimated visuals at Boeing than to those rendered by a high-definition, curved gaming monitor.”
Another pilot explained, “As a retired airline pilot, simulator instructor, and check airman, I wanted to keep my skills sharp and practice flying at home with a realistic, yet affordable, simulator. My main challenge in looking at flight simulation was trying to get that feeling of being in an aircraft cockpit. Regular curved screens and multi-screen displays were too big and required too much space, and projectors would not work well in my home. VR headsets were isolating and not useful for providing instruction to my friends who wish to use the flight simulation programs. In contrast, the Brelyon display gives you a huge, clear picture that is easy to look at, while not taking up a lot of space in my home. The picture is sharp, and it looks very realistic.”
“Our primary driver into the market has been as this headset alternative,” Mehta said, “because it's a known pain point that we can readily address today.”