Getting Mixed Up in Mixed Reality
by Sam Finston | published Feb. 2nd, 2018
It has been a few years since Oculus made virtual reality relevant again. In that time, countless companies have jumped in with their own attempts at the tech that many are hailing as a medium of the future. But as big names such as HTC and Sony (and Google and Samsung ...) fight it out over who has the best electric View-Master in Silicon Valley, a Microsoft-led wave of devices is threatening to make it even harder for consumers to wrap their heads around all these new headsets.
In 2015, Microsoft unveiled the HoloLens, a headset that you can see through while it adds “holographic” images on a transparent display before your very eyes. The effect is that these objects appear to occupy the same space as the user, even adapting to the environment around them.
This ambitious invention is the brainchild of the same man behind 2010’s Kinect, RIT graduate Alex Kipman. HoloLens uses Augmented Reality (AR) to achieve its impressive effects, but Microsoft, along with a sudden onslaught of competitors, has started pushing the term “Mixed Reality Headset.”
As with the still-going attempts of virtual reality, Mixed Reality (MR) technology is taking its time getting more affordable and consumer-ready. But before one can truly judge whether this is the way of the future or just another passing, expensive novelty, one must first clear up the VRs, ARs and MRs to really understand what is being sold.
Mixed or Augmented: Virtually the Same
“With virtual reality, the intention is that you completely immerse the user in this virtual world,” explained Dr. James Vallino, professor of Software Engineering in the Golisano College of Computing and Information Sciences. “You don’t have an interest in them staying in the real world.”
This is what you find in the Oculus Rift and the HTC Vive. MR headsets, however, make use of augmented reality.
“The intention there is that primarily the user is in the real world, but we are augmenting their perception of it,” Vallino explained.
The use of the term “Mixed Reality” over “Augmented Reality” is likely just a result of its easier marketability, but it is not without merit. Many people frame AR and VR as being a part of a continuum, often represented as a line or graph. The extreme ends represent the real world and the fully virtual world. Everything in between, AR included, can be considered mixed reality.
Just Out of Reach
Despite impressive advancements and a resurgence of interest in mixed reality, there are still some major roadblocks in the way of MR headsets becoming more than just an expensive toy for the technologically-inclined to play with. A smooth and affordable experience will be necessary to gain the full appeal of the public, and some technical problems still need to be solved before the devices can reach widespread use.
"People have been working on it for over 20 years now,” Vallino said. “The problems that were really hard are still really hard, and I haven’t seen anyone solve them in a way that would really propel it into widespread usage."
The biggest problem, as Vallino explained, is seamlessly merging the virtual and real worlds.
In the completely virtual world of computer graphics, the developer can work within a three-dimensional space where every element’s position can be defined and controlled absolutely. The difficulty is in interpreting the space that a headset user is viewing and lining it up with a virtual space.
Vallino noted that one way to check if a company has really solved this issue is by observing what happens when a user reaches their hand out to a virtual object. Ideally, the hand should block the view of the object. This requires the MR software to understand the relative position of those things which are real as compared to those that are not.
Many implementations of AR simply overlay a virtual layer in front of the real world backdrop. This technique has been possible with reasonable results for a long time now. It has become relatively easy for software to recognize horizontal surfaces or the orientation of a special flat marker on which to overlay a 3D virtual environment. The challenge now is having that environment be more reactive to the real world around it. One goal, for instance, is to have objects react to real-world lighting.
Where it stands, the AR that has been in devices like smartphones for years is already suitable for many uses. An increasing number of apps use it in conjunction with the camera. As Vallino emphasized, it is still a matter of time before anything much more advanced will see such widespread use. Even so, he noted that HoloLens is starting to solve a lot of hard problems, and the recently unveiled Magic Leap One headset is looking very impressive so far.
Getting Comfy
Even once all the kinks are worked out, MR still needs to be accepted by the public before it can really shine. VR headsets have been steadily growing in appeal and ability and dropping in price, but there is no guarantee that MR headsets will follow the trend anytime soon. So far models have been clunky, if not strange looking, and their currently narrow range of uses will not warrant the price for everyone.
Vallino also mentioned that having the tech ready is a necessary part of gaining interest. There needs to be a demonstration of just how revolutionary it really is. Perhaps by the time the technology makes MR seamless, buyers will be accustomed enough to other sorts of wearable tech to find these new headsets approachable.
Companies such as Nintendo, Sega and Virtuality tried selling the world on virtual reality in the 90s to very little success. The tech just was not there yet. With new interest in AR however, MR headsets at home could very well be somewhere on the horizon.