philip lelyveld The world of entertainment technology

Virtual and Augmented Reality at CES 2015 – a recap

By Philip Lelyveld

Virtual Reality at CES

On a panel addressing the future of Virtual and Augmented Reality, Razer CEO Min-Liang Tan voiced the opinion that VR is 24 months away from going mainstream. A rush to a consumer product, he said, will not do justice to either the consumer or the VR community. The dilemma they face is they are trying to build a product without knowing what the final deliverable that will succeed in the marketplace should be.

This was the undercurrent throughout the show, as most vendors described their VR and AR products as ‘works in progress’ or prototypes. They each hope to engage the developer and creative communities in fleshing out a full experience ecosystem around their product before making a major marketing push.

There are two types of VR HMDs (Head Mounted Displays); fully integrated HMDs and HMD cases into which you insert a mobile phone or tablet.

Oculus, the highest visibility player in the integrated HMD (head mounted display) VR space, advanced their demo of the HMD from a sit-down experience last fall to a stand-up experience at CES. They demo’d fully rendered scenes in which you could walk up to, and look under and over, objects. Audio distinctly positioned in the spherical space was used to guide where to look. Although their product looks ready for market, Oculus is still shipping prototype HMD units to developers under terms that prohibit public demonstrations.

Avegant is positioning their Glyph HMD as a music and movie player. While the unit will come with built-in motion sensors, Avegant will not market the Glyph as a VR device for at least a year.

The Sony Morpheus HMD was notably absent from the Sony booth.

Fove’s HMD with built-in eye-tracking offered a remarkably comfortable and responsive experience, but their one unit was clearly a hand-assembled prototype. Having eye tracking built into the HMD is useful for a number of reasons. It allows for more efficient use of the CPU because the image only needs to be rendered in full detail in the area where the person is looking. Eye tracking within the HMD can also help developers understand where people are looking, and help the creatives develop the language of VR storytelling faster. Tokyo-based Fove was being actively courted by venture capitalists while I was visiting their booth.

Other HMD vendors, including ImmersiON-VRelia, 3D Head, and AntVR, showed HMDs that either offered inferior experiences in their rush to market, or were also clearly works-in-progress.

Virtuix Omni provided the best public demonstration of immersion in a VR experience. Virtuix sells a circular platform with a waist-high railing and a bowl-shaped slippery base. When you put on their booties with embedded sensors, you can run in place in the real world, which simulates running forward in the virtual world. The demonstrated gaming experience integrated this walking controller with a full size position-sensing rifle and an Oculus HMD. Watching the gameplay on a screen beside the professional demonstrator, it was easy to see how good this type of VR experience could be. The Virtuix Omni is priced at $499, which is within the budget of the serious gamer.

Razer is working to accelerate both the development of VR platforms and the language of VR by open sourcing their entire HMD and associated software specifications. As CEO Min-Liang Tan put it, every person thinks their idea is the best. By having open competition, the truly best will rise to the top.

To that end, Mr. Tan also announced at CES the formation of the Open Source VR (OSVR) Alliance to encourage cooperation among input, game, and output developers. Leap Motion, Sixence, Nod, and GearBox are founding members (www.osvr.com)

Adam Orth, the creative director at The One Zone, noted that game developers have been locked into three game consoles for years. Unlike game play on a flat screen, when you turn around in an HMD, your brain builds a 3D model of the world. VR offers game developers a new creative environment, he said, rich with possibilities and pitfalls.

Of the many HMD cases on view at CES into which you insert a cell phone or tablet, the Samsung Gear VR is the only one we saw that is being positioned as more than a generic box with lenses into which you can insert a selection of smart displays. Oculus and Samsung worked together to develop the Gear VR HMD. It only accepts the Samsung Galaxy Note 4 phone. When it launched last fall, Samsung announced VR content from almost 20 VR content providers, including movie and game studios.

The two Gear VR demos in the Oculus booth showed both some strengths and weaknesses of phone-based VR.

Wild, a VR movie experience developed by Fox to compliment their theatrical movie Wild, stars Reese Witherspoon and Laura Dern. The live action short completely surrounds you with image and sound. The camera is stationary. You are able to spin around and see everything, but leaning forward does not bring you closer to objects (as the fully integrated Oculus HMD did). You are positioned within the world of Wild directly between Witherspoon and Dern, so you had to turn 180 degrees to see each of them. Surround spatial audio was used effectively to let you know where you should look.

The Gear VR demo of a CGI-rendered first-person shooter game in which you fly among asteroids and shoot objects was more satisfying because, unlike Wild, you were free to roam and create your own ride-like experience. Directing your flight by tilting and turning your head was very intuitive and great fun. Controlling the flying experience by tilting and turning your head seemed to both override any potential sensory mismatch issues and overcome the Gear VR’s lack of sensors to detect when you are leaning toward something for a closer look.

Augmented Reality at CES

A number of companies showed augmented reality glasses this year. Some were modified versions of hardware developed for special situations such as harsh military and industrial environments, while others started from scratch to target the consumer space. We did not see any entertainment-related AR demos.

Placing the virtual image directly in the wearer’s field of view, rather than off to the side as Google Glass does, may reduce eye strain and allow for longer use of the AR glasses. None of the companies have attempted to map the real world and overlay virtual images onto it so they appear to fit naturally into the environment. Magic Leap, the company that Google and others have invested $452M in, has a portfolio of patents that cover a wide range of components that could enable a realistic augmented reality experience. The company is hiring subject-matter experts to make augmented reality glasses a reality. Magic Leap is in deep stealth mode. While we did run across some Magic Leap employees at CES, none would discuss their work.

Sony SmartEyeglass augmented reality glasses use two forward-facing green-light projectors built into see-thru glasses to overlay data at a controlled stereoscopic distance in space onto the real world. Sony will sell the glasses at “a high price point” to consumers in a few select markets starting this quarter. More importantly, as with many of the VR companies, Sony hopes to interest and involve developers worldwide and build out an ecosystem of apps over time. This is an early proof-of-concept product that needs better design and miniaturization, but it does demonstrate the potential of augmented reality well.

Daqri continues to show industrial-hardened helmets with heads-up visor displays. They are also developing games and moving into the consumer augmented reality space.

Osterhout Design Group (ODG), which for six years has developed heavy-duty smart glasses for the military, is developing a consumer version of augmented reality smart glasses. They describe the glasses, which will sell for around $1,000, as falling somewhere between Google Glass and Oculus Rift. ODG's Android specs are intended for a different purpose than Google Glass. Where Glass brings you glanceable information to enhance the moment you're in, augmented reality offers near-full immersion, they claim.

Atheer has been showing augmented reality glasses with gesture controls for at least 3 years. This year’s model of Augmented interactive Reality (AiR™) Smart Glasses continues their effort to develop AR glasses as worker productivity tools for field services, oil and gas, warehousing, healthcare, and much more. For example, the AiR Smart Glasses can help quickly identify parts and perform hands-free barcode scanning.

Epson showed their Moverio full color stereo AR glasses and shared their booth with a number of firms who demonstrated how they use the Moverio AR glasses in educational, industrial, and training environments. Scope AR, for example, used the forward looking camera in the glasses, plus image recognition software and a database of parts, to identify an industrial part on a physical model, and then overlay the matching virtual part on it with associated text information. One other vendor played the Avengers movie on the Epson AR glasses, with a tinted adapter to make the experience more immersive. It wasn’t a true AR experience, but it did demonstrate the media player capabilities of the gear.