philip lelyveld The world of entertainment technology

Hands-on: FOVE’s Eye-tracking VR Headset Was the Next Best at CES

While the latest FOVE prototype looks similar on the outside, it’s using totally different display tech: currently a single 1440p LCD panel with a field of view that felt like it matched that of the Oculus Rift DK2. Though the current prototype lacks positional tracking (the ability to track the head through 3D space), the rotational tracking felt better than every other new VR headset I tired at CES, save for the Oculus Rift Crescent Bay prototype. It’s good to see that they’re focused on latency, because it’s the keystone that many entrants to the consumer VR headset market are currently lacking.

On top of a 1440p display, solid headtracking, and an a decent field of view, FOVE’s real trick is its ability to track the wearer’s eyes.

Inside the headset, each lens is surrounded on the top, bottom, left, and right with IR LEDs which illuminate the eye, allowing cameras inside to detect the orientation of each. Looking through the lenses looks just like you’d expect from a VR headset without eye-tracking; Wilson told me that this is different from other VR headset eye-tracking solutions, which can have components that obscure parts of the field of view.

As it stands, FOVE is putting a fair amount of emphasis on the ability to aim with your eyes (probably because it’s easy to show and easy to understand), but to me that’s a red herring; what excites me about eye-tracking are the more abstract and enabling possibilities like eye-based interface and contextual feedback, simulated depth-of-field, foveated rending, and avatar eye-mapping. 

See the full story, with expanded explanations of eye-based interface and contextual feedback, simulated depth-of-field, foveated rending, and avatar eye-mapping, here: http://www.roadtovr.com/fove-eye-tracking-vr-headset-hands-on-ces-2015/

Augmented Reality: The Smithsonian’s New App Brings Museum Skeletons and Fossils To Life

[Philip Lelyveld comment: Atlanta’s Fernbank Museum did something like this back in 2011.  By looking through VR binoculars as you walked around the natural history museum, you could see virtual wild animals roaming the halls. More and more museums and public spaces are exploring the use of immersive VR and AR.]

...the Smithsonian has got your back during field trips with a new iOS app called Skin & Bones that brings fossils to life using 3D animation and augmented reality.

The Smithsonian's National Museum of Natural History's Bone Hall is one of the institution's most popular exhibits, housing close to 300 skeletons and fossilized specimens. And using the new Skin & Bones app, 13 skeletons in the collection can now be brought to life by simply pointing the iPad or iPhone's camera at the exhibit. A 3D version of the animal will come to life, accompanied by educational animations and factoids.

See the full story here: http://gizmodo.com/the-smithsonians-new-app-brings-museum-skeletons-and-fo-1679251273

What comes after Google Glass? It was at CES, just in pieces

[Philip Lelyveld comment: This article is a good overview of AR glasses at CES.  Our old friends from the 3D Lab era, Vuzix, are repurposing their glasses for the AR age.]

To capture a mainstream audience, smart glasses need to be compact and stylish enough that they feel just like a regular pair of glasses. The technology needs to disappear. The closest thing I saw at CES that fits that bill is Vuzix's waveguide technology (similar to fiber optics), which creates images on lenses — no "prisms" or weird outgrowths from your temple. That way, the frames look normal, and the lenses stay transparent.

Vuzix's consumer model hadn't been perfected in time for the show, but I did see a working pair of prototype lenses, as well as the frames in which the company intends to build them. They looked just like something Bono would wear if Vuzix can put them together, which is likely, now that Intel has invested $25 million in the company.

"What Google was trying to do, Vuzix is going to do," company CEO Paul Travers said. "Once people start using these guys, they won't go back to their phones."

See the full story here: http://mashable.com/2015/01/09/ces-2015-google-glass-smart-glasses/

Hands-on: Augmented and virtual reality come together in Sulon Cortex

[Philip Lelyveld comment: The Sulon Cortex is the first publicly displayed prototype to demonstrate that the line between virtual and augmented reality storytelling is going to disappear as technology advances.  Virtual and Augmented are the two ends of a continuum of possibilities.]

If you thought VR headsets like the Oculus Rift and Gear VR looked funky, wait until you see the Sulon Cortex. On the front, it looks a lot like those headsets, but then it adds a bulbous growth on its backside. ...

But that bulb-like structure on the back has an important purpose: it houses multiple sensors (the specifics of which the company is staying tight-lipped on) that can map your surroundings. The team tells us that the headset can detect details about your environment – including how far away walls are and what other objects (like furniture) lie in between – and incorporate them into the virtual/augmented world that you see on its display. ...

In our demo, after strapping on the Sulon Cortex, an ordinary meeting room looked almost the same as it did without the headset, including the Sulon reps who were there chatting with us. But – what's this? – on one end of the room sat a swirling and glowing portal. This is, of course, demonstrating the augmented portion of the Sulon Cortex.

As for the virtual part? Well, after being egged on by the Sulon team, I walked through the portal and watched as augmented reality turned into virtual reality. Suddenly I was on a platform in the middle of a fiery pit, with no remaining signs of the meeting room. I could walk around freely (which, unlike with other VR headsets, means physically walking), but the edges of the platform designated where I could go without crashing into walls.  ...

Walking through the portal again brought me back into the meeting room, and, while chatting with the Sulon team, I began to take the headset off. "But wait," they warned me, "what's that behind you?" Holy shitballs, the portal was still there, and the not-quite-slain Hydra suddenly emerged and enjoyed a crunchy Gizmag writer for a mid-afternoon snack (an unfortunate ending to a fascinating demo).

Soon a vicious Hydra emerged from the flames, and I was challenged to defeat it using hand gestures (the headset senses hand position and movement, down to the individual fingers). The gesture tracking was pretty clunky in our demo, but, again, this is a prototype and still a ways off from being a ready-to-roll consumer product.

...

It can go full-on virtual, it can go augmented or – like in this demo – it can cleverly transition from one to the other, and back again.

...

The completely wireless Sulon Cortex dev kit will be available in Q1 or Q2, and will cost developers US$500 a pop. You can read more at the company's page below.

See the full story here: http://www.gizmag.com/sulon-cortex-augmented-virtual-reality-headset/35485/

Smart Helmet Maker Skully Has Raised Another $1.5 Million

[Philip Lelyveld note - this could take market share away from Daqri and other hardened AR HMD makers.]

Last summer, Skully, the motorcycle helmet with a heads-up display, became the fourth-most popular Indiegogo campaign in history. According to an SEC filing, it’s raised another $1.5 million.

Its helmets provide an added layer of safety for thrill-seeking motorcycle riders by providing hands-free GPS navigation and calling capabilities, wide-angle rear-view camera displays on the helmet screen, Bluetooth connectivity, an integrated audio system and streaming music.

See the full story here: http://techcrunch.com/2015/01/14/smart-helmet-maker-skully-has-raised-another-1-5-million/?ncid=tcdaily

Virtual and Augmented Reality at CES 2015 – a recap

By Philip Lelyveld

Virtual Reality at CES

On a panel addressing the future of Virtual and Augmented Reality, Razer CEO Min-Liang Tan voiced the opinion that VR is 24 months away from going mainstream. A rush to a consumer product, he said, will not do justice to either the consumer or the VR community. The dilemma they face is they are trying to build a product without knowing what the final deliverable that will succeed in the marketplace should be.

This was the undercurrent throughout the show, as most vendors described their VR and AR products as ‘works in progress’ or prototypes. They each hope to engage the developer and creative communities in fleshing out a full experience ecosystem around their product before making a major marketing push.

There are two types of VR HMDs (Head Mounted Displays); fully integrated HMDs and HMD cases into which you insert a mobile phone or tablet.

Oculus, the highest visibility player in the integrated HMD (head mounted display) VR space, advanced their demo of the HMD from a sit-down experience last fall to a stand-up experience at CES. They demo’d fully rendered scenes in which you could walk up to, and look under and over, objects. Audio distinctly positioned in the spherical space was used to guide where to look. Although their product looks ready for market, Oculus is still shipping prototype HMD units to developers under terms that prohibit public demonstrations.

Avegant is positioning their Glyph HMD as a music and movie player. While the unit will come with built-in motion sensors, Avegant will not market the Glyph as a VR device for at least a year.

The Sony Morpheus HMD was notably absent from the Sony booth.

Fove’s HMD with built-in eye-tracking offered a remarkably comfortable and responsive experience, but their one unit was clearly a hand-assembled prototype. Having eye tracking built into the HMD is useful for a number of reasons. It allows for more efficient use of the CPU because the image only needs to be rendered in full detail in the area where the person is looking. Eye tracking within the HMD can also help developers understand where people are looking, and help the creatives develop the language of VR storytelling faster. Tokyo-based Fove was being actively courted by venture capitalists while I was visiting their booth.

Other HMD vendors, including ImmersiON-VRelia, 3D Head, and AntVR, showed HMDs that either offered inferior experiences in their rush to market, or were also clearly works-in-progress.

Virtuix Omni provided the best public demonstration of immersion in a VR experience. Virtuix sells a circular platform with a waist-high railing and a bowl-shaped slippery base. When you put on their booties with embedded sensors, you can run in place in the real world, which simulates running forward in the virtual world. The demonstrated gaming experience integrated this walking controller with a full size position-sensing rifle and an Oculus HMD. Watching the gameplay on a screen beside the professional demonstrator, it was easy to see how good this type of VR experience could be. The Virtuix Omni is priced at $499, which is within the budget of the serious gamer.

Razer is working to accelerate both the development of VR platforms and the language of VR by open sourcing their entire HMD and associated software specifications. As CEO Min-Liang Tan put it, every person thinks their idea is the best. By having open competition, the truly best will rise to the top.

To that end, Mr. Tan also announced at CES the formation of the Open Source VR (OSVR) Alliance to encourage cooperation among input, game, and output developers. Leap Motion, Sixence, Nod, and GearBox are founding members (www.osvr.com)

Adam Orth, the creative director at The One Zone, noted that game developers have been locked into three game consoles for years. Unlike game play on a flat screen, when you turn around in an HMD, your brain builds a 3D model of the world. VR offers game developers a new creative environment, he said, rich with possibilities and pitfalls.

Of the many HMD cases on view at CES into which you insert a cell phone or tablet, the Samsung Gear VR is the only one we saw that is being positioned as more than a generic box with lenses into which you can insert a selection of smart displays. Oculus and Samsung worked together to develop the Gear VR HMD. It only accepts the Samsung Galaxy Note 4 phone. When it launched last fall, Samsung announced VR content from almost 20 VR content providers, including movie and game studios.

The two Gear VR demos in the Oculus booth showed both some strengths and weaknesses of phone-based VR.

Wild, a VR movie experience developed by Fox to compliment their theatrical movie Wild, stars Reese Witherspoon and Laura Dern. The live action short completely surrounds you with image and sound. The camera is stationary. You are able to spin around and see everything, but leaning forward does not bring you closer to objects (as the fully integrated Oculus HMD did). You are positioned within the world of Wild directly between Witherspoon and Dern, so you had to turn 180 degrees to see each of them. Surround spatial audio was used effectively to let you know where you should look.

The Gear VR demo of a CGI-rendered first-person shooter game in which you fly among asteroids and shoot objects was more satisfying because, unlike Wild, you were free to roam and create your own ride-like experience. Directing your flight by tilting and turning your head was very intuitive and great fun. Controlling the flying experience by tilting and turning your head seemed to both override any potential sensory mismatch issues and overcome the Gear VR’s lack of sensors to detect when you are leaning toward something for a closer look.

Augmented Reality at CES

A number of companies showed augmented reality glasses this year. Some were modified versions of hardware developed for special situations such as harsh military and industrial environments, while others started from scratch to target the consumer space. We did not see any entertainment-related AR demos.

Placing the virtual image directly in the wearer’s field of view, rather than off to the side as Google Glass does, may reduce eye strain and allow for longer use of the AR glasses. None of the companies have attempted to map the real world and overlay virtual images onto it so they appear to fit naturally into the environment. Magic Leap, the company that Google and others have invested $452M in, has a portfolio of patents that cover a wide range of components that could enable a realistic augmented reality experience. The company is hiring subject-matter experts to make augmented reality glasses a reality. Magic Leap is in deep stealth mode. While we did run across some Magic Leap employees at CES, none would discuss their work.

Sony SmartEyeglass augmented reality glasses use two forward-facing green-light projectors built into see-thru glasses to overlay data at a controlled stereoscopic distance in space onto the real world. Sony will sell the glasses at “a high price point” to consumers in a few select markets starting this quarter. More importantly, as with many of the VR companies, Sony hopes to interest and involve developers worldwide and build out an ecosystem of apps over time. This is an early proof-of-concept product that needs better design and miniaturization, but it does demonstrate the potential of augmented reality well.

Daqri continues to show industrial-hardened helmets with heads-up visor displays. They are also developing games and moving into the consumer augmented reality space.

Osterhout Design Group (ODG), which for six years has developed heavy-duty smart glasses for the military, is developing a consumer version of augmented reality smart glasses. They describe the glasses, which will sell for around $1,000, as falling somewhere between Google Glass and Oculus Rift. ODG's Android specs are intended for a different purpose than Google Glass. Where Glass brings you glanceable information to enhance the moment you're in, augmented reality offers near-full immersion, they claim.

Atheer has been showing augmented reality glasses with gesture controls for at least 3 years. This year’s model of Augmented interactive Reality (AiR™) Smart Glasses continues their effort to develop AR glasses as worker productivity tools for field services, oil and gas, warehousing, healthcare, and much more. For example, the AiR Smart Glasses can help quickly identify parts and perform hands-free barcode scanning.

Epson showed their Moverio full color stereo AR glasses and shared their booth with a number of firms who demonstrated how they use the Moverio AR glasses in educational, industrial, and training environments. Scope AR, for example, used the forward looking camera in the glasses, plus image recognition software and a database of parts, to identify an industrial part on a physical model, and then overlay the matching virtual part on it with associated text information. One other vendor played the Avengers movie on the Epson AR glasses, with a tinted adapter to make the experience more immersive. It wasn’t a true AR experience, but it did demonstrate the media player capabilities of the gear.

Why the quantum dot is the hottest TV tech going

[Philip Lelyveld comment: the full article contains an excellent explanation of how quantum dots work.]

Known alternately as QDots or nanocrystals, quantum dots are the foundation behind what is expected to be one of the biggest sea changes in the television market over the next few years.

At the moment, there are several different approaches to integrating quantum dots into displays. One, used by companies like LG, Samsung, and 3M, is to spread the dots on thin film, which is then added to the stack of films beneath a display. The other is a thin tube coated with quantum dots that’s used in edge-lit LCD backlit configurations, Gagnan explained.

Gagnan said the tube approach — which is from QD Vision — is less expensive, because it uses fewer dots. But not every manufacturer is set up to use tubes. In both cases, however, an advantage over OLED is that existing manufacturing facilities can easily incorporate quantum dots.

In a standard LCD TV, a white light shines through a color filter, generating a picture by blending red, green, and blue lights and projecting them onto pixels. That approach, however, has its limitations. “Today’s TVs use LEDs to provide the ‘white backlight,” according to Digital Trends, “but here’s the problem with this setup: LEDs suck at producing white light.”

The technology is meant to amp up the power of the white light generated by LEDs. “When you illuminate them with a blue LED, they emit full-spectrum light,” Gagnan said.

“Since the size of a quantum dot can now be precisely controlled,” wrote Digital Trends, “the resulting light they put out can be dialed in just as precisely.”

See the full story, with videos, here: http://venturebeat.com/2015/01/08/why-the-quantum-dot-is-the-hottest-tv-tech-going/

Hollywood Tries Its Hand at Virtual Reality Content

What they lack in content is being provided by apps from third-parties like Jaunt, a VR cameramanufacturer looking to stimulate the market with programming such as a Paul McCartney concert video that will bring you closer to the Beatle than a front-row seat.

These are the tentative first steps to reach a user base that has yet to materialize. But if it can match even a fraction of the passion of the nascent VR industry ready to reach it, there’s reason for high hopes. Research firm KZero Worldwide estimates there were only about 200,000 VR users in 2014, but projects that market will mushroom to 170 million worldwide by 2018.

As Oculus co-founder Palmer Luckey put it at the Oculus Connect conference in September: “Without content, nobody would be interested in this whole virtual reality thing.”

“There are no real marching orders, because there’s no defined place where things will end up,” said Ted Schilowitz, who leads the Fox Innovation Lab at 20th Century Fox. “You have to be very nimble at this point.”

Every major studio is already dipping a toe in these waters, with VR extensions to popular franchises including Fox’s “The X-Men,” Warner Bros.’ “Batman” and Paramount’s “Interstellar” and “How to Train Your Dragon” just a few examples of what’s in circulation as installations at events. Gear VR also boasts derivative content from Legendary, Marvel, Imax, Vevo and Cirque du Soleil. There are also plenty of independent projects like “Zero Point,” billed as the first original VR movie by startup Condition One. Even this month’s Sundance Film Festival is catching VR fever, with nine entries employing the technology featured in a program devoted to innovative filmmaking.

We’ve learned you have to create the content for the experience; just converting what you’ve got doesn’t work. You don’t get the immersion you’re looking for,” said Warren Mayoss, head of technology product development at DreamWorks Animation.

DiCarlo said Samsung believes vidgamers represent the most likely consumers of VR, but the company is aiming wider. “If VR is only ever for gaming, it is a good business,” he noted. “Expand it past gaming, it will be a great business.”

“If Magic Leap makes projection-based VR the thing we need to focus on at certain point, then we want to be flexible enough to transition to whatever the right thing is at the right time,” said James Milward, president of Secret Location, which produced a VR experience for the Fox series “Sleepy Hollow.” “VR is an optimal platform that we’ll go hard in, but we won’t be boxed in.”

See the full story here: http://variety.com/2015/digital/features/hollywood-tries-its-hand-at-virtual-reality-content-1201392775/

S.F virtual reality accelerator will promote the technology

Next month, 10 startups will move into Rothenberg Ventures’ South of Market office as the first class in its River accelerator. The companies will each receive $100,000 in investment and mentoring during their three-month stay.

“There’s hundreds if not thousands of people who have been building the various parts of the ecosystem already, (but) funding hasn’t really been there for really passionate people to quit their day jobs to build companies up until now,” Rothenberg said.

Rothenberg, 30, says River is the first accelerator for virtual reality companies. Having invested in virtual reality startups, he was surprised to find there were no accelerators devoted to the sector.

“The investor in us was kind of disappointed in that, and the entrepreneur in us said 'OK then, we’ll build that,’” he said.

See the full story here: http://www.sfgate.com/business/article/S-F-virtual-reality-accelerator-will-promote-the-6005337.php

Next Gen Art Event: Putting games in galleries

At the Fries Museum in Leeuwarden, The Netherlands, such prejudices seem to have been dispelled. Their New Horizons exhibition celebrates the power of the virtual landscape, displaying videos, concept drawings and promotional pieces from a variety of video games alongside the work of prominent artists such as Jennifer Steinbeck, Harun Farocki and Cory Arcangel.

The exhibition finds wonder in even the most seemingly mainstream and consumer-focused of games, encouraging visitors to view them in a new light. Screenshot prints of the multiplayer maps of Battlefield 4, with all soldiers and action absent, are strangely haunting and affecting when taken out of their usual context. The reality-bending gameplay of Portal seems that much more thoughtful when shown alongside the influential impossible spaces of M.C. Escher.

See the full story here: http://www.gamesindustry.biz/articles/2015-01-12-next-gen-art-event-taking-video-game-art-seriously