philip lelyveld The world of entertainment technology

Life through Lens: inside Google’s plan to re-skin reality

Originally locked to the Google Pixel range, Google has opened up the SDK called ARCore 1.0. It’s now out of preview and comes complete with improvements and learnings Google has amassed since first announcing ARCore in August 2017 and before that with its Project Tango platform.

“It’s the ultimate try before you buy,” says Amit Singh, vice president of AR and VR at Google.

“The ability to re-skin reality is compelling here,” says Aparna Chennapragada, Product Lead of Lens. “But to even create that space you need to know what the semantic meaning is of what you are looking at, and that is why we started Lens.

“Yes, we’re putting digital into the real world but we’re attaching semantic meaning to it. If you want to know something about an object, you tap into the rest of Google. The knowledge graph, maps, the whole shebang.

"Lens is a powerful visual intelligence layer.”

“On the web, you know something in a bigger font is important, something that starts a paragraph is another thing. We use all of that experience and apply it to being able to browse the real world.”

“It sounds deceptively simple but it’s incredibly powerful,” continues Chennapragada. “Say, for something like Coachella - you have a lineup on a poster and you want to see what each of these bands are like. You can use Lens, immediately find them on YouTube, and instantly get a sense of who is playing there.”

See the full story here: https://www.techradar.com/news/life-through-lens-inside-googles-plan-to-re-skin-reality

Google AI now can give YouTube videos a wacky background

Machine learning proves its worth for new video effects tech: distinguishing between faces and backgrounds at 100 frames per second.

Google, though, trained a neural network with lots of carefully labeled imagery that helped it learn how to distinguish facial features -- eyes, hair, glasses, mouths and so on -- from everything else. The result is a system that can swap out backgrounds fast enough to keep up with video. Digital video frames whip by at 30 frames per second, but Google's technology works at 40 frames per second on a Google Pixel 2 phone and more than 100 frames per second on an Apple iPhone 7.

Want to try it out for yourself? Sorry. For now, it's available to only a pretty small number of high-profile YouTubers with access to the YouTube stories service, which offers an ability to share Snapchat-esque short videos.

Google detailed the research in a blog post Thursday.

"Our new segmentation technology allows creators to replace and modify the background, effortlessly increasing videos' production value without specialized equipment," Google programmers Valentin Bazarevsky and Andrei Tkachenka said in the blog post.

See the full story here: https://www.cnet.com/news/google-ai-can-give-youtube-videos-a-wacky-background/

Home Society & Culture Society Virtual Reality Technology Enters A Chinese CourtroomSOCIETYVirtual Reality Technology Enters A Chinese Courtroom

On Thursday, March 1, at a courtroom in Beijing’s No. 1 Intermediate People’s Court, a man surnamed Dong became the first person in China using virtual reality technology to revisit a crime scene from last September, where he was the only witness at the scene.

Wearing a VR headset and with his hands on a console, Dong immersed himself in the middle of the scene, while a projector screen in front of him simultaneously showed what he saw to everyone in the courtroom, including judges and lawyers.

The case that Dong is testifying about, according to Legal Daily, concerns a 30-year-old suspect named Zhang who is charged with murder. On September 13, 2017, the suspect came to an office building in Haidian District in Beijing to argue with his 19-year-old girlfriend, Liu. During the dispute, Zhang stabbed Liu to death after he cut himself with a knife three times.

The introduction of VR technology is part of a “courtroom evidence visualization system” developed by the local court. The system also includes a newly developed computer program that allows lawyers to present evidence with higher quality and efficiency, which will replace a traditional PowerPoint slideshow.

It is reported that the system will soon be implemented in courtrooms across the city of Beijing.

See the full story here: http://supchina.com/2018/03/01/virtual-reality-technology-enters-a-chinese-courtroom/

‘Myst’ creators surprise with announcement of Firmament – new, virtual-reality based game

Cyan Worlds, creators of the wildly popular 1990s “Myst” computer game franchise, released an unexpected teaser Thursday morning for a new game, called “Firmament,” whose future remains up in the air.

“Honestly, it’s a really cool project,” said Rand Miller, chief executive officer and co-founder of the company. “Hopefully we’ll get to the point where we can make it happen.”

The team has developed what Miller called a working “experience,” similar to a demo, that will be shown later this month at the Game Developers Conference in San Francisco in an attempt to garner interest and financial backing. It features the Mead-based company’s signature puzzle-solving gameplay, he said, in a world built from the ground up to be experienced through a virtual reality headset.

Virtual reality theme park opens at Dubai Mall

The two-storey attraction comes complete with 18 experiences suitable for different ages. Entry to VR park is free, with each activity and experience priced individually from Dhs15 to Dhs45.

Combining augmented reality, virtual reality and experiential attractions, expect to feel like you’re propelling down the side of Burj Khalifa in “Burj Drop”, or hold on tight as your “Dubai Drone Taxi” spins out of control on a rollercoaster adventure ride.

Adrenaline junkies should head to “Plummet”, the skydiving experience giving aerial views of the city, or take on a hair-raising “Dune Bash” experience featuring encounters with camel spiders, scorpions and snakes.

Level 2, The Dubai Mall, Downtown Dubai, 10am to 11pm Sunday to Wednesday,  10am to 1am Thursday to Saturday, free entry, experiences priced from Dhs15 to Dhs45. VRparkdubai.com

See the full story here; http://whatson.ae/dubai/2018/03/virtual-reality-theme-park-opens-dubai-mall/

FoxNext VR Selects ‘Planet of the Apes’ for its First VR Game

VentureBeat reports that, in “Crisis on the Planet of the Apes,” “players can expect to band together with other captive apes, confront human violence and channel primate instincts to survive.”

“VR has always held the promise of bringing together the cinematic storytelling of film and the deeply immersive interactivity of games,” said FoxNext VR Studio senior vice president/general manager Brendan Handler. “‘Crisis on the Planet of the Apes’ VR takes this potential to the next level. We set out to give the fan a lead role in this dark Planet of the Apes world — to feel the ape’s strength, power and movement as he fights his way to safety.”

See the full story here: http://www.etcentric.org/foxnext-vr-selects-planet-of-the-apes-for-its-first-vr-game/

Song Made By AI Gets Stunning 360° Music Video

Taryn Southern is an artist, singer, producer and popular YouTuber. Danny Bittman is one of the most prolific creators of immersive art in the world today. And together they have created Life Support — a piece of music composed and created entirely by artificial intelligence.

Danny Bittman: Taryn and I first worked closely for a month to define the story. We ultimately wanted to display someone who was plugged into a fantastical motion based immersion system, I.E their life support system, but we wanted to convey that some kind of balance between VR and RR is necessary in order to maintain sanity.

I wanted to keep the streak going though, so I used the virtual overlay in conjunction with Adobe Premiere’s live 360 previewer. It was quiet a dream as a traditional filmmaker to utilize my old workflows while working with this new Medium. Actually, the only difference between traditional film editing and this 360 workflow was that instead of having a 2D monitor taking up my UI space, my world became my monitor. I had so much more room in Adobe for my timeline and color correction nodes.

We knew most people would be watching this video on their phones, not on headsets – so we wanted to be cognizant to create a great visual experience for them. The downside, of course, is then you have to make choices that can negatively effect the headset experience.

At the end of the day, while Taryn focuses of music and I focus on visuals, we are both just trying to figure out how to partner with emerging technology to prototype and polish our ideas faster. While Taryn worked with Amper’s AI to create Life Support, she still used her creativity to select, tweak, and add to the ideas her computer generated.

See the full story here: http://www.virtualrealitypulse.com/edition/daily-razer-htc-2018-02-20?open-article-id=7855675&article-title=song-made-by-ai-gets-stunning-360--music-video&blog-domain=vrscout.com&blog-title=vrscout

High Fidelity Takes Us Dancing In A Virtual Reality Club

Yes, I went dancing in the dance club in VR, and talked to a bunch of people all at once. I even allowed strangers to touch my avatar in VR so that I could physically feel what it was like to have a sense of touch in the cyber world.

The experience felt real, as the movements of the real dancers were in sync with their virtual characters. High Fidelity made it possible by bringing down the latency, or interaction delays, to about a tenth of a second. The quick interaction, combined with the lights and the sound of music and voices, made the dancing experience more believable in VR, Rosedale said.

We saw a dancer floating in the air, like they do in Ready Player One. And we saw a dancer upside down, defying gravity, walking on the bottom of a crystal disco ball. We gathered around Rosedale and struggled to listen to him amid the dance music. But I could hear him. The audio seemed to come from different directions, but I had trouble identifying which avatar was talking until I saw the avatar’s lips move in sync with the voice.

High Fidelity brought in a half-dozen dancers and equipped them with HTC Vive VR headsets. They also had Vive Trackers, or sensors that were attached to their legs, hips, and backs. Those sensors captured their full body movements, and High Fidelity translated those moves into much better action on the dance floor. I could only wave my arms around, but those dancers were moving everything.

See the full story here: https://uploadvr.com/high-fidelity-takes-us-dancing-virtual-reality-club/

Paul Allen Wants to Teach Machines Common Sense

“To make real progress in A.I., we have to overcome the big challenges in the area of common sense,” said Mr. Allen, who founded the software giant Microsoft in the 1970s with Bill Gates.

But these machines struggle with other basic tasks. Though Amazon’s Alexa does a good job of recognizing what you say, it cannot respond to anything more than basic commands and questions. When confronted with heavy traffic or unexpected situations, driverless cars just sit there.

A.I. “recognizes objects, but can’t explain what it sees. It can’t read a textbook and understand the questions in the back of the book,” said Oren Etzioni, a former University of Washington professor who oversees the Allen Institute for Artificial Intelligence. “It is devoid of common sense.”

Mr. Allen helped fund Cyc, and he believes it is time to take a fresh approach, he said, because modern technologies make it easier to build this kind of system.

Mr. Lenat welcomed the new project. But he also warned of challenges: Cyc has burned through hundreds of millions of dollars in funding, running into countless problems that were not evident when the project began. He called them “buzz saws.”

See the full story here: https://www.nytimes.com/2018/02/28/technology/paul-allen-ai-common-sense.html

Who is seeding the AR Cloud

Technologically speaking, enabling spatial computing needs a Universal Visual Browser (UVB) that can detect the world around it with hyper-precision, and filter contextually relevant data to be displayed to the user based on location, activity, or time of day. The UVB will interface with everything we own, everyone we know, and everywhere we go.

The big dog of computer vision is Amazon Web Services (AWS) which offers Amazon Rekognition (sic) – "Image Detection and Recognition Powered by Deep Learning."

In November 2017, AWS introduced Rekognition Video, which extracts even more data, correctly identifying the activity, direction, and intention of people and objects, making the system capable to tracking subjects that are obscured.

"Blippar is where AR and AI meet," says Tayeb. "It's not enough to identify a woman and a stroller. A universal visual browser will also understand the relationship. The stroller could have a baby inside it and the woman is possibly the mother. It's not just visual, it's contextual. That's what makes Blippar unique."

Toronto based startup Vertical.ai, a 2015 Y- Combinator graduate which has raised 2M to use computer vision and simultaneous localization and mapping (SLAM) to build an AR Cloud, just announced the launch of Placenote. an app that allows mobile developers to build AR apps that permanently place virtual objects in the real world for others to discover. If Blippar works at the city scale, Placenote works best at the room scale.

“One of the big things holding back engaging AR is for content to feel like it’s actually physically part of the world,” 6D.ai CEO Matt Miesnieks told TechCrunch in an interview.

Miesnieks describes 6D.ai as "Waze for AR". The app would take all those rooms, all those locations, all those cities, and stitch together buildings, objects, landmarks and other camera data to create a world map, both inside and outside the home, school, office, or wherever the app is used. In addition to the providing developers the 3D structure of the world without needing depth cameras, 6D.ai will semantically identify 3D objects to enable intelligent AR applications. Miesnieks told me in an email 6D.ai will have significant company news within the next four weeks. The company first garnered attention when Tim Cook visited Oxford in 2017 and expressly asked to see Prisacariu's project.

Dent Reality of London is announcing a family of developer tools to make it simple to create AR location experiences, beginning with "Point Of Interest" experiences.The toolkit uses Computer Vision to identify the surrounding landscape, and then precisely overlays digital content.

Charlie Fink is a former Disney & AOL exec and Forbes columnist. In the 90s, he ran VR pioneer Virtual World. He's the author of Charlie Fink's Metaverse, An AR Enabled Guide to VR & AR.

See the full story here: https://www.forbes.com/sites/charliefink/2018/03/01/who-is-seeding-the-ar-cloud/#50055e7f5ae2