Measuring the Latency of an Augmented Reality System for Robot-assisted Minimally Invasive Surgery
Minimal latency is important for augmented reality systems and teleoperation interfaces as even small increases in latency can affect user performance. Previously, we have developed an augmented reality system that can overlay stereoscopic video streams with computer graphics in order to improve visual communication in training for robot-assisted minimally invasive surgery with da Vinci surgical systems. To make sure that our augmented reality system provides the best possible user experience, we investigated the video latency of the da Vinci surgical system and how the components of our system affect the overall latency.
To measure the photon-to-photon latency, we used a microcontroller to determine the time between the activation of a light-emitting diode in front of the endoscopic camera and the corresponding increase in intensity of the surgeon’s display as measured by a phototransistor. The latency of the da Vinci S surgical system was on average 62 ms. None of the components of our overlay system (separately or combined) significantly affected the latency. However, the latency of the assistant’s monitor increased by 14 ms. Passing the video streams through CPU or GPU memory increased the latency to 147 ms and 256 ms, respectively.
See the full story here: http://www.scitepress.org/Papers/2017/62742/index.html
New Report from ILMxLab & Disability Visibility Project Shares Insights on VR Accessibility Design
79 VR users with disabilities from around the world participated in the survey, offering insights into their experiences with VR technology. Of the 98 different types of disabilities described, the most common were deafness, arthritis, scoliosis, cerebral palsy, autism, asthma and PTSD. ILMxLAB helped to promote the survey, and users’ experiences with their VR game Trials on Tatooine (2016) was an optional section.
The report offers six key takeaways from the survey: accessibility should be integrated into VR software from the start, VR software/hardware needs maximum flexibility and customisation, developers should interrogate cultural norms and diversify representation, be sensitive to the diverse and varied communities and disability types, and VR development teams should hire disabled people. A large portion of the report sheds light on the specific challenges that disabled users face when using VR hardware and software.
See the full story and download the report here: http://www.virtualrealitypulse.com/edition/daily-apple-razer-2017-12-06?open-article-id=7609870&article-title=new-report-from-ilmxlab---disability-visibility-project-shares-insights-on-vr-accessibility-design&blog-domain=roadtovr.com&blog-title=road-to-vr
Sundance 2018 New Frontier Further Pushes Into Future Entertainment With Virtual Reality and Artificial Intelligence

Mandatory Credit: Photo by George Frey/EPA/REX/Shutterstock (8115388g)
Snow piles up as people walk past the Egyptian Theater at the 2017 Sundance Film Festival in Park City, Utah, USA, 26 January 2017. The festival runs from 19 to 29 January.
2017 Sundance Film Festival, Park City, Usa - 26 Jan 2017
PhilNote: This article has one-paragraph descriptions supplied by Sundance of all of the New Frontier pieces.
“Wolves in the Walls (Chapter 1)” / U.S.A. (Lead Artists: Pete Billington, Jessica Shamash, Key Collaborators: Edward Saatchi, Saschka Unseld, Jennine Willett, Zach Morris) — All is not as it seems when 8-year-old Lucy‘s imagination proves to be reality. Help her discover what’s hiding inside the walls of her house in this immersive fable, based on the work by Neil Gaiman, and choreographed by New York‘s critically acclaimed immersive theater company, Third Rail. Cast: Elisa Davis, Elizabeth Carena, Cadence Goblirsch
“Chorus” / U.S.A. (Lead Artist: Tyler Hurd, Key Collaborators: Chris Milk, Megan Ellison, Justice) — Crystals, lasers, monsters, heroines. Transform into fantastical female warriors in this social virtual reality experience. Six people can band together to battle evil in this epic journey of empowerment, all orchestrated to the song “Chorus” by Justice.
Eon Studio 9
Philip Lelyveld comment: the link takes you to a useful table of EON's suite of products.]
EON Studio is our flagship 3D authoring software for developing interactive 3D applications. With EON Studio, users of all experience levels can build complex, high-quality interactive 3D content with little to no programming experience.
Whether you create 3D applications for architecture, marketing or training, EON Studio provides a unique range of benefits and functions making it a developer’s preferred choice.
See the full story here: https://www.eonreality.com/platform/eon-studio/
A look at the new BMW i3s in augmented reality with Apple’s ARKit
BMW was one of the first to jump on the opportunity earlier this year with their app BMW i Visualiser. They have now updated it with the latest versions of their BMW i3 and i8.
I do like the app – though that’s maybe more a compliment to Apple’s new augmented reality tools than to BMW.
By the way, Tesla hasn’t jumped on board the augmented reality train yet, but fans have produced third-party apps for the Model 3 similar to what BMW launched this week.
It does look like a glimpse of the future of car retail.
See the full story here: https://electrek.co/2017/12/06/bmw-i3-augmented-reality-apple-arkit/
Snapchat Debuts Six New Augmented Reality Artworks at Art Basel Miami
In celebration of this year's Miami Art week, Snapchat collaborated with artists Deborah Kass, D*Face, Hebru Brantley, Lionel Smit, Nathan Sawaya, and Daniel Arsham to translate their works into augmented reality. Using Lenses, the virtual overlays within the app that gave the world memes like the dancing hot dog, fairgoers at Art Basel and Design Miami—as well as anyone in the world—are invited to explore the six works of art in their surroundings as if they were public art installations. The works are anchored in place and rendered in 3-D, which invites viewers to walk around the sculptures and examine them from all angles.
See the full story here: https://www.architecturaldigest.com/story/snapchat-debuts-six-new-augmented-reality-artworks-at-art-basel-miami
Future Farm Hires RwE In Preparation For Spin Off Of New Augmented Reality Cannabis Company
Future Farm Technologies Inc. (the “Company” or “Future Farm”) (CSE: FFT) (OTCQB: FFRMF) is pleased to announce that it has engaged RwE Growth Partners Inc. to conduct a business valuation and fairness opinion on its recently acquired augmented reality asset (see links to Augmented Reality Presentation and Demo Video). The work is being done in preparation for a stock dividend, which will be issued to shareholders of record upon spin off that is expected to occur within the next 120 days.
“We believe that in order to maximize shareholder value, spinning off this asset is in the best interest of our shareholders,” comments William Gildea, CEO of Future Farm. “Once spun off, Future Farm’s shareholders will own shares in a new company traded on the CSE, with a singular focus on bringing augmented reality to the cannabis industry.”
See the full story here: https://globenewswire.com/news-release/2017/12/07/1247427/0/en/Future-Farm-Hires-RwE-In-Preparation-For-Spin-Off-Of-New-Augmented-Reality-Cannabis-Company.html
Qualcomm (and Machinima / WB) Has a New Reality Show Competition For Creating Virtual Reality Apps
To promote its new virtual reality platform, Qualcomm created the “VR Developer Challenge,” pairing game developers with YouTube stars in a competition to create the most engaging VR experience. The series, which debuted on YouTube last week, gives viewers a peek behind the scenes of what it takes to think of and execute a project within VR.
The show features three VR developers (Sam Maliszewski, Jordan Mann and E McNeill) selected by Warner Bros.-owned Machinima, which produced the series with Qualcomm. Each developer had a month to create a project of their choosing using Qualcomm’s new standalone headset, the Snapdragon 835 VR developer kit. The headset is more advanced than some of the mobile phone-enabled headsets, but—unlike the more popular headsets such as the Oculus Rift and HTC Vive—is untethered.
The episodes capture the developers going from concept to creation: talking about their ideas, overcoming personal obstacles, and even facing user testing.
See the full story here: http://www.adweek.com/digital/qualcomm-has-a-new-reality-show-competition-for-creating-virtual-reality-apps/
Inside China’s Big Tech Conference, New Ways to Track Citizens
An artificial intelligence company touted a robot that could help doctors with diagnoses. A start-up displayed a drone designed to carry a single passenger 60 miles per hour.
And in a demonstration worthy of both wonder and worry, a Chinese facial recognition company showed how its technology could quickly identify and describe people.
If there were any doubts about China’s technological prowess, the presentations made this week at the country’s largest tech conference should put them to rest. The event, once a setting for local tech executives and leaders of impoverished states, this year attracted top American executives like Tim Cook of Apple and Sundar Pichai of Google, as well as executives of Chinese giants like Jack Ma of Alibaba and Pony Ma of Tencent.
The technology enabling a full techno-police state was on hand, giving a glimpse into how new advances in things like artificial intelligence and facial recognition can be used to track citizens — and how they have become widely accepted here.
A fast-growing facial recognition company, Face++, turned its technology on conferencegoers. On a large screen in its booth, the software identified their gender, described their hair length and color and characterized the clothes they wore.
The people overseeing China Unicom’s booth openly discussed the data, a sign of how widely accepted such surveillance and data collection have become in China.
China Telecom showed off its ability to measure the amount of trash in several garbage cans and detect malfunctioning fire hydrants.
Investors and analysts say China’s unabashed fervor for collecting such data, combined with its huge population, could eventually give its artificial intelligence companies an edge over American ones. If Silicon Valley is marked by a libertarian streak, China’s vision offers something of an antithesis, one where tech is meant to reinforce and be guided by the steady hand of the state.
The contradictions of using sites like Twitter to change opinions abroad, while blocking them domestically, were often evident but almost as often unremarked upon.
See the full story here: https://www.nytimes.com/2017/12/05/business/china-internet-conference-wuzhen.html?_r=0
Academic Research on depth perception in VR
In 2015, Fulvio found that people were flunking her simple test of three-dimensional perception using a flat screen and standard 3D movie glasses. They were not good at discerning which direction a target was moving.
"Most importantly, they confused whether the object was coming toward them or going away from them," she says. "It was a surprising finding. Nobody believed it, because it's not something that happens often in the real world. You'd get hurt."
What Fulvio and Rokers found was that when most people put on a virtual reality headset, they still treat what they see like it's happening on any run-of-the-mill TV screen.
"There's no depth to a computer screen. There are no binocular cues. Close one eye, close the other eye, nothing changes," says Rokers, whose work was funded by Google. "If you take that expectation into a VR headset, where you do have binocular cues, you somehow just don't use them."
The visual feedback nearly doubled success rates.
When she turned off the VR system's head tracking, taking away the effects of players' head movements and making them passive viewers, they were bad again. When she gave a little of that freedom back—restoring the system's response to head movements, but making the virtual world shifts lag behind players by as much as half a second—they were still bad.
The results—that tiny head movements and typical binocular cues of motion are there for the taking in virtual reality, but that most people only use them if they are actively shown how VR differs from a flat computer screen—should help virtual reality creators improve uptake of their products.
Rokers says showing the effects of teaching people to use cues to three-dimensional motion that they are otherwise ignoring may ultimately help refine treatment for vision disorders such as blind spots or amblyopia ("lazy eye") in which the brain can be trained to compensate for perceptual limitations.
See the full story here: https://medicalxpress.com/news/2017-12-virtual-reality-users.html
Pages
- About Philip Lelyveld
- Mark and Addie Lelyveld Biographies
- Presentations and articles
- Trustworthy AI – A Market-Driven approach
- Tufts Alumni Bio