5Oct/14Off
The Last Medium
[Philip Lelyveld comment: This story is an excellent overview of recent events and current thinking on VR in entertainment. It is good prep material for the upcoming ETC DTS.]And stories, as Marshall McLuhan famously observed, adapt to the mediums that convey them. The ultimate story of virtual reality may turn out to be a story about stories and how we co-evolve with them.
Whether or not we’re consciously aware of its existence, we’re all fluent in the language of cinema. We’re native speakers. Virtual reality is in such a germinal phase that to use it as a medium to tell stories is to participate in creating a whole new language. The morphemes of cinema — framing, cutting, close-ups, pans, zooms — disappear or stop making sense. It’s no longer obvious how a filmmaker directs attention or advances plot. What about perspective and point of view? Should the viewer be free to explore a world or follow a set path through a story — on rails, as gamers say? Should characters respond to your presence?
“It’s bigger than color,” said Chris Milk, the influential music-video director, multimedia artist, and technologist. “It’s bigger than sound. It’s the audience literally inhabiting the narrative.”
Last year, Milk filmed the artist Beck and his father, composer David Campbell, performing an elaborate arrangement of David Bowie’s “Sound and Vision” — in the round, accompanied by more than 200 musicians, including an orchestra, a gospel choir, and a gamelan ensemble — and later stitched the footage together to put a virtual reality viewer in the room with Beck. “People have watched this and talked to me months later and said that the way that they remember it is not as a piece of media that they consumed. They remember it as a place in time that they existed within,” he said. “It’s making a deeper memory.”
I was in Framestore’s Culver City office. Framestore, a visual-effects studio that won an Academy Award for its work on Gravity, was an early backer of the Rift on Kickstarter. Mike Woods, the executive creative director, said they were commissioned by a live-events company to create a Game of Thrones experience that would take festival-goers up the Wall. …
As someone who went to film school and studied mise-en-scène, classical narrative, and framing, Woods finds it intriguing that he now has to learn a whole new language: “You can’t cut. You can’t fade. You can’t move the camera. You can’t pull focus. What you can do is create an intriguing hybrid of Brechtian in-the-round theatrical stuff and game design.” If you’re going to get hit in the chest with a flaming arrow, before that can happen, three perfectly timed and directed arrows have to be released (one to call your attention, one to focus it, and one for good measure) so that by the time the fourth one hits, you’re looking right at it.
FOR SEVERAL YEARS, Félix Lajeunesse and Paul Raphaël, partners in the Montreal-based Felix & Paul, have explored different ways to create cinematic experiences without relying on traditional storytelling conventions. … “Cinema tends to be really preoccupied with always moving on to the next thing, cutting to something else,” Lajeunesse said. “But in VR, we feel an urge to slow down, to land and really explore the moment.” ... Lajeunesse and Raphaël want viewers to stay focused on a moment for five, six, even ten minutes. “For us, it’s not just about the content itself,” Lajeunesse told me. “It’s also about creating the right conditions for the viewer to feel a part of it.”
Jaunt VR, which is based in Palo Alto and is located somewhat ingloriously above a mattress showroom, is developing a system for capturing and delivering virtual reality films: cameras, software, and, eventually, a player. The version of the Jaunt VR camera I saw — an orb covered with lenses… the image is not so much captured as computationally assembled. It’s not a record; it’s a construct. We are talking not about things here but ideas. “It’s a notional camera,” said Scott Broock, Jaunt’s V.P. of content. As for what it creates: “You can think of it as image, but really it’s just data for the brain.”
Shot at 60 frames a second — more frames and therefore much sharper than traditional film — the footage is startlingly detailed and life-like. ... This isn’t film. It’s something completely different. It’s all about experience.”