How to act in a virtual reality movie

If you don’t like video games, you don’t need a virtual reality headset, do you?

Wrong, according to a burgeoning group of filmmakers and performance artists. Their creations, many of which premiered at the Sundance Film Festival last month, use virtual and augmented reality to blur the lines between a film’s characters and audiences. With everything, wacky film-video game hybrids like Asteroids to the art of high-level performance such as Hero, artists explore the limits of their audience’s willingness to participate in a movie once they put on a VR headset.

The danger is that “if you allow users to have too much interactivity, they are not engaged,” said Larry Cutler of Baobab Studios, who produced Asteroids. Like many of his peers, he has roots in Hollywood – a 10-year stint developing characters at Pixar – and in the tech world, as vice president of big data company Metanautix. So it is perhaps not surprising that he refers to his audience as “users”. Indeed, most of the artists working in what they call the medium “AR / VR”, abbreviation of augmented and virtual reality, take care to make the same distinction: the public does not watch a film, it uses a product.

Like conventional video game developers, Baobab artists create their work using the Unity game development platform. The studio, along with several peers who also use Unity, brought their designs to Unity headquarters in San Francisco this week, where I got to watch them – er, I got to use them – on the HTC Vive (99 , $ 99 at Amazon), Samsung Gear VR ($ 99.99 at Amazon), and Microsoft HoloLens. Here is the best of what I have seen.

Asteroids

baobabs Asteroids animated short film is directed by Eric Darnell, of Madagascar Fame. It takes place on an alien spaceship and is a sequel to Invasion, which received rave reviews at the Cannes and Tribeca festivals last year. As a member of the audience, you take on the role of a “junior C-class task robot” and the film’s main characters – the Mac and Cheez robots and their dog, Peas – lead you.

At the beginning of the film, we have the impression of being a dog banished from his cage. Every now and then the main characters will pay you some attention, even asking you to perform a few tasks like playing catch with the dog Peas, but most of the time you just sit there. Robots don’t speak English either, so forget to understand what they’re telling you.

To explain this ban, Maureen Fan, CEO of Baobab, says that “most people don’t know what to do” when attaching a VR headset. Initially, relegating the audience to the role of antagonist serves two purposes, she says: it helps virtual reality newbies not feel overwhelmed, and it creates suspense for the film’s climax, when Cheez (voiced by Elizabeth Banks) suddenly falls ill.

Spoiler alert: it’s up to you to save Cheez. If you want revenge on her for making your life miserable, you can just let her die. If instead you’re in the mood to forgive, you can use your Vive controllers to zap her with a giant laser to restart her heart. I opted for forgiveness, and I finished the film released from my corner. I didn’t rehearse the movie to find out what would happen if you chose to let Cheez die.

The life of us

The life of us, from the studio Within, is firmly in the adventure category. It’s multiplayer, to borrow a term from the video game industry, so to fully enjoy the experience you’ll need a friend who also owns a VR headset.

The journey essentially follows the history of human evolution, so creationists can stop reading here and move on to the next section. You start out as an amoeba-like underwater creature, using Vive controllers to flap fin-shaped appendages as you follow oxygen bubbles across the water’s surface. (While you can beat as much as you want, you can’t really control where you move – it’s a movie, not a video game).

You move quickly on earth, where you transform into a succession of beings, from monkeys and birds to briefcase businessmen. Along the way, you’ll need to outrun dinosaurs, survive the desert and freezing cold, and fend off attacks from what appear to be cute lemur monkeys.

Perhaps the most unique part of The life of us is that you complete the entire story with a partner, who appears next to you in the movie at all times. Everything you say to him is recorded by Vive’s microphone and distorted into an alien voice in the movie. It’s whimsical, but hey, at least you don’t have anyone else to blame but yourself if you don’t like canned dialogue.

Hero

Have you ever wondered what David Bowie’s iconic song “Heroes” would sound like if you could see it? Advertising company MAP Design Lab did it, and this two-part experience is what it envisioned.

To watch the first part of the film, you put on a Samsung Gear VR headset. A more powerful headset like the Vive should also work, but you don’t need its motion-tracking capabilities because there’s no interactivity. Instead, you’re suspended in mid-air, watching two dancers twist together in multiple locations in and around the splendid United Artists Theater in Los Angeles. It’s like attending your own private ballet.

Once you’re done, it’s time for part two of Hero: Take off the Gear VR, put on the Microsoft HoloLens and create your own ballet. Taking advantage of HoloLens’ augmented reality capabilities, you walk through the physical room, where tiny versions of the ballet scenes from Part 1 are played out. Using voice commands, you can make the dancers bigger or stop and start the action. As you walk around the room, the HoloLens records your movements with a flowing virtual tape, which is placed at your feet at the end of the movie so you can see where you’ve “danced”.

Each of the three films should be available for Vive owners to download from the Steam store this year, their creators have said. Like its prequel, Asteroids will likely also be available for the Oculus Rift ($ 99.99 on Amazon). It should be noted, however, that you will not be able to experience the second part of Hero unless you’re willing to shell out the thousands of dollars it takes to own a pre-production HoloLens.