Jaunt VR Takes You Up A Mountain

Virtual reality is full of tantalizing potential, but has a bit of a chicken-and-egg problem.  Users won’t pony up for even a budget headset until there’s cool stuff to see, yet producers don’t want to invest in creating VR “experiences” without a big user base to pursue.  We’ll have to wait for the launch of the Oculus Rift next year for a true deluge of VR games and big-budget experiences, but in the meantime, a few interesting forays trickle out.  Last week, the New York Times released a haunting VR documentary about child refugees from 3 different war zones.

And yesterday, Jaunt Studios and The North Face released a virtual Nepalese mountain adventure.  Beginning near Kathmandu’s “monkey temple,” the experience follows mountaineers Renan Ozturk and Tim Kemple as they ascend up from about 3,000 feet through the Khumbu glacier to about 20,000 feet on the Lobuche Peak.  The new production gives Jaunt’s camera and post-production platform a chance to shine.

The “experience” is undoubtedly too short, lasting just a few minutes, but it is indeed memorable.  Particularly striking are the shots of Ozturk walking through the thriving commercial district of Thamel, with the viewer seeming to hover behind him, but able to gaze in any direction.  (The experience was filmed just two months before the calamitous earthquake on April 27, so the footage takes on a poignant quality).

But of course the real star is the mountainous scenery, and that’s where the heavy-duty hardware and software capable of imaging 360 degrees in three dimensions really pays off. Though Jaunt announced a brand new VR camera rig codenamed the “Neo” this summer, the company shot the February footage with its older unit – which consists of a 3D-printed housing for 16 modded GoPro cameras.

There are open-source 360 camera and VR software projects online for creating basic scenes, but if you’re going all the way to Nepal, you’re better off using a professional class rig, like Jaunt’s or Samsung’s Project Beyond. That’s because conventional VR camera arrays create “monoscopic” 360-degree images of a space that can then be rendered in software into a sphere — with the user’s vantage point in the center.  But there is no depth to the resulting VR world, because of the lack of separate feeds for the left and right eye.  “We call that ‘2-and-a-half’ D,” says Scott Broock, Jaunt’s VP of content.

The way traditional filmmakers get around the “2-and-a-half-D problem” is by shooting two identical versions of the same scene, using a pair of stereoscopic cameras. Both versions are projected on top of one another, and once a viewer dons polarized glasses, each eye sees only one of the two feeds, allowing the brain to process the footage as having depth.

But for shooting 360-degree VR, with as many as 16 different cameras shooting in all directions around a room, simply pairing cameras won’t work.   That’s because the point where each GoPro camera sensor obtains an image from the lens is on the circumference of the array, not the center. “You can’t collapse all of those cameras into one single point in space,” says Broock.  “So we took the Silicon Valley approach of letting software solve a physics problem.”

Proprietary algorithms synchronize each of the 16 HD video streams and transform every point in space to appear as if it were shot by a single “virtual camera” in the center of the array.  And from that virtual camera footage, the software then computes a separate feed for both the left and right eye.  With each of the 16 cameras producing 60 frames per second, the software must process nearly two billion pixels per second.

The sheer computational complexity underlying post-production perhaps explains why the Nepal VR experience is so short.  Also, while the actual shots on the glacier and mountain are indeed impressive, your mileage may very depending on the quality of your headset.  Using an iPhone 6 and Google Cardboard, the 3D effect is pronounced, as long as your head wasn’t moving – but panning around the scene causes unpleasant blurring.  Visitors who live near The North Face outlets in New York, San Francisco, or London will undoubtedly get a better experience using high-quality units in the stores.   And the experience is occasionally marred by some soapy narration about the mountains that sounds like something the dudes behind the Point Break redo left on the cutting room floor: “There’s little words to describe it.  Just deep breaths.  And memories that will last forever.”

But until VR evolves past its chicken-and-egg problem, we’ll have to endure producers skimping on the screenwriting budget.  And for those of us who can’t make it to Nepal in person, and don’t have access to a developer’s Oculus Rift, the virtual mountaineering experience will probably convince a few more people to deem Google Cardboard a “just good enough” plan B.