Here’s How Microsoft Is Making 3D Videos For HoloLens

When Microsoft unveiled holographic Minecraft on HoloLens at this year’s E3, the crowd let out a huge cheer. Now Microsoft has released a video showing how it creates its stunning high quality free-viewpoint video by capturing a traditional Māori haka war dance. Other action that they grabbed includes martial arts, break dancing, and a little boy destroying some Solo cups with an axe.

The HoloLens headset mixes reality with impressive and sometimes lifelike holograms that users can interact with. To capture dynamic actions in such great detail, Microsoft created a huge TV studio in Redmond, WA. The space is outfitted with a large calibrated green screen and 106 synchronized RGB and infrared cameras. All of these cameras take in information at various angles and compile it together so that they can create realistic three-dimensional models and spaces. And unlike the process that was required to capture the performance of Gollum in Lord of the Rings, no special green screen suit is required by the performer.

The cameras turn the action into a 3D point cloud, then algorithms further refine the model into tens of thousands of points, which is ultimately reduced to thousands of triangles per frame. Greater detail is put in places such as hands and faces before texture is finally added.

Although the HoloLens has been criticized for having a limited field of view, these incredibly detailed performances will undoubtedly change the way we consume entertainment in the future.

Far away from the Redmond studio, astronauts on the International Space Station will eventually test HoloLens in hopes that they will soon be able to give people a firsthand look at what they’re seeing. If the visuals are anywhere near as awe-inspiring as what astronaut Scott Kelly posts on Twitter, they’re bound to be popular.