Immersive Multiplayer Gaming System Integrates Virtual and Physical Worlds

Virtual Gaming in Action

Tedjokusumo, et al

Wii wand not cutting it for you anymore? A group of game designers at the National University of Singapore have developed a next-gen immersive gaming system that blends virtual reality and augmented reality with the physical environment. Wearing head-mounted displays and using other peripherals, players use the open space around them to move, jump, crouch and aim within a game, collecting virtual objects and fighting opponents real and virtual along the way.

The group created a few variations of the same first-person shooter (FPS), one of which works through virtual reality, another through augmented reality. In the virtual reality orientation, the players wear head-mounted displays and control either a sword or gun in the game using a peripheral wand. A system of sensors tracks the head display – which projects images of the in-game scenes in front of the players' eyes – as well as the wand, giving players real time feedback on the virtual environment. Players can move around the game by physically moving about the room (which more or less needs to be obstacle free), picking up virtual objects and engaging in virtual duels and shootouts by actively aiming, ducking and maneuvering.

But while virtual reality is undeniably awesome next to the usual keyboard or paddle-controlled FPS, the augmented reality variation takes it to the next level, letting gamers play in any open environment. In this setup, players wear an augmented reality headset with a camera looking forward that captures the users' real-world views. The game engine then syncs the real-world environment with game data, so players can run, jump and hide within the real world environment while shooting, slashing and otherwise racking up points.

One key innovation to make the systems work well allows for a split screen option that shows where the head and weapon are looking simultaneously. In cases where the head and weapon are pointing in different directions, a small window showing the weapon's viewpoint opens at the bottom of a player's view, allowing him or her to aim. The feature lets players to shoot one direction while looking another – because who doesn't want to cap off a Killtacular with a showboating shot from behind the back?

In some preliminary tests, users seemed to prefer the virtual reality console to both AR and traditional keyboard and mouse controls, though objective results showed users made more bull's-eye shots with the keyboard and mouse. The system, especially in virtual reality configuration, very accurately mimics body movement as well as aiming/shooting, so the systems could have important police and military training implication aside from some pretty sick potential for turning a living room, a park, even a whole neighborhood into an augmented gaming playground.