An entirely controller-free gaming experience for Microsoft’s Xbox 360 will be commercially available as early as Christmas 2010 for as cheap as $80, according to recently leaked information from a secret Microsoft tour in the United Kingdom. Known as Project Natal, the new technology will allow gamers to seamlessly interact with virtual worlds and characters by speaking, gesturing and presenting physical objects to a 3D camera.
According to the inside source, Microsoft already has 14 games in the works. One is called Ricochet, in which a player uses his entire body to bounce balls at blocks and score points. In Paint Party, gamers create an onscreen masterpiece by posing in the shape of a desired stencil, choosing colors through speech recognition, and splashing paint onto a virtual canvas with throwing motions.
This project vision video features some other possibilities, including martial arts between a gamer and an onscreen character—without the need for cumbersome sensors—and the ability to embody a Godzilla avatar on a destructive rampage.
In this earlier video, a woman interacts with a realistic virtual world and a pixilated boy named Milo, who can recognize the emotion in her voice, hold a conversation and even comment on a drawing she holds up to the camera.
The wizard behind these unprecedented gaming features is a small black box called the Project Natal sensor, which is compatible with existing Xbox 360 consoles. The magic box relies heavily on sophisticated software for human-computer interaction. “The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain,” Johnny Chung Lee, a researcher in the Applied Sciences group at Microsoft, wrote on his blog. “The sophistication and performance of the algorithms rival or exceed anything that I’ve seen in academic research, never mind a consumer product.”
Working together, a depth camera, a multi-array microphone, and a set of specialized sensors enable full body 3D motion capture, as well as face, voice and object recognition. “Essentially we do a 3D body scan of you. We graph 48 joints in your body and then those 48 joints are tracked in real-time, at 30 frames per second,” project director Alex Kipman said in one interview. Not only will Natal register just how forcefully you kick a virtual soccer ball or how skillfully you jump on your digitized skateboard, it will recognize how many people are playing with you, distinguishing between familiar gamers and guests. Depending on how far away you are from Natal, the sensors can even differentiate the individual fingers on your hand.
The applications of such interactive technology are not limited to videogames. “These are not just the kind of experiences that you’re going to see with the 360,” Phil Spencer, general manager of Microsoft Game Studios, told The Seattle Times. “It’s a foundational event for the company, to think about how we will evolve our platforms.” If things keep going well, perhaps Microsoft will liberate laptops of keyboards and mice in time for Christmas 2011.