Xbox 720 may project game graphics into real world: Microsoft files patent for the Holodeck

"Immersive display system" could support 3D, true "off-screen" gameplay

Microsoft has filed a patent for an "immersive display system" which would project game graphics right into your living room, possibly in 3D, and allow you to interact with them using an enhanced version of Kinect. It's the Holodeck, in short.

The patent, filed 2nd March 2011 and published 6th September, notes that "interactive media experiences, such as video games, are commonly delivered by a high quality, high resolution display.


"Such displays are typically the only source of visual content, so that the media experience is bounded by the bezel of the display. Even when focused on the display, the user may perceive architectural and decorative features of the room the display is in via the user's peripheral vision.

"Such features are typically out of context with respect to the displayed image, muting the entertainment potential of the media experience.

"Further, because some entertainment experiences engage the user's situational awareness (e.g., in experiences like the video game scenario described above), the ability to perceive motion and identify objects in the peripheral environment (i.e., in a region outside of the high resolution display) may intensify the entertainment experience."

The proposed "immersive display system" comprises (deep breath) "a peripheral input configured to receive depth input from a depth camera; a primary display output configured to output a primary image to a primary display device; an environmental display output configured to output a peripheral image to an environmental display; a logic subsystem operatively connectable to the depth camera via the peripheral input, to the primary display via the primary display output, and to the environmental display via the environmental display output; and a data-holding subsystem" which allows the beast to track a user's position, output the aforesaid peripheral image such that it appears to be an extension of the image on-screen.

Elsewhere in the patent, the inventor elaborates that "the peripheral image may include images of scenery and objects that exhibit the same style and context as scenery and objects depicted in the primary image, so that, within an acceptable tolerance, a user focusing on the primary image perceives the primary image and the peripheral image as forming a whole and complete scene.


"In some instances, the same virtual object may be partially displayed as part of the primary image and partially displayed as part of the peripheral image."

The environment display could take the form of "a wide-angle RGB projection display device configured to project a peripheral image in a 360-degree field around the environmental display." Ergo, you'll be able to look at or even interact with things behind the telly.

It might also project in 3D, both the more usual kind where users don active shutter glasses, or the fancy 3DS kind where parallax barriers or lenticular lenses create an autostereoscopic "glassless 3D" effect.

The environment display and depth sensors could be built right into the Kinect unit, which may use eye-tracking to gauge the user's position.

Leaked Xbox 720 design materials from 2010 make mention of just such a combination of 3D, Kinect and environment projection. There's tonnes more through the link. Have a read and let us know what you think.