Virtual reality can be amazing if done right. Oculus, HTC & Steam, Sony and others have shown us some great VR experiences. However for the most part when you're in the VR environment and move your body in the real world that same movement is not reflected in the VR environment.
Researchers at Microsoft have brought that capability a bit closer, with an accurate and flexible hand-tracking system called Handpose that utilizes a standard Xbox One Kinect motion sensor without any hardware modifications.
The system uses the depth camera and software developed by the research team to track the movement of your hands and fingers and transfer them to the screen. Microsoft are not the first to demonstrate this ability but are doing so with off the shelf hardware available today.
Shown in the video below the system is very flexible in terms of camera placement and the distance at which it works accurately. In a video, the researchers show how they can walk several meters from the camera and even out the office door and Handpose continues to read their gestures accurately.
It even works when the camera is shaking around. All these advancements make the system much more reliable and therefore a better candidate for real world use. Microsoft will present Handpose at the Computer Human Interaction conference this week in Seoul.
Kinect with the Xbox one already tracks fully body movements and gestures like a kick or a punch and side to side movement. Tracking wrist and finger movements is much harder to solve, as movement is more varied, less pronounced and more difficult to perceive.
Handpose or similar technology could be used in a gaming system like Kinect, or in a virtual reality headset like the Oculus Rift, allowing people to reach out and see real world body movement reflected in the VR world.
With the Oculus purchase of Nimble VR we can be assured that Oculus are working on something similar.
Microsoft Shows Handpose Technology with Accurate Hand-Tracking for VR" data-width="50" data-layout="button_count" data-action="like" data-show-faces="true" data-share="true">