Virtual reality technologies are advancing into more and more areas. The simulation of environments, coupled with technologies such as virtual reality glasses, offers almost unlimited possibilities. However, VR systems are still rarely used in everyday applications.
Most of today’s VR applications are either operated with controllers that are held in the user’s hand or with hands in the air so that the position can be captured by a camera. The user is also typically standing during the interaction. Typing on a virtual keyboard, for example, presents another problem: the fingers move only slightly, and cameras cannot capture the movement as precisely as current mechanical keyboards do. With in-air typing, the usual haptic feedback is also lacking.
Researchers at ETH Zurich have developed TapID, a wrist-based inertial sensing system that complements headset-tracked hand poses to trigger input in VR.
The prototype embeds several acceleration sensors in a normal, flexible rubber wristband. From the combination of registered signals, TapID reliably detects surface touch events. More importantly, thanks to vibrations, the sensors can identify which finger the person has used for touching the surface – such as a table, a wall, or even your own thigh. Because when you type, each finger makes the wrist vibrate differently. This enables hands-free typing in virtual space; for example, the table surface or your own thigh can be used as a keyboard.
The researchers found that their novel sensor design can detect tiny differences in the vibration profile on the wrist in order to differentiate between each characteristic finger movement. A custom machine learning pipeline the researchers developed processes the collected data in real-time. In combination with the camera system built into a set of VR glasses, which captures the position of the hands, TapID generates extremely precise input.
The researchers have already tested their development on a virtual keyboard and a virtual piano. However, the method does not only work with the rubber wristband; it could also transfer to existing fitness wristbands and everyday smartwatches because they are all equipped with inertia sensors.
“Our sensor solution is portable, and it has the potential to make VR systems suitable for productivity work on the go. TapID allows users to operate applications with their hand or thighs – anywhere and any time,” says Christian Holz, a professor at ETH Zurich’s Institute for Intelligent Interactive Systems.
At the end of March, the researchers will present their development at the IEEE VR conference. Then they want to improve the technology with more test subjects and develop more applications to integrate TapID.
