Microsoft keyboard prototype integrates gesture control

Microsoft Research has designed a keyboard that integrates gesture controls for a more seamless user interface.

The Leap Motion gesture control system seemed pretty exciting at the time — but when it finally launched, users found that it didn't have many practical, everyday uses. Part of the problem was that it had its own software ecosystem. But another part was that, as a piece of hardware, it didn't work together particularly well with other desk hardware.

Most people are used to having their hands on keyboard and mouse, resting on the desk, rather than hovering somewhere 30 centimetres above it. This is what Microsoft Research is seeking to solve with a new prototype keyboard, titled Type–Hover–Swipe in 96 Bytes.

The core of the hardware is a mechanical keyboard, but — as you might have guessed — it integrates motion control. Moreover, it does so in such a way that the user's hands never really need to be more than a centimetre or two above the keyboard, meaning you can maintain the comfortable position you're accustomed to.

To achieve this, it uses a 16 x 4 grid of 64 low-resolution infrared proximity sensors. These are embedded in the base of the keyboard, between the keys, which are spaced a little wider to allow them room (you don't want to go spilling your chip crumbs in there). These sense both the proximity and the motion of the hands above the keyboard.

Although they're low-resolution — just 64 pixels — it has a frame rate above 300 Hertz, which means it can track rapid movement with ease. A set of gesture recognition algorithms are able to recognise what Microsoft Research calls "motion signatures" — programmed gesture presets based on motion history data that allow the keyboard to recognise specific gestures as commands.

These include swiping a hand across the keyboard to navigate, making a pinching gesture to zoom, drawing a simple line with an index finger to switch between applications and even holding two fists above the keyboard to simulate a driving motion for a game.

The research paper was presented at the ACM CHI Conference on Human Factors in Computing Systems 2014 in Toronto. Microsoft has made no mention of bringing the keyboard to production — but it looks like the most intuitive integration of motion control we've seen to date.