ThermalRing: Gesture and Tag Inputs Enabled by a Thermal Imaging Smart Ring

The heterogeneous and ubiquitous input demands in smart spaces call for an input device that can enable rich and spontaneous interactions. We propose ThermalRing, a thermal imaging smart ring using low-resolution thermal camera for identity-anonymous, illumination-invariant, and powerefficient sensing of both dynamic and static gestures. We also design ThermalTag, thin and passive thermal imageable tags that reflect the heat from the human hand. ThermalTag can be easily made and applied onto everyday objects by users. We develop sensing techniques for three typical input demands: drawing gestures for device pairing, click and slide gestures for device control, and tag scan gestures for quick access.

Our current implementation of ThermalRing uses a MLX90640 thermal camera module for thermal imaging. The module consists of a thermal camera and a STM32F103 microcontroller. The thermal camera is chosen because of its wide Field of View (110^o × 75^o ) for gesture capture, low resolution (32 × 24) for user privacy, as well as small size and low power consumption (20mA typical) for ring integration.

ThermalRing can recognize a user’s drawing gestures on a flat surface in an asymmetrical bimanual interaction setup. This allows the user to pair and interact with different smart devices remotely via gestures. Specifically, the user places the auxiliary hand on any surface, palm down and stretching the index finger and thumb in orthogonal directions. The index finger and thumb together indicate the interaction area. During interaction, the user uses the index finger of the interacting hand to perform drawing gestures. A gesture starts when the finger touches down on the surface and ends when the finger is lifted up.

Such asymmetrical bimanual interaction is used for three reasons. 1) Asymmetrical bimanual interaction is in line with many common user activities and feels natural. For example, a user usually need to anchor a paper using the auxiliary hand first, then write on it using the interacting hand; 2) By allowing the user to naturally place the entire interacting hand on the surface, such interaction requires less physical efforts compared with mid-air interactions and capacitive touch panels; 3) Users can better locate imaginary UI elements and interact within the recommended area by referring the auxiliary hand.

After paired with the target device, it is more convenient to continue control the devices with the same bimanual setup by manipulating virtual buttons and sliders in the interaction area via click and slide gestures. The user can refer to the locations and visual features of the auxiliary hand’s thumb and index finger to localize virtual UI elements. Figure shows an virtual interface with two buttons and one slider

Such asymmetrical bimanual interaction is used for three reasons. 1) Asymmetrical bimanual interaction is in line with many common user activities and feels natural. For example, a user usually need to anchor a paper using the auxiliary hand first, then write on it using the interacting hand; 2) By allowing the user to naturally place the entire interacting hand on the surface, such interaction requires less physical efforts compared with mid-air interactions and capacitive touch panels; 3) Users can better locate imaginary UI elements and interact within the recommended area by referring the auxiliary hand.

Thermal cameras can identify materials with different heat reflectivity. We leverage this nature and propose ThermalTag-thin and passive tags made of high heat reflection materials and applied on surfaces with lower heat reflectivity. When close to human hand, the tag reflects the heat from the hand and form contrast on the thermal image between the tag and the surface. The thermal image can then be processed to recognize the tag shape.

In this application, the tags with different shapes (e.g. triangle for ’Play’, rectangle for ’Stop’) are applied on surfaces of various objects. For example, users can apply tags on a desk for TV control, on a wall for AC control, or even on a cup handle for lights control. One advantage of ThermalRing is that such tags can even be embedded inside a surface since ThermalRing is robust under different illumination conditions. Users can easily make such tags by cutting widely available materials like aluminum foil or tapes.