Hand gesture recognition

This study takes place in a large research project (projet REVES) in collaboration with academics and industrials and the French Cité des sciences, in order to build a long-term show of the Cité des Sciences based on interactive vision technologies, which should start on june 2009.

The main theme of this show is "Earth seen from the sky". It will show informations about Earth observation satellites and photographs they can take. All those informations will be displayed in an augmented reality goggle which will superpose images and data over the real world.

The goal of this project is to detect the naked hands of the user and to determine their position and their configuration (closed fist, one finger showing or touching something, etc.). To do so, we use a miniaturized video camera included in the augmented reality goggle which records the user's field of view in the real world (not with the displayed picture) and try to recognize into the video frames the hands, using the adaptation of evolutionary algorithm techniques.

The algorithm we use is the generic Hough technique, which consists in seeking for pre-defined models in an image.

Results

The following images shows examples of models we can use :

Figure 1 : Examples of hand models

Then, in an image shot by the video camera, we try to find those models, which give the following results : Figure 2 : Examples of hand recognition

[Video file: result1.avi]: Video of an execution example. File format: avi - Codec: Xvid - Resolution: 160x120 - Framerate: 30 fps - Duration: 33 sec. - File size: 1.3 MB Caption: The blue form correspond to the recognised model The red dots correspond to the x and y repartition of the genetic population The yellow cross corresponds to the position of the cursor associated with the hand gesture

• Benoit Kaufmann
• Évelyne Lutton
• Jean Louchet