Gesture Recognition with Kinect-like Devices

In the last decade, many authors have investigated and studied touchless and gestural interactions as a novel tool for interacting with personal computers, as well as with large displays and tablets or smartphones. There are many devices suitable for gesture recognition: while prior works were based on vision-based algorithms only, nowadays the growing availability of low cost depth cameras allow for more sophisticated solutions.

The USI group developed many solutions in order to recognize mid-air gestures, considering both hands and the whole body. In particular, we addressed both the need of recognizing static poses (i.e. postures) and dynamic gestures, proposing valid alternatives to the wide range of available solutions from literature, with the aim of being used in real world scenarios, i.e. in real time.

To this end, our group focused on the use of Kinect-like devices, i.e. low cost sets of sensors able to gather RGB-D of low-medium quality. Using these data as input, we have been able to recognize hand and body gestures, and to apply these algorithms in real-time applications. The following video shows an example of application:

Our work in gesture recognition has led to several publication in the field of pattern recognition and machine learning. The following list includes the most relevant papers in this area:

Publications

• F. Milazzo, V. Gentile, S. Sorce, and A. Gentile, “Real-time Body Gestures Recognition using Training Set Constrained Reduction,” in Complex, Intelligent, and Software Intensive Systems: Proceedings of the 11th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS-2017), L. Barolli and O. Terzo, Eds., Springer International Publishing, 2018, pp. 216-224.
  [Bibtex]

  @inbook{Milazzo2017CISIS,
  author = {Fabrizio Milazzo and Vito Gentile and Salvatore Sorce and Antonio Gentile},
  title = {{Real-time Body Gestures Recognition using Training Set Constrained Reduction}},
  booktitle = {{Complex, Intelligent, and Software Intensive Systems: Proceedings of the 11th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS-2017)}},
  year = {2018},
  location = {{Torino, Italy}},
  editor = {Leonard Barolli and Olivier Terzo},
  publisher = {Springer International Publishing},
  pages = {216--224},
  isbn = {978-3-319-61566-0},
  doi = {10.1007/978-3-319-61566-0_21}
  }

• V. Gentile, S. Sorce, A. Malizia, and A. Gentile, “Gesture recognition using low-cost devices: Techniques, applications, perspectives [Riconoscimento di gesti mediante dispositivi a basso costo: Tecniche, applicazioni, prospettive],” Mondo Digitale, 2016.
  [Bibtex]

  @article{Gentile2016MondoDigitale,
  title = {{Gesture recognition using low-cost devices: Techniques, applications, perspectives [Riconoscimento di gesti mediante dispositivi a basso costo: Tecniche, applicazioni, prospettive]}},
  author = {Vito Gentile and Salvatore Sorce and Alessio Malizia and Antonio Gentile},
  year = 2016,
  journal = {{Mondo Digitale}},
  publisher = {{AICA - Associazione Italiana per l'Informatica ed il Calcolo Automatico}},
  url = {http://mondodigitale.aicanet.net/2016-2/articoli/02_Riconoscimento_di_gesti_mediante_dispositivi_a_basso_costo.pdf}
  }

• V. Gentile, S. Sorce, and A. Gentile, “Continuous Hand Openness Detection Using a Kinect-Like Device,” in 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems, 2014, pp. 553-557.
  [Bibtex]

  @inproceedings{Gentiile2014CISIS,
  author = {Vito Gentile and Salvatore Sorce and Antonio Gentile},
  booktitle = {{2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems}},
  title = {{Continuous Hand Openness Detection Using a Kinect-Like Device}},
  year = {2014},
  pages = {553-557},
  keywords = {computer animation;gesture recognition;image sensors;interactive devices;solid modelling;Kinect-like device;Microsoft KinectTM;animation;continuous hand openness detection;hand opening degree;human hand gesture;level-based estimation;three-dimensional model;Animation;Neural networks;Real-time systems;Reliability;Sensors;Solid modeling;Three-dimensional displays;3D animation;depth data;gesture recognition;human-computer interaction;microsoft kinect},
  doi = {10.1109/CISIS.2014.80},
  month = {July},
  url = {http://ieeexplore.ieee.org/document/6915573/}
  }

• S. Sorce, V. Gentile, and A. Gentile, “Real-Time Hand Pose Recognition Based on a Neural Network Using Microsoft Kinect,” in 2013 Eighth International Conference on Broadband and Wireless Computing, Communication and Applications, 2013, pp. 344-350.
  [Bibtex]

  @inproceedings{Sorce2013BWCCA,
  author = {Salvatore Sorce and Vito Gentile and Antonio Gentile},
  booktitle = {{2013 Eighth International Conference on Broadband and Wireless Computing, Communication and Applications}},
  title = {{Real-Time Hand Pose Recognition Based on a Neural Network Using Microsoft Kinect}},
  year = {2013},
  pages = {344-350},
  keywords = {gesture recognition;human computer interaction;image colour analysis;image denoising;neural nets;object detection;palmprint recognition;pose estimation;Kinect device;Microsoft Kinect sensor;body gestures detectopm;body gestures recognition;body parts;color information;depth information;hand pose detection;immersive multimedia environments;input data fluctuation;neural network;noise reduction;real-time hand pose recognition;skeleton;time average;user interaction;Arrays;Biological neural networks;Lighting;Neurons;Skeleton;Training;Microsoft Kinect;gesture recognition;gesture-based interaction;human-computer interaction},
  doi = {10.1109/BWCCA.2013.60},
  month = {Oct},
  url = {http://ieeexplore.ieee.org/document/6690908/}
  }