How to detect hand gesture in live webcam using javascript?

Question

I embedded live web cam to html page. Now i want to find hand gestures. How to do this using JavaScript, I don't have idea. I Googled lot but didn't get any good idea to complete this. So any one know about this? how to do this.

Answer 1

Accessing the webcam requires the HTML5 WebRTC API which is available in most modern browsers apart from Internet Explorer or iOS.

Hand gesture detection can be done in JavaScript using Haar Cascade Classifiers (ported from OpenCV) with js-objectdetect or HAAR.js.

Example using js-objectdetect in JavaScript/HTML5: Open vs. closed hand detection (the "A" gesture of the american sign language alphabet)

Answer 2

Here is a JavaScript hand-tracking demo -- it relies on HTML5 features which are not yet enabled in all typical browsers, it doesn't work well at all here, and I don't believe it covers gestures, but it might be a start for you: http://code.google.com/p/js-handtracking/

Answer 3

You need to have some motion detecting device (Camera) and you can use kinect to get the motion of different parts of the body. You will have to send data in browser telling about the body parts and position where you can manipulate the data according to your requirement

Here you can find how you can make it. Motion detection and rendering

More about kinect General info

Answer 4

While this is a really old question, theres some new opportunities to do handtracking using fast neural networks and images from a webcam. And in Javascript. I'd recommend the Handtrack.js library which uses Tensorflow.js just for this purpose.

Simple usage example.

<!-- Load the handtrackjs model. -->
<script src="https://cdn.jsdelivr.net/npm/handtrackjs/dist/handtrack.min.js"> </script>

<!-- Replace this with your image. Make sure CORS settings allow reading the image! -->
<img id="img" src="hand.jpg"/> 
<canvas id="canvas" class="border"></canvas>

<!-- Place your code in the script tag below. You can also use an external .js file -->
<script>
  // Notice there is no 'import' statement. 'handTrack' and 'tf' is
  // available on the index-page because of the script tag above.

  const img = document.getElementById('img'); 
  const canvas = document.getElementById('canvas');
  const context = canvas.getContext('2d');

  // Load the model.
  handTrack.load().then(model => {
    // detect objects in the image.
    model.detect(img).then(predictions => {
      console.log('Predictions: ', predictions); 
    });
  });
</script>

Demo Running codepen

Also see a similar neural network implementation in python -

Disclaimer .. I maintain both projects.