How to select proper backfacing camera in Javascript?

Question

I am using navigator.mediaDevices.getUserMedia to open MediaStream from camera device within a web browser. My app wants to do some realtime image processing in WebAssembly and for that, I need to provide a live stream of images directly from the camera.

My solution works pretty well on most devices, however, I have a problem on devices with multiple back-facing cameras, such as Samsung Galaxy S10 on Google Chrome for Android. The problem is that the following snippet:

const constraints = {     audio: false,     video: {         width: { min: 640, ideal: 1280, max: 1920 },         height: { min: 480, ideal: 720, max: 1080 },         facingMode: { ideal: 'environment' },     } }; const stream = await navigator.mediaDevices.getUserMedia( constraints ); 

always opens a wrong camera - the wide-lens camera which does not support autofocus and provides images that are too distorted for my code. The wide-lens camera is good for landscape photography but it is terrible for barcode and text scanning.

How can I select the correct camera by using MediaTrackConstraints? I've tried adding also

focusMode: { ideal: 'continuous' } 

to the constraints (according to MDN documentation, this should be a possible constraint for image tracks) but it does not seem to work.

I've also tried enumerating all devices (from this SO answer), but I don't know how to properly select the correct camera.

Notably, this code snippet:

const devices = await navigator.mediaDevices.enumerateDevices(); devices.forEach( ( device: MediaDeviceInfo ) => {     console.log( "Found device: " + JSON.stringify( device ) ); }); 

produces the following console output:

Found device: {"deviceId":"default","kind":"audioinput","label":"","groupId":"4852f187ff6a41e6d3fb3ba41c4897f46bd8ff153579da6fcb8f485432a32f66"} Found device: {"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","kind":"audioinput","label":"","groupId":"c2cfc78763f7668263b0033c44d0f906ca0f33264ebfa6b96e9846265a21ff09"} Found device: {"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","kind":"audioinput","label":"","groupId":"7a86866423279d7b1e12dbe585a14a677a2f2df4e41ec5d388b6c90f7319e88d"} Found device: {"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","kind":"videoinput","label":"camera2 1, facing front","groupId":"b1bd1a6ed8a87cd07ca0fa84744ae515b1ab2bed61cc257765c37d3426269af7"} Found device: {"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","kind":"videoinput","label":"camera2 3, facing front","groupId":"a23c2f0e311c0ca0c80a56a8b5ff7c1f8aa093df4f6ac080b051c1d95f60a94e"} Found device: {"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","kind":"videoinput","label":"camera2 2, facing back","groupId":"9b6b1a429e0db2d5094ddebe205d23309464650d8bcd585b2fe4ae8196b86f1c"} Found device: {"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","kind":"videoinput","label":"camera2 0, facing back","groupId":"0de9556e1253763d7203b3b9e5db313cf89e05dd4bdd4ea0c5aff52d2952cf11"} Found device: {"deviceId":"default","kind":"audiooutput","label":"","groupId":"default"} 

The correct camera has the label camera2 0, facing back and for some reason Chrome always selects camera2 2, facing back.

Of course, I could hardcode the selection of camera by its label, but that would work only on Samsung Galaxy S10 and I would like my code to work on any device having multiple back-facing cameras.

I still haven't tried running my page on iPhone 11 Pro (which has 3 back-facing cameras), but it works correctly on Huawei Mate 30 Pro (4 back-facing cameras) and Oppo Reno 9. Also, the problem seems to related to Google Chrome on Android. When I open my page on Firefox for Android, the browser asks me to select which camera should be used just after closing the camera permission dialog and only if multiple cameras satisfy the given constraints. That is quite fair, as it makes it possible to select the correct camera for performing the scan. I haven't tried opening my page on Samsung Internet and Opera browsers (yet).

Since Google Chrome is the most popular web browser on Android, I would be even satisfied with Chrome-specific solution, but of course, the best would be to get answer that works everywhere.

EDIT

Following the comment from jib, I've tried also using focusDistance with

focusDistance: { min: 0.05, ideal: 0.12, max: 0.3 } 

and it didn't help.

I've also tried logging the outputs of getSettings and getCapabilities with the following snippet:

const devices = await navigator.mediaDevices.enumerateDevices(); let videoDevices: Array< MediaDeviceInfo > = []; devices.forEach( ( device: MediaDeviceInfo ) => {     if ( device.kind == 'videoinput' ) {         console.log( "Found video device: " + JSON.stringify( device ) );         videoDevices.push( device );     } });  console.log( '' );  // open every video device and dump its characteristics for ( let i in videoDevices ) {     const device = videoDevices[ i ];     console.log( "Opening video device " + device.deviceId + " (" + device.label + ")" );     const stream = await navigator.mediaDevices.getUserMedia( { video: { deviceId: { exact: device.deviceId } } } );     stream.getVideoTracks().forEach( track => {             const capabilities = track.getCapabilities();             console.log( "Track capabilities: " + JSON.stringify( capabilities ) );             const settings = track.getSettings();             console.log( "Track settings: " + JSON.stringify( settings ) );             console.log( '' );         }     )      stream.getTracks().forEach( track => track.stop() ); } 

The output is the following:

Found video device: {"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","kind":"videoinput","label":"camera2 1, facing front","groupId":"500dd57c6795399100a5ca8bf7f0cc4d7ed8b1bcb0877101d1bef7eb74921868"} Found video device: {"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","kind":"videoinput","label":"camera2 3, facing front","groupId":"245693c8d34be77fe2f15be31b6054a19edb8ea9ed4116d966d2a03695bebebe"} Found video device: {"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","kind":"videoinput","label":"camera2 2, facing back","groupId":"c219595df2c2a430aea7007f64e6ce8fbfa783b038cf53069336361cc07e71af"} Found video device: {"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","kind":"videoinput","label":"camera2 0, facing back","groupId":"96a68b6d6429786317e3b2c4773082604c5ab9a5cdaaf49c481878e40d67e987"}  Opening video device b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978 (camera2 1, facing front) Track capabilities: {"aspectRatio":{"max":3216,"min":0.0004528985507246377},"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","facingMode":["user"],"frameRate":{"max":30,"min":0},"groupId":"500dd57c6795399100a5ca8bf7f0cc4d7ed8b1bcb0877101d1bef7eb74921868","height":{"max":2208,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":3216,"min":1}} Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","facingMode":"user","frameRate":30,"groupId":"500dd57c6795399100a5ca8bf7f0cc4d7ed8b1bcb0877101d1bef7eb74921868","height":480,"resizeMode":"none","width":640}  Opening video device 39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6 (camera2 3, facing front) Track capabilities: {"aspectRatio":{"max":3968,"min":0.0003654970760233918},"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","facingMode":["user"],"frameRate":{"max":30,"min":0},"groupId":"245693c8d34be77fe2f15be31b6054a19edb8ea9ed4116d966d2a03695bebebe","height":{"max":2736,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":3968,"min":1}} Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","facingMode":"user","frameRate":30,"groupId":"245693c8d34be77fe2f15be31b6054a19edb8ea9ed4116d966d2a03695bebebe","height":480,"resizeMode":"none","width":640}  Opening video device 4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c (camera2 2, facing back) Track capabilities: {"aspectRatio":{"max":4608,"min":0.00028935185185185184},"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","facingMode":["environment"],"frameRate":{"max":60,"min":0},"groupId":"c219595df2c2a430aea7007f64e6ce8fbfa783b038cf53069336361cc07e71af","height":{"max":3456,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":4608,"min":1}} Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","facingMode":"environment","frameRate":60,"groupId":"c219595df2c2a430aea7007f64e6ce8fbfa783b038cf53069336361cc07e71af","height":480,"resizeMode":"none","width":640}  Opening video device 86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d (camera2 0, facing back) Track capabilities: {"aspectRatio":{"max":4032,"min":0.00033068783068783067},"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","facingMode":["environment"],"frameRate":{"max":60,"min":0},"groupId":"96a68b6d6429786317e3b2c4773082604c5ab9a5cdaaf49c481878e40d67e987","height":{"max":3024,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":4032,"min":1}} Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","facingMode":"environment","frameRate":60,"groupId":"96a68b6d6429786317e3b2c4773082604c5ab9a5cdaaf49c481878e40d67e987","height":480,"resizeMode":"none","width":640} 

So, still, nothing to distinguish between camera2 0, facing back and camera2 2, facing back, except different maximum available resolution.

I've also tried with Samsung Internet browser and it behaves the same as Google Chrome.

Any other ideas (except iterating over all back-facing camera and selecting the one with least resolution)?

Answer 1

I placed all the cameras by id into array like that

      navigator.mediaDevices.enumerateDevices()         .then(function(devices) {                  for(;devices[i];){         if(devices[i].kind == "videoinput"){             that.aCameras.push(   [devices[i].deviceId , devices[i].label]   )             j++;                         }         i++;         }     });

Than on the event that flip the camera by pressing the button i did this:

      var defaultsOpts = { audio: false, video: true };       defaultsOpts.video = {                deviceId: that.aCameras[that.currentCamera][0]       };       if ( that.aCameras.length-1 != that.currentCamera ){           that.currentCamera++;       }       else{           that.currentCamera = 0;       }             navigator.mediaDevices.getUserMedia(defaultsOpts)             .then(function (stream) {                   vid.srcObject = stream;                   localstream = stream;                   vid.play();                });                   });

like that instead of using user/enviroment, my problem was sort of solved.

hope will help you too.

Regards, Avi.

Answer 2

as this problem is somehow still relevant today, currently the best way of detecting "non-telephoto" / "non-wide-lens" camera is IMHO simply to check torch parameter.

(As this is the literally only parameter, which on some devices different “standard” camera and the others.)

I am doing this:

• open default camera stream( = facingMode: { ideal: 'environment' },, find out, if torch is present
• if not, close this camera stream and iterate for each device; try to detect torch
• if nothing is found, fall back to the first camera; or possibly get better by some combination of other parameters - eg. focusDistance.
• (save selected camera id to eg. cookie, so next time is for this user quicker)