getUserMedia (Selfie) Full Screen on Mobile

Question

I've the following constraints which are working perfectly fine over Chrome in Desktop (simulating mobile resolution)

const constraints = {
    audio: false,
    video: {
        width: screen.width,
        height: screen.height
    }
};

navigator.mediaDevices.getUserMedia(constraints).then(stream => {})

However when actually trying this on iPhone / Safari the camera doesn't respects this at all and gets super small or distorted - removing the width / height from the constraints makes it better ratio but not full screen at all, just centralized.

I've also tried with min / max constraints without lucky.

Is there any way to get this working on iPhones?

Answer 1

I have built a few AR Websites which are mobile first. When you request a resolution the web browser sees if the resolution exists, and if it doesn't it then decides if it should emulate the feed for you. Not all browsers do emulation (even though it is part of the spec). This is why it may work in some browsers and not others. Safari won't emulate the resolution you are asking for with the camera you have picked (I presume the front).

You can read more about this here (different problem, but provides a deeper explaination): Why the difference in native camera resolution -vs- getUserMedia on iPad / iOS?

Solution

The way I tackled this is:

1. Without canvas

• Ask for a 720p feed, fallback to 480p feed if 720 gives an over-constrained error. This will work cross-browser.
• Have a div element which is 100% width and height, fills the screen, and sets overlay to hidden.
• Place the video element connected to the MediaStream inside, make it 100% height of the container. The parent div overlay hidden will in effect crop the sides. There will be no feed distortion.

2. With canvas

• Do not show the video element, use a canvas as the video view. Make the canvas the same size as your screen or the same aspect ratio and use CSS to make it fill the screen (latter is more performant).
• Calculate the top, left, width and height variables to draw the video in the canvas (make sure your calculation centers the video). Make sure you do a cover calculation vs fill. The aim is to crop the parts of the video which do not need to be shown (I.e. like the descriptions of various methods in https://css-tricks.com/almanac/properties/o/object-fit) . Example on how to draw video into a canvas here: http://html5doctor.com/video-canvas-magic/

This will give you the same effect of what you are looking for. Production examples of something similar.

• https://www.maxfactor.com/vmua/
• https://demo.holitionbeauty.com/

P.s. when I get time I can code an example, short on hours this week.

Answer 2

There are a couple of quirks on mobile gUM() you need to know about.

First, if the device is in portrait orientation things work weirdly. You need to swap the width and height. So, let's say you're on a 480x640 device (do those even exist? who cares? it's an example). To get the appropriate size video you need

const constraints = {
   audio: false,
   video: {
    width: screen.height,
    height: screen.width
  }
};

I can't figure out exactly why it's like this. But it is. On iOS and Android devices.

Second, it's hard to get the cameras to deliver exactly the same resolution as the device screen size. I tweak the width and height to make them divisible by eight and I get a decent result.

Third, I figure the sizes I need by putting a <video ...> tag in my little web app with CSS that makes it fill the browser screen, then querying its size with

   const rect = videoElement.getBoundingClientRect()
   const width = rect.width > rect.height ? rect.width : rect.height
   const height = rect.width > rect.height ? rect.height : rect.width

This makes the mobile browser do the work of figuring out what size you actually need, and adapts nicely to the browser's various toolbars.