transmitted getUserMedia / MediaRecorder video format larger than requested format. How to tell?

Question

Background:

On Windows 10 I'm using getUserMedia (gUM) and MediaRecorder in Google Chrome (v71) to capture and encode a video stream.

• I'm using the constraints parameter to gUM to tell it I want a video format of 352x288.

• I'm requesting video/webm; codecs="avc1.42E01E" as the MIME type of the encoded stream (that's H.264 boxed in Matroska).

• I'm selecting a cheezy webcam built into a laptop as the video source. It's called "EasyCamera" made by DMAX-AVC. It's tempting to call it CheezyCamera.

The video stream gets generated just fine.

Problem:

The dimensions of the encoded video in the stream are 440x360 rather than my requested 352x288. This information is embedded in the recorded stream, and only visible from the consumer of that data. Use of the various APIs reveals the gUM stream, MediaRecorder, and <video> element metadata all think the dimensions are the ones I asked for.

Of course, webcam, gUM, and MediaRecorder treat the constraints parameter as suggestions, and are free to respond with something different. In this case they respond with 440x360 when I request 352x288. This system functions as designed; that is not my problem.

To clarify, the unexpected 440x360 dimensions are only visible to the consumer of the recorded stream. I hope to find a way to know the producer-side webcam, gUM, and MediaEncoder signal chain is producing a different resolution than I requested.

How does the stream consumer know the stream dimensions? They're in the 'PixelWidth' and 'PixelHeight' Matroska boxes, and they're baked in to the H.264 stream. (Oddly enough, considering this is a software-chosen resolution, it isn't an integral number of 16x16 macroblocks. It still works of course.)

I can't parse the recorded data in the browser because it's stored in opaque blobs.

When I use a different, better, webcam (a Logitech C615) my encoded video stream is the size I requested.

My question:

Is there any way in the webcam / gUM / MediaRecorder / <video> signal chain to find the actual dimensions of the encoded stream in the browser actually recording the stream? That is, can I find the signal chain's response to my requested dimensions without decoding the generated stream?

Answer 1

Use MediaStream.getVideoTracks() method to get the video track (MediaStreamTrack), then use MediaStreamTrack.getSettings() to get MediaTrackSettings object, which contains the height and width of the video of the stream.

So if I request a video of 0 height specified as Constraints, I get a video of height 1 pixels. While streaming we can retrieve both the height I requested and the height I am getting as output.

function handleMediaStream(mystream){
  let videoStreamTrack = mystream.getVideoTracks()[0];
  let constraints = videoStreamTrack.getConstraints();
  console.log(constraints.width, constraints.height);
  // outputs: 640 0
  let settings = videoStreamTrack.getSettings();
  console.log(settings.width, settings.height);
  // outputs: 640 1
}

let videoConstraints = { width: 640, height: 0 }
navigator.mediaDevices.getUserMedia({ video: videoConstraints })
.then(function create_media_recorder(mystream) {
  handleMediaStream(mystream);
});

Answer 2

You might need to use the exact constraint keyword like in this test

const sdPalConstraints = {
  video: {width: {exact: 352}, height: {exact: 288}}
};
// Assuming |video| exists and represents a <video> element.
navigator.mediaDevices.getUserMedia(sdPalConstraints)
                      .then(stream) => {video.srcObject = stream};

This is no guarantee that the WebCam will be streaming in that resolution, but if the produced one differs from the requested, VideoTrackAdapter will be engaged to adapt it for you (and for MediaRecorder).