How to fix an extensive echoing while recording audio using navigator.mediaDevices.getUserMedia?

Question

So, I am working on a small electron desktop app that captures desktop screen and records video and audio. When I am trying to add audio to the stream it starts echoing really badly and I am not sure why.

I am using:

• Windows 10 PRO 18362.778
• Chrome 81.0.4044.113
• Electron 8.2.3

Here is some code.

I create these constraints when I want to capture and record video only:

const constraints = {
        audio: false,
        video: {
            mandatory: {
                chromeMediaSource: 'desktop',
                chromeMediaSourceId: source.id
            }
        }
    }

Then I pass is to the stream like that:

const stream = await navigator.mediaDevices.getUserMedia(constraints)

It works like a charm. However when I start adding audio it gives me echo:

const constraints = {
        audio: {
            mandatory: {
                chromeMediaSource: 'desktop',
            }
        },
        video: {
            mandatory: {
                chromeMediaSource: 'desktop',
            }
        }
    }

Also, I can't just set the audio to true. It then gives me this error:

Uncaught (in promise) DOMException: Error starting screen capture

An interesting fact. When I go to Mozilla documentation page on audio constraints and use the demo button it gives me echo too. I tried doing it on Edge and the result was better, but still had echo. So can it be the audio codec?

Here it says that echoCancellation constraint is supported and on by default starting Chrome version 62.

Here is the branch on the Github where I tried to find solution, but failed.

Here is my git repo if you want to look at it more closely.

PS: this is my first post here. Let me know if I did something wrong here and can improve the post. Thank you!

Answer 1

The problem comes from attempting to initiate a single stream of your microphone audio and computer screen video at the same time. To fix this issue, you will need to create an audio stream first, then separately create a video stream that captures your computer screen, and finally, combine the streams into one.

// create audio and video constraints
const constraintsVideo = {
    audio: false,
    video: {
        mandatory: {
            chromeMediaSource: 'desktop',
            chromeMediaSourceId: source.id
        }
    }
}
const constraintsAudio = {audio: true}

// create audio and video streams separately
const audioStream = await navigator.mediaDevices.getUserMedia(constraintsAudio)
const videoStream = await navigator.mediaDevices.getUserMedia(constraintsVideo)

// combine the streams 
const combinedStream = new MediaStream([...videoStream.getVideoTracks(), ...audioStream.getAudioTracks()])

After you combine the streams, you can use combinedStream as if it originated as a single stream.

Answer 2

I think the easy fix here would be to add a muted element to the playback on your page.

 // Preview the source in a video element
    videoElement.srcObject = stream
    videoElement.muted = true
    videoElement.play()

That will work in all browsers and you'll still record the audio.