Properly using chrome.tabCapture in a manifest v3 extension

Question

Edit:

As the end of the year and the end of Manifest V2 is approaching I did a bit more research on this and found the following workarounds:

• The example here that uses the desktopCapture API: https://github.com/GoogleChrome/chrome-extensions-samples/issues/627

  The problem with this approach is that it requires the user to select a capture source via some UI which can be disruptive. The --auto-select-desktop-capture-source command line switch can apparently be used to bypass this but I haven't been able to use it with success.

• The example extension here that works around tabCapture not working in service workers by creating its own inactive tab from which to access the tabCapture API and record the currently active tab:

  https://github.com/zhw2590582/chrome-audio-capture

  So far this seems to be the best solution I've found so far in terms of UX. The background page provided in Manifest V2 is essentially replaced with a phantom tab.

The roundaboutedness of the second solution also seems to suggest that the tabCapture API is essentially not intended for use in Manifest V3, or else there would have been a more straightforward way to use it. I am disappointed that Manifest V3 is being enforced while essentially leaving behind Manifest V2 features such as this one.

Original Post:

I'm trying to write a manifest v3 Chrome extension that captures tab audio. However as far as I can tell, with manifest v3 there are some changes that make this a bit difficult:

• Background scripts are replaced by service workers.
• Service workers do not have access to the chrome.tabCapture API.

Despite this I managed to get something that nearly works as popup scripts still have access to chrome.tabCapture. However, there is a drawback - the audio of the tab is muted and there doesn't seem to be a way to unmute it. This is what I have so far:

1. Query the service worker current tab from the popup script.

let tabId;

// Fetch tab immediately
chrome.runtime.sendMessage({command: 'query-active-tab'}, (response) => {
    tabId = response.id;
});

This is the service worker, which response with the current tab ID.

chrome.runtime.onMessage.addListener(
    (request, sender, sendResponse) => {
        // Popup asks for current tab
        if (request.command === 'query-active-tab') {
            chrome.tabs.query({active: true}, (tabs) => {
                if (tabs.length > 0) {
                    sendResponse({id: tabs[0].id});
                }
            });

            return true;
        }
        ...

2. Again in the popup script, from a keyboard shortcut command, use chrome.tabCapture.getMediaStreamId to get a media stream ID to be consumed by the current tab, and send that stream ID back to the service worker.

// On command, get the stream ID and forward it back to the service worker
chrome.commands.onCommand.addListener((command) => {
    chrome.tabCapture.getMediaStreamId({consumerTabId: tabId}, (streamId) => {
        chrome.runtime.sendMessage({
            command: 'tab-media-stream',
            tabId: tabId,
            streamId: streamId
        })
    });
});

4. The service worker forwards that stream ID to the content script.

chrome.runtime.onMessage.addListener(
    (request, sender, sendResponse) => {
        ...
        // Popup sent back media stream ID, forward it to the content script
        if (request.command === 'tab-media-stream') {
            chrome.tabs.sendMessage(request.tabId, {
                command: 'tab-media-stream',
                streamId: request.streamId
            });
        }
    }
);

5. The content script uses navigator.mediaDevices.getUserMedia to get the stream.

// Service worker sent us the stream ID, use it to get the stream
chrome.runtime.onMessage.addListener((request, sender, sendResponse) => {
    navigator.mediaDevices.getUserMedia({
        video: false,
        audio: true,
        audio: {
            mandatory: {
                chromeMediaSource: 'tab',
                chromeMediaSourceId: request.streamId
            }
        }
    })
    .then((stream) => {
        // Once we're here, the audio in the tab is muted
        // However, recording the audio works!
        const recorder = new MediaRecorder(stream);
        const chunks = [];
        recorder.ondataavailable = (e) => {
            chunks.push(e.data);
        };
        recorder.onstop = (e) => saveToFile(new Blob(chunks), "test.wav");
        recorder.start();
        setTimeout(() => recorder.stop(), 5000);
    });
});

Here is the code that implements the above: https://github.com/killergerbah/-test-tab-capture-extension

This actually does produce a MediaStream, but the drawback is that the sound of the tab is muted. I've tried playing the stream through an audio element, but that seems to do nothing.

Is there a way to obtain a stream of the tab audio in a manifest v3 extension without muting the audio in the tab?

I suspect that this approach might be completely wrong as it's so roundabout, but this is the best I could come up with after reading through the docs and various StackOverflow posts. I've also read that the tabCapture API is going to be moved for manifest v3 at some point, so maybe the question doesn't even make sense to ask - however if there is a way to still properly use it I would like to know.

Answer 1

Mute issue is resolved with

const media = await navigator.mediaDevices.getUserMedia(constraints);

// Continue to play the captured audio to the user.
const output = new AudioContext();
const source = output.createMediaStreamSource(media);
source.connect(output.destination);

From official docs