AudioContext stereo output from MediaStreamDestination in Chrome M34

Question

It seems that AudioContext.createMediaStreamDestination() defaults to mono output. This default is being changed, but is there a way to set the number of desired output channels manually?

Or is there any other way to get the stream from WebAudio into WebRTC while preserving the correct amount of channels?

Answer 1

This currently seems pretty hard to do. After some research I found that the documentation says it is possible to set more channels. Despite calling context.createMediaStreamDestination(2) I still receive a mono stream on the other side. According to this discussion it seems you can add multiple streams. When trying this out, both local(streamer) and remote(receiver) seem to show the right streams when calling pc.getLocalStreams() and pc.getRemoteStreams() respectively. So what you could do is use a channel splitter to split the input to two mono outputs which you can stream together.

Sadly, as webRTC and webAudio are still in development, we are not yet able to pipe received streams through the web Audio API (using context.createMediaStreamSource(<RTCStream>), this only works for microphone streams) and then just send it through a channel merger to make it line again. I am still testing wether adding the received streams to two separate Audio elements gives synchronised audio.

Small update

After some testing with adding multiple tracks to a stream and then send that stream, the receiving side's <audio> tag played both both tracks, and it actually added them up, like the track containing the left channels is palyed over both speakers, and so is the right. So I have not yet been able to split them receiver side but have at least been able to send left and right audio over two different channels. So yeah, it is about time that the developers give us the oportunity to pipe received audio through the web audio API.

Another update

It seems that a stream created by a streamDestinationNode automatically is a stereo stream. You can find an example of that here (remember to add some music in the src field).
This behavior is also not related to the way we play received streams right now - creating an objectURL from it and assign it to the <audio> element's src.
This would mean that it is related to the RTC streaming system itself. After testing the above code with an RTCPeerConnection, it seems that it just mixes the channels.

Using codecs

So I tought about using another codec, Opus. I found some code to change it to that codec, and searched for ways to set the codec to use stereo. I found this and this, although I have no idea how to use it. Besides that changing to Opus doesn't automatically change it to stereo (it could have been if the codec was set to automatically do that).

I have found a draft of creating sdp's for Opus. I am investigating.

EDIT: FOUND IT

I found it in the third version of the rtp opus draft. It is in an example, and it is pretty simple. Add stereo=1; sprop-stereo=1 to the a=fmtp line, and you are done. The document also contains a spec for these properties:

stereo: specifies whether the decoder prefers receiving stereo or mono signals. Possible values are 1 and 0 where 1 specifies that stereo signals are preferred and 0 specifies that only mono signals are preferred. Independent of the stereo parameter every receiver MUST be able to receive and decode stereo signals but sending stereo signals to a receiver that signaled a preference for mono signals may result in higher than necessary network utilisation and encoding complexity. If no value is specified, mono is assumed (stereo=0).

sprop-stereo: specifies whether the sender is likely to produce stereo audio. Possible values are 1 and 0 where 1 specifies that stereo signals are likely to be sent, and 0 speficies that the sender will likely only send mono. This is not a guarantee that the sender will never send stereo audio (e.g. it could send a pre- recorded prompt that uses stereo), but it indicates to the receiver that the received signal can be safely downmixed to mono. This parameter is useful to avoid wasting receiver resources by operating the audio processing pipeline (e.g. echo cancellation) in stereo when not necessary. If no value is specified, mono is assumed (sprop-stereo=0).

I hope this helps you. As I expect this is for better quality audio, I suggest you to read through that doc to find out more about quality audio streaming.

Downside

After testing this out with friends, it seems the encoding side (the side that sends the audio) generates noise in chrome 34. I have been using chrome 36 (dev and canary), and this error doesn't occur there. (when streaming from chrome 34 to 36 you get the noise, when streaming from 36 to 34 you don't, so it is a problem on the encoding side).

(Why I write such an elaborated answer? I want to know myself too :D).