how to stream audio chunks using web audio API coming from web-socket?

Question

I am streaming audio data in chunks through web-Socket from server

ws.on('message', function incoming(message) {
    var readStream = fs.createReadStream("angular/data/google.mp3",
        {
            'flags': 'r',
            'highWaterMark': 128 * 1024
        }
    );
    readStream.on('data', function(data) {
        ws.send(data);
    });

    readStream.on('end', function() {
        ws.send('end');
    });

    readStream.on('error', function(err) {
        console.log(err)
    });
});

on the client side

var chunks = [];
var context = new AudioContext();
var soundSource;

var ws = new WebSocket(url);
    ws.binaryType = "arraybuffer";

ws.onmessage = function(message) {
    if (message.data instanceof ArrayBuffer) {
        chunks.push(message.data)
    } else {
        createSoundSource(chunks);
    }
};

function createSoundSource(audioData) {
    soundSource = context.createBufferSource();

    for (var i=0; i < audioData.length;i++) {
        context.decodeAudioData(audioData[i], function(soundBuffer){
            soundSource.buffer = soundBuffer;
            soundSource.connect(context.destination);
            soundSource.start(0);
        });
    }
}

But setting buffer soundSource.buffer = soundBuffer; for the second time causing an error

Uncaught DOMException: Failed to set the 'buffer' property on 'AudioBufferSourceNode': Cannot set buffer after it has been already been set

Any advice or insight into how best to update Web Audio API playback with new audio data would be greatly appreciated.

Answer 1

You cannot reset a buffer on an AudioBufferSourceNode once it's been set. It's like fire-and-forget. Each time you want to play a different buffer, you have to create a new AudioBufferSourceNode to continue playback. Those are very lightweight nodes so don't worry about the performance even when creating tons of them.

To account for this, you can modify your createSoundSource function to simply create an AudioBufferSourceNode for each chunk inside the cycle body, like that:

function createSoundSource(audioData) {
    for (var i=0; i < audioData.length;i++) {
        context.decodeAudioData(audioData[i], function(soundBuffer){
            var soundSource = context.createBufferSource();
            soundSource.buffer = soundBuffer;
            soundSource.connect(context.destination);
            soundSource.start(0);
        });
    }
}

I tried to keep the code style as close to original as possible, but it's 2020, and a function taking advantage of modern features could actually look like this:

async function createSoundSource(audioData) {
  await Promise.all(
    audioData.map(async (chunk) => {
      const soundBuffer = await context.decodeAudioData(chunk);
      const soundSource = context.createBufferSource();
      soundSource.buffer = soundBuffer;
      soundSource.connect(context.destination);
      soundSource.start(0);
    })
  );
}

If you want to stop the old nodes as soon as new data arrives (it looks like you wanted that by resetting the .buffer but I'm not sure), you'll have to store them and call disconnect on all of them when it's time.

Answer 2

Not positive, but I think you have to handle your streaming websocket buffer a bit differently. Maybe websocket-streaming-audio package source code can give you better clues on how to handle your scenario.