Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

audio Blob not working in IOS / Safari

I am recording audio, sending it as a blob to a nodejs server. The nodejs server then sends it to all connected users that are not currently recording.

Sending the blob:

mediaRecorder.onstop = function(e) {
  var blob = new Blob(this.chunks, { 'type' : 'audio/ogg; codecs=opus' });
  socket.emit('radio', blob);
};

Server receiving the blob:

socket.on('radio', function(blob) {
  socket.broadcast.emit('voice', blob);
});

Listener receiving the blob:

socket.on('voice', function(arrayBuffer) {
  var blob = new Blob([arrayBuffer], { 'type' : 'audio/ogg; codecs=opus' });
  var audio = document.getElementById('audio');
  audio.src = window.URL.createObjectURL(blob);
  audio.play();
});

This works in all browsers/devices except safari and any ios device. Taking it further with safari's inspector I found this:

1st blob object returned 2nd blob object returned 3rd blob object returned

Does safari require something else in its headers for blob objects to be interpreted properly? I've researched accepted audio types, tried aac/mp3/ogg without any success. Upon reading further I've heard references to the fact that there is a bug with streaming blob audio/video data in safari and IOS though I'm not too clear any the details.

Guidance in the rite direction would be very helpful!

EDIT: It looks like this line audio.src = window.URL.createObjectURL(blob); in the receiving blob is what is causing the blob errors (image i linked)

EDIT 2: I tried to see if using another format other than blob would work, opting for base64 encoded string. Looks like this works on all devices and browsers except for IOS and Safari. I'm getting the impression it has something to do with how IOS interprets/loads the data...

like image 807
zillaofthegods Avatar asked Apr 25 '17 20:04

zillaofthegods


2 Answers

For me the solution was to insert a source element into the audio element, and use the sourceElement.src attribute to refer to the blob. I didn't even need to attach the audio-element to the DOM. Example below, hope it helps someone.

var audioElement = document.createElement('audio')
var sourceElement = document.createElement('source')

audioElement.appendChild(sourceElement)

sourceElement.src = '<your blob url>'
sourceElement.type = 'audio/mp3' // or whatever

audioElement.load()

audioElement.play()
like image 186
Andreas Goodstein Avatar answered Sep 20 '22 13:09

Andreas Goodstein


I haven't been able to find a solution using an audio element, however the Web Audio Api seems to do the trick: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API

    var audioContext = new (window.AudioContext || window.webkitAudioContext);
    socket.on('voice', function(arrayBuffer) {
      audioContext.decodeAudioData(arrayBuffer, audioData => {
        var source = audioContext.createBufferSource();
        source.buffer = audioData;
        source.connect(audioContext.destination);
        source.start()
    });

You may still have an issue on iOS as any audio/video must be triggered by a user action.

like image 20
scottmizo Avatar answered Sep 19 '22 13:09

scottmizo