Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Extracting audio from a video file

Edit: This post is no duplicate of mine. I am trying to extract the audio data as binary, got no problems with playing the audio file as separate as I mentioned before.

I am trying to extract audio from a video file on client-side by using Web Audio Api.

var audioContext = new(window.AudioContext || window.webkitAudioContext)();
fileData = new Blob([input.files[0]]);
var videoFileAsBuffer = new Promise(getBuffer);
videoFileAsBuffer.then(function (data) {
    audioContext.decodeAudioData(data).then(function (decodedAudioData) {
        mySoundBuffer = decodedAudioData;
        soundSource = audioContext.createBufferSource();
        soundSource.buffer = mySoundBuffer;
        // soundSource.connect(audioContext.destination);
        // soundSource.start();
    });

When I uncomment the two lines at the end, I am hearing the sound of the uploaded video file. Though, when I create a link to download the file with the help of getChannelData method, it's almost the same size as video file.

I was expecting decodedAudioData to have only audio binary data, and send that to my webservice, which is the only one I need. However, that didn't work out as I expected. Anyone knows a way to extract audio of a video file on client-side? Thanks in advance.

Here is the getBuffer method in case anyone wants to know:

function getBuffer(resolve) {
    var reader = new FileReader();
    reader.onload = function () {
        var arrayBuffer = reader.result;
        resolve(arrayBuffer);
    }
    reader.readAsArrayBuffer(fileData);
}

Edit: I was able to decode the video file and get audiobuffer by using OfflineAudioContext inside decodeAudioData function.

var offlineAudioContext = new OfflineAudioContext(2, 44100 * 100, 44100);
var soundSource = offlineAudioContext.createBufferSource();
...
soundSource.connect(offlineAudioContext.destination);
soundSource.start();
offlineAudioContext.startRendering().then(function (renderedBuffer) {
    console.log(renderedBuffer); // outputs audiobuffer
    var song = audioContext.createBufferSource();
    song.buffer = renderedBuffer;
    song.connect(audioContext.destination);
    song.start();
}).catch(function (err) {
    console.log('Rendering failed: ' + err);
});

renderedBuffer is an audiobuffer, had no problem outputting the data, tested with Audacity's import raw data option. But the problem is, the new file (filled with renderedBuffer.getChannelData(0)) has higher size than the original video has. Isn't that supposed to have lower size since it only contains audio of the video file?

like image 355
starkm Avatar asked Mar 06 '18 21:03

starkm


People also ask

Can I extract audio from a video file?

If you are wondering how to take audio from a video, use VLC to extract audio from all forms of video files. Open VLC, click Media on the top right of the window and choose Convert/Save from the drop-down menu.


1 Answers

Ok, I actually had the answer already. Raw audio data is huge, that's why it's even greater than the video itself in size.

var offlineAudioContext = new OfflineAudioContext(numberOfChannels, sampleRate * duration, sampleRate);
var soundSource = offlineAudioContext.createBufferSource();
...
reader.readAsArrayBuffer(blob); // video file
reader.onload = function () {
  var videoFileAsBuffer = reader.result; // arraybuffer
  audioContext.decodeAudioData(videoFileAsBuffer).then(function (decodedAudioData) {
    myBuffer = decodedAudioData;
    soundSource.buffer = myBuffer;
    soundSource.connect(offlineAudioContext.destination);
    soundSource.start();

    offlineAudioContext.startRendering().then(function (renderedBuffer) {
      console.log(renderedBuffer); // outputs audiobuffer
    }).catch(function (err) {
      console.log('Rendering failed: ' + err);
    });
  });
};

After that, I was able to convert the audiobuffer (renderedbuffer) into a wav file by using audiobuffer-to-wav library. OfflineAudioContext is just needed to modify the cropped audio.

Edit: Here is the js fiddle example. decodedAudioData method will suffice if you don't want to override the audio data.

like image 162
starkm Avatar answered Sep 19 '22 11:09

starkm