How to combine video and audio through API or JS? [closed]

Question

I am working to design a system which does the following:

1. User uploads a video, JS code finds the length of the video.

2. Performs HTTP calls to an already-existing service to retrieve an audio track of the same length.

3. Synchronize and combine the audio and video (which are the exact same length). The best thing I could think to do is to play them both at the same time (entirely doable with HTML5), but I want to be able to have a user download a combined file, or be able to say... upload it to YouTube through their API.

I have been doing lots of googling, and so far have not found any services or code which might be able to do this.

I am familiar with JavaScript, Ruby on Rails, HTML, CSS, jQuery, and both AngularJS and Backbone.js. If a solution exists in one of those languages, or perhaps something I can access through HTTP, that would be wonderful.

Answer 1

I am currently using FFmpeg.wasm. It is a great tool, but I have not found a way to do this with plain JavaScript because the audio and video have different codecs sometimes. I made a function to merge videos and return a Uint8Array, which can be used in a blob.

<script src='https://unpkg.com/@ffmpeg/[email protected]/dist/ffmpeg.min.js'></script>

async function mergeVideo(video, audio) {
    let { createFFmpeg, fetchFile } = FFmpeg;
    let ffmpeg = createFFmpeg();
    await ffmpeg.load();
    ffmpeg.FS('writeFile', 'video.mp4', await fetchFile(video));
    ffmpeg.FS('writeFile', 'audio.mp4', await fetchFile(audio));
    await ffmpeg.run('-i', 'video.mp4', '-i', 'audio.mp4', '-c', 'copy', 'output.mp4');
    let data = await ffmpeg.FS('readFile', 'output.mp4');
    return new Uint8Array(data.buffer);
};

Answer 2

you could give videoconverter.js a try, I have not tried it before, but I think that's the only way to do it front-end for now... and the smallest minified version also takes 6.1 MB.

Another option is upload the video to your server, merge it using ffmpeg and give user the link to output file, either to download or stream.