Merging/concatenating video and keeping sound in R using AV package

Question

I am trying to merge/concatenate multiple videos with sound sequentially into one video using only R, (I don't want to work with ffmpeg in the command line as the rest of the project doesn't require it and I would rather not bring it in at this stage).

My code looks like the following:

dir<-"C:/Users/Admin/Documents/r_programs/"

videos<-c(
  paste0(dir,"video_1.mp4"),
  paste0(dir,"video_2.mp4"),
  paste0(dir,"video_3.mp4")
)

#encoding
av_encode_video(
  videos,
  output=paste0(dir,"output.mp4"),
  framerate=30,
  vfilter="null",
  codec="libx264rgb",
  audio=videos,
  verbose=TRUE
)

It almost works, the output file is an mp4 containing the 3 videos sequentially one after the other, but the audio present is only from the first of the 3 video and then it cuts off.

It doesn't really matter what the videos are. I have recreated this issue with the videos I was using or even 3 randomly downloaded 1080p 30fps videos from YouTube.

Any help is appreciated & thank you in advance.

Answer 1

The experienced behavior (only 1 audio source) is exactly how it is designed to do. In the C source code, you can identify that encode_video only takes the first audio entry and ignores the rest. Overall, audio is poorly supported by ropensci/av atm as its primary focus is to turn R plots into videos. Perhaps, you can file a feature request issue on GitHub.

Meanwhile, why not just use base.system function to call FFmpeg from R? This will likely speed up your process significantly assuming the videos have identical format by using concat demuxer + stream-copy (-c copy). The av library does not support this feature as far as I can tell. (If formats differ, you need to use the concat filter which is also explained in the link above.)