How to capture a video (AND audio) in python, from a camera (or webcam)

Question

i'm looking for a solution, either in linux or in windows, that allows me to

• record video (+audio) from my webcam & microphone, simultaneously.
• save it as a file.AVI (or mpg or whatever)
• display the video on the screen while recording it

Compression is NOT an issue in my case, and i actually prefer to capture RAW and compress it later.

So far i've done it with an ActiveX component in VB which took care of everything, and i'd like to progress with python (the VB solution is unstable, unreliable).

so far i've seen code that captures VIDEO only, or individual frames...

I've looked so far at

• OpenCV - couldn't find audio capture there
• PyGame - no simultaneous audio capture (AFAIK)
• VideoCapture - provide only single frames.
• SimpleCV - no audio
• VLC - binding to VideoLAN program into wxPthon - hopefully it will do (still investigating this option)
• kivy - just heard about it, didn't manage to get it working under windows SO FAR.

The question - is there a video & audio capture library for python?

or - what are the other options if any?

Answer 1

Answer: No. There is no single library/solution in python to do video/audio recording simultaneously. You have to implement both separately and merge the audio and video signal in a smart way to end up with a video/audio file.

I got a solution for the problem you present. My code addresses your three issues:

• Records video + audio from webcam and microphone simultaneously.
• It saves the final video/audio file as .AVI
• Un-commenting lines 76, 77 and 78 will make the video to be displayed to screen while recording.

My solution uses pyaudio for audio recording, opencv for video recording, and ffmpeg for muxing the two signals. To be able to record both simultaneously, I use multithreading. One thread records video, and a second one the audio. I have uploaded my code to github and also have included all the essential parts it here.

https://github.com/JRodrigoF/AVrecordeR

Note: opencv is not able to control the fps at which the webcamera does the recording. It is only able to specify in the encoding of the file the desired final fps, but the webcamera usually behaves differently according to specifications and light conditions (I found). So the fps have to be controlled at the level of the code.

import cv2 import pyaudio import wave import threading import time import subprocess import os  class VideoRecorder():        # Video class based on openCV      def __init__(self):          self.open = True         self.device_index = 0         self.fps = 6               # fps should be the minimum constant rate at which the camera can         self.fourcc = "MJPG"       # capture images (with no decrease in speed over time; testing is required)         self.frameSize = (640,480) # video formats and sizes also depend and vary according to the camera used         self.video_filename = "temp_video.avi"         self.video_cap = cv2.VideoCapture(self.device_index)         self.video_writer = cv2.VideoWriter_fourcc(*self.fourcc)         self.video_out = cv2.VideoWriter(self.video_filename, self.video_writer, self.fps, self.frameSize)         self.frame_counts = 1         self.start_time = time.time()       # Video starts being recorded      def record(self):  #       counter = 1         timer_start = time.time()         timer_current = 0           while(self.open==True):             ret, video_frame = self.video_cap.read()             if (ret==True):                      self.video_out.write(video_frame) #                   print str(counter) + " " + str(self.frame_counts) + " frames written " + str(timer_current)                     self.frame_counts += 1 #                   counter += 1 #                   timer_current = time.time() - timer_start                     time.sleep(0.16) #                   gray = cv2.cvtColor(video_frame, cv2.COLOR_BGR2GRAY) #                   cv2.imshow('video_frame', gray) #                   cv2.waitKey(1)             else:                 break                  # 0.16 delay -> 6 fps                 #        # Finishes the video recording therefore the thread too     def stop(self):          if self.open==True:              self.open=False             self.video_out.release()             self.video_cap.release()             cv2.destroyAllWindows()          else:              pass       # Launches the video recording function using a thread               def start(self):         video_thread = threading.Thread(target=self.record)         video_thread.start()      class AudioRecorder():       # Audio class based on pyAudio and Wave     def __init__(self):          self.open = True         self.rate = 44100         self.frames_per_buffer = 1024         self.channels = 2         self.format = pyaudio.paInt16         self.audio_filename = "temp_audio.wav"         self.audio = pyaudio.PyAudio()         self.stream = self.audio.open(format=self.format,                                       channels=self.channels,                                       rate=self.rate,                                       input=True,                                       frames_per_buffer = self.frames_per_buffer)         self.audio_frames = []       # Audio starts being recorded     def record(self):          self.stream.start_stream()         while(self.open == True):             data = self.stream.read(self.frames_per_buffer)              self.audio_frames.append(data)             if self.open==False:                 break       # Finishes the audio recording therefore the thread too         def stop(self):          if self.open==True:             self.open = False             self.stream.stop_stream()             self.stream.close()             self.audio.terminate()              waveFile = wave.open(self.audio_filename, 'wb')             waveFile.setnchannels(self.channels)             waveFile.setsampwidth(self.audio.get_sample_size(self.format))             waveFile.setframerate(self.rate)             waveFile.writeframes(b''.join(self.audio_frames))             waveFile.close()          pass      # Launches the audio recording function using a thread     def start(self):         audio_thread = threading.Thread(target=self.record)         audio_thread.start()      def start_AVrecording(filename):      global video_thread     global audio_thread      video_thread = VideoRecorder()     audio_thread = AudioRecorder()      audio_thread.start()     video_thread.start()      return filename     def start_video_recording(filename):      global video_thread      video_thread = VideoRecorder()     video_thread.start()      return filename   def start_audio_recording(filename):      global audio_thread      audio_thread = AudioRecorder()     audio_thread.start()      return filename     def stop_AVrecording(filename):      audio_thread.stop()      frame_counts = video_thread.frame_counts     elapsed_time = time.time() - video_thread.start_time     recorded_fps = frame_counts / elapsed_time     print "total frames " + str(frame_counts)     print "elapsed time " + str(elapsed_time)     print "recorded fps " + str(recorded_fps)     video_thread.stop()       # Makes sure the threads have finished     while threading.active_count() > 1:         time.sleep(1)   #    Merging audio and video signal      if abs(recorded_fps - 6) >= 0.01:    # If the fps rate was higher/lower than expected, re-encode it to the expected          print "Re-encoding"         cmd = "ffmpeg -r " + str(recorded_fps) + " -i temp_video.avi -pix_fmt yuv420p -r 6 temp_video2.avi"         subprocess.call(cmd, shell=True)          print "Muxing"         cmd = "ffmpeg -ac 2 -channel_layout stereo -i temp_audio.wav -i temp_video2.avi -pix_fmt yuv420p " + filename + ".avi"         subprocess.call(cmd, shell=True)      else:          print "Normal recording\nMuxing"         cmd = "ffmpeg -ac 2 -channel_layout stereo -i temp_audio.wav -i temp_video.avi -pix_fmt yuv420p " + filename + ".avi"         subprocess.call(cmd, shell=True)          print ".."     # Required and wanted processing of final files def file_manager(filename):      local_path = os.getcwd()      if os.path.exists(str(local_path) + "/temp_audio.wav"):         os.remove(str(local_path) + "/temp_audio.wav")      if os.path.exists(str(local_path) + "/temp_video.avi"):         os.remove(str(local_path) + "/temp_video.avi")      if os.path.exists(str(local_path) + "/temp_video2.avi"):         os.remove(str(local_path) + "/temp_video2.avi")      if os.path.exists(str(local_path) + "/" + filename + ".avi"):         os.remove(str(local_path) + "/" + filename + ".avi")