How to parse mjpeg http stream from ip camera?

Question

Given below is the code written for getting live stream from an IP Camera.

from cv2 import * from cv2 import cv import urllib import numpy as np k=0 capture=cv.CaptureFromFile("http://IPADDRESS of the camera/axis-cgi/mjpg/video.cgi") namedWindow("Display",1)  while True:     frame=cv.QueryFrame(capture)     if frame is None:         print 'Cam not found'         break     else:         cv.ShowImage("Display", frame)     if k==0x1b:         print 'Esc. Exiting'         break 

On running the code the output that I am getting is:

Cam not found 

Where am I going wrong? Also, why is frame None here? Is there some problem with the conversion?

Answer 1

import cv2 import urllib  import numpy as np  stream = urllib.urlopen('http://localhost:8080/frame.mjpg') bytes = '' while True:     bytes += stream.read(1024)     a = bytes.find('\xff\xd8')     b = bytes.find('\xff\xd9')     if a != -1 and b != -1:         jpg = bytes[a:b+2]         bytes = bytes[b+2:]         i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)         cv2.imshow('i', i)         if cv2.waitKey(1) == 27:             exit(0)    

edit (explanation)

I just saw that you mention that you have c++ code that is working, if that is the case your camera may work in python as well. The code above manually parses the mjpeg stream without relying on opencv, since in some of my projects the url will not be opened by opencv no matter what I did(c++,python).

Mjpeg over http is multipart/x-mixed-replace with boundary frame info and jpeg data is just sent in binary. So you don't really need to care about http protocol headers. All jpeg frames start with marker 0xff 0xd8 and end with 0xff 0xd9. So the code above extracts such frames from the http stream and decodes them one by one. like below.

...(http) 0xff 0xd8      --| [jpeg data]      |--this part is extracted and decoded 0xff 0xd9      --| ...(http) 0xff 0xd8      --| [jpeg data]      |--this part is extracted and decoded 0xff 0xd9      --| ...(http) 

edit 2 (reading from mjpg file)

Regarding your question of saving the file, yes the file can be directly saved and reopened using the same method with very small modification. For example you would do curl http://IPCAM > output.mjpg and then change the line stream=urllib.urlopen('http://localhost:8080/frame.mjpg')so that the code becomes this

import cv2 import urllib  import numpy as np  stream = open('output.mjpg', 'rb') bytes = '' while True:     bytes += stream.read(1024)     a = bytes.find('\xff\xd8')     b = bytes.find('\xff\xd9')     if a != -1 and b != -1:         jpg = bytes[a:b+2]         bytes = bytes[b+2:]         i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)         cv2.imshow('i', i)         if cv2.waitKey(1) == 27:             exit(0)    

Of course you are saving a lot of redundant http headers, which you might want to strip away. Or if you have extra cpu power, maybe just encode to h264 first. But if the camera is adding some meta data to http header frames such as channel, timestamp, etc. Then it may be useful to keep them.

edit 3 (tkinter interfacing)

import cv2 import urllib  import numpy as np import Tkinter from PIL import Image, ImageTk import threading  root = Tkinter.Tk() image_label = Tkinter.Label(root)   image_label.pack()  def cvloop():         stream=open('output.mjpg', 'rb')     bytes = ''     while True:         bytes += stream.read(1024)         a = bytes.find('\xff\xd8')         b = bytes.find('\xff\xd9')         if a != -1 and b != -1:             jpg = bytes[a:b+2]             bytes = bytes[b+2:]             i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)                         tki = ImageTk.PhotoImage(Image.fromarray(cv2.cvtColor(i, cv2.COLOR_BGR2RGB)))             image_label.configure(image=tki)                             image_label._backbuffer_ = tki  #avoid flicker caused by premature gc             cv2.imshow('i', i)         if cv2.waitKey(1) == 27:             exit(0)    thread = threading.Thread(target=cvloop) thread.start() root.mainloop()