Multiple moving object detection under changing light conditions.

Question

I am trying to extract several moving objects in a video frame, and extract them as foreground. The data is from video frames.

The current problem is: the light is changing, so there are some shadows, or more brighter parts than the actual background. This leads to false background/foreground extraction by OpenCV MoG background segmentation method.

For this, I haven't get any straightforward method, but have an idea like this: if I can extract the edges of those moving objects in the previous frame, then maybe I could track them using algorithms like SIFT in the next frame, see where they are, and regard them as foreground.

I think in this case, the light change won't affect the result. If I am right on this point, then my question is:

How can I do the edge detection of those moving objects efficiently with OpenCV? If I need to use SIFT algorithm in OpenCV, is it freely available? From the web, I saw it was nonfree, am I right?

And my second question is: does anyone have a better idea for this?

Thank you.

Answer 1

If you want to do human detection/tracking you should look for research papers or projects regarding exactly that. There is a lot of it, you will even find some questions regarding that topic here on SOF:

How can I detect and track people using OpenCV? (maybe outdated)

single person tracking from video sequence

How to do motion tracking of an object using video?

Also, there are several questions addressing feature detectors/descriptors like SIFT and its more recent alternatives (SURF, ORB, FREAK - just to name some implemented in OpenCV):

Are there any fast alternatives to SURF and SIFT for scale-invariant feature extraction?

OpenCV for ANDROID image compare

To put it simple, SIFT is not an algorithm to track moving objects, it is to detect image regions which are somehow unique and robust to several distortions (translation, rotation, scale...). Meaning, the same feature can be later detected in different image conditions. You can indeed use SIFT-like algorithms to identify objects, but maybe for person tracking there are better alternatives. To those image regions you can then apply some tracking algorithm, optical flow, for example - but there are more specific ones for human tracking.

SIFT and SURF are "freely" available with OpenCV but parts of them are patented so people are avoiding to use them so they have no issues in the future because of those patents - this is the reason they where moved to the "nonfree" OpenCV module.

Other problems you will have, besides the light issues you mentioned, are object occlusion and people entering-exiting the scene.