Framework for benchmarking my tracking algorithm

Question

I am currently developing an application based on OpenCV/C++ to track small animals: Here is a example of the kind of video it should process. The program simply output the x,y position of the animal and the time for each area. This is graphical representation of the result onto the original.

My question is a bit awkward --- and maybe off topic --- in that I am not asking how to improve my program, but how to assess it. I am aware of the existence Bonn Benchmark on Tracking dataset, but it is not appropriate to my case.

The program is meant to process very long video, therefore, I cannot realistically ask independent humans to assess the position of the animals and compare human vs. program. I have also considered using robots or putting transponders on bigger animals in order to have the precise positions, but I do not really have the resources.

I came out with the idea of using a program to generate videos of blobs moving in a 2d environment. My question is simple:

Are you aware of any programmable high-level framework that I could use to graphically simulate the motion of an object moving stochastically in a parametrisable background ?

My dream would be to have a command line tool that would work like this:

$ program [BACKGROUND_OPTIONS] [OBJECT_OPTIONS] -V VIDEO_OUTPUT -P POSITIONS_OUTPUT

The background texture could be manipulated as well as the shape, colour and motion pattern of the object moving.

I know that I could probably "easily" make it myself (and I will if I cannot find anything), but I would prefer if the program assessed and the reference were as much independent as they can (for instance not both made by the same person).

Answer 1

One thing I've seen several motion detection/tracking projects do is create test videos with some 3D rendering software such as Blender. It doesn't have the simple interface of your dream test creator, but it's a good testing tool for lots of reasons:

• You can set up whatever scenario you want (varying perspective, number of objects, test length, motion paths, etc.)

• You completely control lighting parameters, shapes, sizes, etc.

• You can design simple tests to verify basic functionality (solid color background with solid colored moving spheres makes a good starting point), then branch into more complex scenarios (other static objects, objects occluding other objects, background images, and so on).