Challenging image segmentation: background and objects are similar

Question

I'm new to image segmentation, but I need to do it to get a database for the machine learning classifier.

Essentially I have a video similar to this image:

My job is to identify cows in the foreground, or at least any cow at all. I realize there is an occlusion problem, but for a starter I'd like to correctly segment a lonely cow, like the one with the red rectangle around it (hand-drawn).

In less challenging problems, such as this, I discriminate by adding a threshold for every pixel, that either becomes (0,0,0) for the object or (255,255,255) for the background:

Then I label the pixels with the same values to get classes and obtain the rectangle for large enough 'blobs'.

For the image above this approach will not work as the objects and the background are similar + there are a lot of shadows, side lighting etc, so I'm not sure how to approach it. Any suggestions are welcome.

Answer 1

I realise this is an old thread, but I'd like to suggest an approach for problems like this one.

You can try with a texture based segmentation, as the grassy background has a different texture from the cow.

Take a look at this link where the texture energy features for an image are defined according to Laws' technique.

Here's an implementation of the laws technique in Python. It works by defining 2D kernels used to extract different features in an image, for instance edges, ripples, blobs and combinations thereof. The function below returns 9 images, from which texture features can be extracted.

def laws(array):

    # Define the 1D kernels
    L5 = np.array([1,4,6,4,1]) # level
    E5 = np.array([-1,-2,0,2,1]) # edge 
    S5 = np.array([-1,0,2,0,-1]) # spot
    R5 = np.array([1,-4,6,-4,1]) # ripples

    # Generate 2D kernels
    L5E5 = np.outer(L5,E5)
    E5L5 = np.outer(E5,L5)

    L5R5 = np.outer(L5,R5)
    R5L5 = np.outer(R5,L5)

    E5S5 = np.outer(E5,S5)
    S5E5 = np.outer(S5,E5)

    S5S5 = np.outer(S5,S5)

    R5R5 = np.outer(R5,R5)

    L5S5 = np.outer(L5,S5)
    S5L5 = np.outer(S5,L5)

    E5E5 = np.outer(E5,E5)

    E5R5 = np.outer(E5,R5)
    R5E5 = np.outer(R5,E5)

    S5R5 = np.outer(S5,R5)
    R5S5 = np.outer(R5,S5)


    return (0.5*(correlate(array, L5E5) + correlate(array, E5L5)), \
            0.5*(correlate(array, L5R5) + correlate(array, R5L5)), \
            0.5*(correlate(array, E5S5) + correlate(array, S5E5)), \
            correlate(array, S5S5), \
            correlate(array, R5R5), \
            0.5*(correlate(array, L5S5) + correlate(array, S5L5)), \
            correlate(array, E5E5), \
            0.5*(correlate(array, E5R5) + correlate(array, R5E5)), \
            0.5*(correlate(array, R5S5) + correlate(array, S5R5)))