How to define the markers for Watershed in OpenCV?

Question

I'm writing for Android with OpenCV. I'm segmenting an image similar to below using marker-controlled watershed, without the user manually marking the image. I'm planning to use the regional maxima as markers.

minMaxLoc() would give me the value, but how can I restrict it to the blobs which is what I'm interested in? Can I utilize the results from findContours() or cvBlob blobs to restrict the ROI and apply maxima to each blob?

Answer 1

First of all: the function minMaxLoc finds only the global minimum and global maximum for a given input, so it is mostly useless for determining regional minima and/or regional maxima. But your idea is right, extracting markers based on regional minima/maxima for performing a Watershed Transform based on markers is totally fine. Let me try to clarify what is the Watershed Transform and how you should correctly use the implementation present in OpenCV.

Some decent amount of papers that deal with watershed describe it similarly to what follows (I might miss some detail, if you are unsure: ask). Consider the surface of some region you know, it contains valleys and peaks (among other details that are irrelevant for us here). Suppose below this surface all you have is water, colored water. Now, make holes in each valley of your surface and then the water starts to fill all the area. At some point, differently colored waters will meet, and when this happen, you construct a dam such that they don't touch each other. In the end you have a collection of dams, which is the watershed separating all the different colored water.

Now, if you make too many holes in that surface, you end up with too many regions: over-segmentation. If you make too few you get an under-segmentation. So, virtually any paper that suggests using watershed actually presents techniques to avoid these problems for the application the paper is dealing with.

I wrote all this (which is possibly too naïve for anyone that knows what the Watershed Transform is) because it reflects directly on how you should use watershed implementations (which the current accepted answer is doing in a completely wrong manner). Let us start on the OpenCV example now, using the Python bindings.

The image presented in the question is composed of many objects that are mostly too close and in some instances overlapping. The usefulness of watershed here is to separate correctly these objects, not to group them into a single component. So you need at least one marker for each object and good markers for the background. As an example, first binarize the input image by Otsu and perform a morphological opening for removing small objects. The result of this step is shown below in the left image. Now with the binary image consider applying the distance transform to it, result at right.

With the distance transform result, we can consider some threshold such that we consider only the regions most distant to the background (left image below). Doing this, we can obtain a marker for each object by labeling the different regions after the earlier threshold. Now, we can also consider the border of a dilated version of the left image above to compose our marker. The complete marker is shown below at right (some markers are too dark to be seen, but each white region in the left image is represented at the right image).

This marker we have here makes a lot of sense. Each colored water == one marker will start to fill the region, and the watershed transformation will construct dams to impede that the different "colors" merge. If we do the transform, we get the image at left. Considering only the dams by composing them with the original image, we get the result at right.

import sys import cv2 import numpy from scipy.ndimage import label  def segment_on_dt(a, img):     border = cv2.dilate(img, None, iterations=5)     border = border - cv2.erode(border, None)      dt = cv2.distanceTransform(img, 2, 3)     dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8)     _, dt = cv2.threshold(dt, 180, 255, cv2.THRESH_BINARY)     lbl, ncc = label(dt)     lbl = lbl * (255 / (ncc + 1))     # Completing the markers now.      lbl[border == 255] = 255      lbl = lbl.astype(numpy.int32)     cv2.watershed(a, lbl)      lbl[lbl == -1] = 0     lbl = lbl.astype(numpy.uint8)     return 255 - lbl   img = cv2.imread(sys.argv[1])  # Pre-processing. img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)     _, img_bin = cv2.threshold(img_gray, 0, 255,         cv2.THRESH_OTSU) img_bin = cv2.morphologyEx(img_bin, cv2.MORPH_OPEN,         numpy.ones((3, 3), dtype=int))  result = segment_on_dt(img, img_bin) cv2.imwrite(sys.argv[2], result)  result[result != 255] = 0 result = cv2.dilate(result, None) img[result == 255] = (0, 0, 255) cv2.imwrite(sys.argv[3], img) 

Answer 2

I would like to explain a simple code on how to use watershed here. I am using OpenCV-Python, but i hope you won't have any difficulty to understand.

In this code, I will be using watershed as a tool for foreground-background extraction. (This example is the python counterpart of the C++ code in OpenCV cookbook). This is a simple case to understand watershed. Apart from that, you can use watershed to count the number of objects in this image. That will be a slightly advanced version of this code.

1 - First we load our image, convert it to grayscale, and threshold it with a suitable value. I took Otsu's binarization, so it would find the best threshold value.

import cv2 import numpy as np  img = cv2.imread('sofwatershed.jpg') gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) ret,thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU) 

Below is the result I got:

( even that result is good, because great contrast between foreground and background images)

2 - Now we have to create the marker. Marker is the image with same size as that of original image which is 32SC1 (32 bit signed single channel).

Now there will be some regions in the original image where you are simply sure, that part belong to foreground. Mark such region with 255 in marker image. Now the region where you are sure to be the background are marked with 128. The region you are not sure are marked with 0. That is we are going to do next.

A - Foreground region:- We have already got a threshold image where pills are white color. We erode them a little, so that we are sure remaining region belongs to foreground.

fg = cv2.erode(thresh,None,iterations = 2) 

fg :

B - Background region :- Here we dilate the thresholded image so that background region is reduced. But we are sure remaining black region is 100% background. We set it to 128.

bgt = cv2.dilate(thresh,None,iterations = 3) ret,bg = cv2.threshold(bgt,1,128,1) 

Now we get bg as follows :

C - Now we add both fg and bg :

marker = cv2.add(fg,bg) 

Below is what we get :

Now we can clearly understand from above image, that white region is 100% foreground, gray region is 100% background, and black region we are not sure.

Then we convert it into 32SC1 :

marker32 = np.int32(marker) 

3 - Finally we apply watershed and convert result back into uint8 image:

cv2.watershed(img,marker32) m = cv2.convertScaleAbs(marker32) 

m :

4 - We threshold it properly to get the mask and perform bitwise_and with the input image:

ret,thresh = cv2.threshold(m,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU) res = cv2.bitwise_and(img,img,mask = thresh) 

res :

Hope it helps!!!

ARK