How do I get the ROI coordinates based on my prediction?

Question

I am using SVM to predict my ROI, I trained SVM and now in testing phases, it's giving me output with labels in the form of 1 and 0.

I am trying that if SVM predicts 1 mean image contains eyebrow, now I want that it should be rectangle around the eyebrow because the algorithm is predicting based on the eyebrow. How can I do this? Below is the code I use for prediction.

h_og = cv2.HOGDescriptor()
histogram = h_og.compute(photo)
arr.append(histogram)
arr = np.float32(arr)
result = svm.predict(arr)

Now the result is in the form of number or label. How I can draw a rectangle on that ROI of testing image.

Positive data: image with eyebrows
Negative data: images not containing eyebrow
Testing data: Full face of the person

If I have to use detectMultiScale() with it how I will use it with the above logic.

Code use for training purpose

sam = []
lab = []    
# Get positive samples
for filename in glob.glob('D:\*.png'):
    im = cv2.imread(filename, 1)
    h_og = cv2.HOGDescriptor()
    hist = h_og.compute(im)
    sam.append(hist)
    lab.append(1)

# Get negative samples
for file in glob.glob('D:\\*.png'):
    im = cv2.imread(file, 1)
    im = cv2.resize(img, (240, 160))
    h_og = cv2.HOGDescriptor()
    hist = h_og.compute(im)
    sam.append(hist)
    lab.append(0)

# Convert objects to Numpy Objects
sam = np.float32(sam)
lab = np.array(lab)


# Shuffle Samples
rand = np.random.RandomState(321)
shuffle = rand.permutation(len(sam))
sam = sam[shuffle]
lab = lab[shuffle]    


svm = cv2.ml.SVM_create()
svm.setType(cv2.ml.SVM_C_SVC)
#svm.setKernel(cv2.ml.SVM_RBF)
cv2.ml.SVM_LINEAR
# svm.setDegree(0.0)
svm.setGamma(5.383)
# svm.setCoef0(0.0)
svm.setC(2.67)
# svm.setNu(0.0)
# svm.setP(0.0)
# svm.setClassWeights(None)

svm.train(sam, cv2.ml.ROW_SAMPLE, lab) 
svm.save('file.xml')

More Explanation:

If my image contains eyebrow, my SVM prediction is successfully returning me 1 But after that I want it to display that image the below way, the rectangle should be on the coordinates based on which SVM predicts 1

Above image is just a sample image, I am doing it for eyebrow and after prediction, I want to achieve or trying to achieve the output in the above way.

Answer 1

svm.predict(arr) can only predict a single image.

To get coordinates and ROIs you need something that uses this method with different parts of the image at different scales.

So yes, you need to use detectMultiScale(). It is a method of cv2.HOGDescriptor(), therefore you first need to set h_og.setSVMDetector(array) with the SVM supports vectors and the rho that you have trained. You can get them using svm.getSupportVectors() and svm.getDecisionFunction(0).

After that, with found, w = h_og.detectMultiScale(img) you will have a list of rects (found) containing positive data that you can use to draw your boxes.

---

Try something like this, for example:

hog = cv2.HOGDescriptor() 
svm = cv2.ml.SVM_load('svm.xml')

sv = svm.getSupportVectors()
rho, alpha, svidx = svm.getDecisionFunction(0)
svm_new = np.append(sv, -rho)
hog.setSVMDetector(svm_new)

for file in glob.glob("Testing\\*.jpg"): 
    img = cv2.imread(file, 1) 
    img = cv2.resize(img, (240, 160))
    found, w = hog.detectMultiScale(img)

    for (x, y, w, h) in found:
        cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)

    cv2.imshow("Image", img)
    cv2.waitKey()

---

Alternatively, try following this example. Save the model of the SVM using this:

svm.save("svm.xml")
tree = ET.parse('svm.xml')
root = tree.getroot()

SVs = root.getchildren()[0].getchildren()[-2].getchildren()[0] 
rho = float( root.getchildren()[0].getchildren()[-1].getchildren()[0].getchildren()[1].text )
svmvec = [float(x) for x in re.sub( '\s+', ' ', SVs.text ).strip().split(' ')]
svmvec.append(-rho)
pickle.dump(svmvec, open("svm.pickle", 'w'))

In order to do this, you need to import pickle and ElementTree of XML:

import xml.etree.ElementTree as ET
import pickle

And then load it and use it with this:

svm = pickle.load(open("svm.pickle"))
hog.setSVMDetector( np.array(svm) )

found, w = hog.detectMultiScale(img)