Extract cow number from image

Question

every now and then my mom has to shift through these type of photos to extract the number from the image and rename it to the number.

I'm trying to use OpenCV, Python, Tesseract to get the process done. I'm really lost trying to extract the portion of the image with the numbers. How could I do this? Any suggestions i'm really new at OpenCV.

I tried to extract the white rectangular board using thresholds and contours, but no avail because the RGB I choose for thresh doesn't always work and I don't know how to choose the contour.

EDIT:

Looking at this paper http://yoni.wexlers.org/papers/2010TextDetection.pdf . Looks prominisn

Answer 1

I have been having another look at this, and had a couple of inspirations along the way....

1. Tesseract can accept custom dictionaries, and if you dig a little more, it appears that from v3.0, it accepts the command-line parameter digits to make it recognise digits only - seems a useful idea for your needs.

2. It may not be necessary to find the boards with the digits on - it may be easier to run Tesseract multiple times with various slices of the image and let it have a try itself as that is what it is supposed to do.

So, I decided to preprocess the image by changing everything that is within 25% of black to pure black, and everything else to pure white. That gives pre-processed images like this:

Next, I generate a series of images and pass them, one at a time to Tesseract. I decided to assume that the digits are probably between 40% to 10% of the image height, so I made a loop over strips 40, 30, 20 and 10% of the image height. I then slide the strip down the image from top to bottom in 20 steps passing each strip to Tesseract, till the strip is essentially across the bottom of the image.

Here are the 40% strips - each frame of the animation is passed to Tesseract:

Here are the 20% strips - each frame of the animation is passed to Tesseract:

Having got the strips, I resize them nicely for Tesseract's sweet spot and clean them up from noise etc. Then, I pass them into Tesseract and assess the quality of the recognition, somewhat crudely, by counting the number of digits it found. Finally, I sort the output by number of digits - presumably more digits is maybe better...

There are some rough edges and bits that you could dink around with, but it is a start!

#!/bin/bash
image=${1-c1.jpg}

# Make everything that is nearly black go fully black, everything else goes white. Median for noise
# convert -delay 500 c1.jpg c2.jpg c3.jpg -normalize -fuzz 25% -fill black -opaque black -fuzz 0 -fill white +opaque black -median 9 out.gif
   convert "${image}" -normalize \
       -fuzz 25% -fill black -opaque black \
       -fuzz 0   -fill white +opaque black \
       -median 9 tmp_$$.png 

# Get height of image - h
h=$(identify -format "%h" "${image}")

# Generate strips that are 40%, 30%, 20% and 10% of image height
for pc in 40 30 20 10; do
   # Calculate height of this strip in pixels - sh
   ((sh=(h*pc)/100))
   # Calculate offset from top of picture to top of bottom strip - omax
   ((omax=h-sh))
   # Calculate step size, there will be 20 steps
   ((step=omax/20))

   # Cut strips sh pixels high from the picture starting at top and working down in 20 steps
   for (( off=0;off<$omax;off+=$step)) do
      t=$(printf "%05d" $off)
      # Extract strip and resize to 80 pixels tall for tesseract
      convert tmp_$$.png -crop x${sh}+0+${off}      \
          -resize x80 -median 3 -median 3 -median 3 \
          -threshold 90% +repage slice_${pc}_${t}.png

      # Run slice through tesseract, seeking only digits
      tesseract slice_${pc}_${t}.png temp digits quiet

      # Now try and assess quality of output :-) ... by counting number of digits
      digits=$(tr -cd "[0-9]" < temp.txt)
      ndigits=${#digits}
      [ $ndigits -gt 0 ] && [ $ndigits -lt 6 ] && echo $ndigits:$digits
   done
done | sort -n

Output for Cow 618 (first number is the number of digits found)

2:11
2:11
3:573
5:33613    <--- not bad

Output for Cow 2755 (first number is the number of digits found)

2:51
3:071
3:191
3:517
4:2155   <--- pretty close
4:2755   <--- nailed that puppy :-)
4:2755   <--- nailed that puppy :-)
4:5212
5:12755  <--- pretty close

Output for Cow 3174 (first number is the number of digits found)

3:554
3:734
5:12732
5:31741  <--- pretty close

Cool question - thank you!