MATLAB: Segment individual letters from a binary image

Question

I'm working on an optical character recognition project where I am trying to create a program which will recognize alphabetic letters from an image. I'm following the tutorial located on Mathworks(Digit Classification). In their example, their training images are already separated. Unfortunately, I was provided with training images which contain hundreds of letters in a single file.

Here is a sample:

I need an efficient way to segment each individual letter into an image, so I would have a 26Xn array where 26 is each letter in the alphabet and n is n image data variables containing individual letters. It would be extremely tedious to manually segment letters from each training image or attempt to segment letters by a specified length since the separation between letters isn't always equal.

Does anyone know of a MATLAB function or a simple way where I can identify the height and length of every continuous white colored object and store all the individual white objects with their black background in the 26Xn array described above (or at least stored in some type of array so I can later process it into the 26xn array)?

Answer 1

If you want to extract every individual character in your image, you can very easily do that with regionprops. Simply use the BoundingBox attribute to extract the bounding box surrounding each character. After you do this, we can place each character in a cell array for further process. If you want to store this into a 26 x N array, you would need to recognize what each letter was first so that you can choose the slot that the letter is supposed to go in for the first dimension. Because you want to segment out the characters first, we will focus on that. As such, let's load in the image into MATLAB. Note that the original image was in GIF and when I loaded it on my computer... it looked pretty messed up. I've resaved the image into PNG and it's shown below:

Let's read this into MATLAB:

im = imread('http://i.stack.imgur.com/q7cnA.png');

Now, you may notice that there are some discontinuities between some letters. What we can do is perform a morphological opening to close these gaps. However, we aren't going to use this image to extract what the actual characters are. We are only using these to get the bounding boxes for the letters:

se = strel('square', 7);
im_close = imclose(im, se);

Now, you'd call regionprops like this to find all of the bounding boxes in the image (after applying morphology):

s = regionprops(im_close, 'BoundingBox');

What is returned in s is a structure where each element in this structure contains a bounding box that encapsulates an object detected in the image. In our case, this is a single character. The BoundingBox property for each object is a 4 element array that is formatted like so:

[x y w h]

(x,y) are the column and row co-ordinates of the upper left corner of the bounding box and w and h are the width and height of the bounding box. What we will do next is create a 4 column matrix that encapsulates all of these bounding box properties together, where each row denotes a single bounding box:

bb = round(reshape([s.BoundingBox], 4, []).');

It's necessary to round the values because if you want to extract the letters from the image, we have to do this in integer co-ordinates as that is how the image is naturally defined. If you want a good illustration of these bounding boxes, this code below will draw a red box around each character we have detected:

imshow(im);
for idx = 1 : numel(s)
    rectangle('Position', bb(idx,:), 'edgecolor', 'red');
end

This is what we get:

The final job is to extract all of the characters and place them into a cell array. I'm using a cell array because the character sizes are uneven, so putting this into a cell array will accommodate for the different sizes. As such, simply loop over every bounding box we have, then extract the bounding box of pixels to get each character and place it into a cell array. Therefore:

chars = cell(1, numel(s));
for idx = 1 : numel(s)
    chars{idx} = im(bb(idx,2):bb(idx,2)+bb(idx,4)-1, bb(idx,1):bb(idx,1)+bb(idx,3)-1);
end

If you want a character, simply do ch = chars{idx}; where idx is any number from 1 to as many characters as we have. You can also see what this character looks like by doing imshow(ch);

This should hopefully give you enough to get started. Good luck!

Answer 2

You are looking for bwlabel to label each letter in your image.
Another tool you might find useful is regionprops, especially the 'Image' property.

In your comment you state that you struggle with the order at which Matlab is labeling your regions: Matlab's matrices and images are stored in memory in a column major order (that is column after column), thus is "discovers" new components in the binary image from top to bottom and from left to right. In order to explore the image row by row from to to bottom, you might want to consider transposing the image:

[ind map] = imread('http://i.gyazo.com/0ca8d4416a52b8bc3401da0b71a527fd.gif'); %//read indexed image
BW = max( ind2rgb(ind,map), [], 3 ) > .15; %//convert RGB image to binary mask
seg = regionprops( BW.', 'Image' ); %'// transpose input mask
seg = arrayfun( @(x) x.Image.', seg, 'Uni', 0 ); %'// flip back

Now you separate letters are in cells in cell array seg.

Note that by providing regionprops with a binary input, you do not need to explicitly call bwlabel.