Improve car plate recognition

Question

I am working on school project which base on car plate recognition. I am testing it on simple movie: one car, static camera etc.

This how it looks like:

My first step was finding only car on this frame (I think it will be helpful for more "difficult" video):

Then I search car plate. Here is my code:

std::vector<cv::Rect> boundRect;
    cv::Mat img_gray, img_sobel, img_threshold, element;
    cvtColor(detectedMats[i], img_gray, CV_BGR2GRAY);
    cv::Sobel(img_gray, img_sobel, CV_8U, 1, 0, 3, 1, 0, cv::BORDER_DEFAULT);
    cv::threshold(img_sobel, img_threshold, 0, 255, CV_THRESH_OTSU + CV_THRESH_BINARY);
    element = getStructuringElement(cv::MORPH_RECT, cv::Size(30, 30));
    //element = getStructuringElement(cv::MORPH_RECT, cv::Size(17, 3) ); 
    cv::morphologyEx(img_threshold, img_threshold, CV_MOP_CLOSE, element);
    std::vector< std::vector< cv::Point> > LP_contours;
    cv::findContours(img_threshold, LP_contours, 0, 1);
    std::vector<std::vector<cv::Point> > contours_poly(LP_contours.size());
    for (int ii = 0; ii < LP_contours.size(); ii++)
        if (LP_contours[ii].size() > 100 && contourArea(LP_contours[ii]) > 3000 && contourArea(LP_contours[ii]) < 10000) //można się pobawić parametrami
        {
            cv::approxPolyDP(cv::Mat(LP_contours[ii]), contours_poly[ii], 3, true);
            cv::Rect appRect(boundingRect(cv::Mat(contours_poly[ii])));


            if (appRect.width > appRect.height)
                boundRect.push_back(appRect);
        } 

And result you can see at the second picture.

Then tried get good contours of detect plate. I did few steps.

1. Given the drastic changes in brightness, by histogram equalization:

2. Use filter and threshold:

       cv::Mat blur;
       cv::bilateralFilter(equalized, blur, 9, 75, 75);
       cv::imshow("Filter", blur);
   
       /* Threshold to binarize the image */
   
       cv::Mat thres;
       cv::adaptiveThreshold(blur, thres, 255, cv::ADAPTIVE_THRESH_GAUSSIAN_C, cv::THRESH_BINARY, 15, 2); //15, 2
       cv::imshow("Threshold", thres);
   

3. Finally I find contours but they aren't so good. Numbers are a little bit blurred:

       std::vector<std::vector<cv::Point> > contours;
       cv::findContours(thres, contours, cv::RETR_LIST, cv::CHAIN_APPROX_SIMPLE);
       //cv::findContours(thres, contours, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
   
       double min_area = 50;
       double max_area = 2000;
       std::vector<std::vector<cv::Point> > good_contours;
       for (size_t i = 0; i < contours.size(); i++)
       {
           double area = cv::contourArea(contours[i]);
           if (area > min_area && area < max_area)
               good_contours.push_back(contours[i]);
       }
   

   

Maybe you have some idea how to improve result? I tried change some parameters but it still doesn't work so well.

Thanks for help

---------------------------EDIT: tesseract installation------------------------

1. I install vcpkg.

2. Use

     .\vcpkg install tesseract:x64-windows-static
   

   At the end of installation I get some errors:

3. But when I check it seems fine:

4. And after integrate install I get this:

5. Add lib to project:

So it seems fine but when I tried run example the VS doesn't see libraries:

SOLVED:

Change

.\vcpkg install tesseract:x64-windows-static

into

.\vcpkg install tesseract:x64-windows

and it works nice.

Answer 1

Use tesseract OCR to detect the text. After successful installation of tesseract, add tesseract305.lib and leptonica-1.74.4.lib in the additional dependencies. Use the following code ( from tutorial) :

#include "stdafx.h"
#include "winsock2.h"

#include <tesseract/baseapi.h>
#include <leptonica/allheaders.h>

#pragma comment(lib, "ws2_32.lib")

int main()
{
    char *outText;

    tesseract::TessBaseAPI *api = new tesseract::TessBaseAPI();

    // Initialize tesseract-ocr with English, without specifying tessdata path
    if (api->Init(NULL, "eng")) {
        fprintf(stderr, "Could not initialize tesseract.\n");
        exit(1);
    }

    // Open input image with leptonica library
    Pix *image = pixRead("test.tif");
    api->SetImage(image);
    // Get OCR result
    outText = api->GetUTF8Text();
    printf("OCR output:\n%s", outText);

    // Destroy used object and release memory
    api->End();
    delete[] outText;
    pixDestroy(&image);

    return 0;
}