CSCE 5013 - Homework 3
Due Date - October 20, 2011 at 11:59 PM

1. Problem Statement:

In this assignment, your task is to implement OCR on car images in order to "read" the license plates. To do this, you will need to extract the letters from car images, perform character identification using either template matching or feature matching, and finally perform spatial analysis to put the letters in the correct order.

2. Design:

In src/hw3/letters you will find license plate images with two examples of each letter from A..Z and numbers from 0..9. These images will names of the form: A-1.jpg, A-2.jpg, ... 9-1.jpg, 9-2.jpg. In src/hw3/cars you will find several car images with license plates roughly centered within the image. Your task is to read in the letter images save everything you need for template matching or feature matching, and then perform OCR on each of the car images to "read" the license plates. Your program should output something like "car36.jpg has plate ABC 123".

In this project, we will be making use of the character identification functions from your previous assignment to perform OCR. Students have the option of using their code from the previous assignment or borrowing anything they need from the sample solution in src/hw2/hw2.cpp. This code is described below:

letter extraction: The functions "extract" and "extract_letters" are used to process a car image to and create letter images. The function "extract_letters" reads a directory of car images, and sends each image into the "extract" function. This function performs thresholding and blob_coloring and then scans the blob_colored image to find the min and max (x,y) coordinates of each potential letter. The "Extract" method is then used to grab this rectangular region out of the car image so it can be written to a jpeg file. One thing you will notice about the "extract" function is that will output regions that are NOT valid letters. The other thing you will notice is that the letters are not always found in left-to-right order by the "extract" function. Hence you may need to do some analysis to separate letter and non-letter regions and to spatially order your OCR match results to correctly output "ABC 123".

template matching: The function "read_letters" reads the "truth.txt" file and then reads each of the letter images into a global array called "Image". Each image is interpolated to be the same height, and then they are smoothed and thresholded to get B/W images. The "read_letters" function also stores the names of each image in an array called "Name" and the ascii code for each letter in an array called "Truth". The "compare_images" and "compare_all_images" perform the template matching to compare all letters to each other and output the best match for each input image. The global constants SEARCH and STEP control the image alignment process. We try all (dx,dy) alignments within the range dx=[-SEARCH..SEARCH], dy=[-SEARCH..SEARCH] with a step size of STEP.

feature matching: The function "read_letters" reads images into a global array called "Image" and the function "get_moments" calculates the 1st, 2nd and 3rd central moments for each letter. These are stored in a global array called "Moments". All moment values are scaled to be in the range [0..1] according to the min/max values of each moment. The "compare_moments" and "compare_all_moments" perform feature matching to compare all letters to each other and output the best match for each input image. Feature matching is MUCH faster then template matching with very similar error rates.

3. Implementation:

You can implement this program using either a bottom-up approach or a top-down approach. If you go for a bottom-up approach, start by creating basic methods and classes, and test theses methods using a simple main program that calls each method. When this is working, you can create the main program that uses these methods to solve the problem above.

If you go for a top-down approach, start by creating your main program that reads user input, and calls empty methods to pretend to solve the problem. Then add in the code for these methods one at a time. This way, you will get an idea of how the whole program will work before you dive into the details of implementing each method and class.

Regardless of which technique you choose to use, you should develop your code incrementally adding code, compiling, debugging, a little bit at a time. This way, you always have a program that "does something" even if it is not complete.

When you think you are about 1/2 way through the program, upload a copy of your source code and your program output at that point. Be sure to hand in something that compiles even if it does not do much when it runs.

4. Testing:

Test your program to check that it operates correctly for all of the requirements listed above. Also check for the error handling capabilities of the code. Try your program on 2-3 input documents, and save your testing output in text files for submission on the program due date.

5. Documentation:

When you have completed your C++ program, write a short report (1-2 pages long) describing what the objectives were, what you did, and the status of the program. Does it work properly for all test cases? Are there any known problems? Save this report in a separate text file to be submitted electronically.

6. Project Submission:

In this class, we will be using electronic project submission to make sure that all students hand their programming projects and labs on time, and to perform automatic analysis of all programs that are submitted. When you have completed the tasks above go to the class web site to "submit" your documentation, C++ program, and testing files.

The dates on your electronic submission will be used to verify that you met the due date above. All late projects will receive reduced credit (50% off if less than 24 hours late, no credit if more than 24 hours late), so hand in your best effort on the due date.

You should also PRINT a copy of these files and hand them into your teaching assistant in your next lab. Include a title page which has your name and uaid, and attach your hand written design notes from above.