Character Recognition for NP
Outside India Number Plates
Wild Indian Number Plates
DSOD Prediction layers
White NP - Private Vehicle
Yellow NP - Taxis
Blue NP - Embassies/Foreign Dignitaries
Black - Private Vehicles on lease as Taxis
Red NP - Test Vehicles
Red NP with Emblem - President of India/ Governers
Our first community project is ANPR (Automatic Number Plate Recognition) for Indian number plates. As the group learns further, the contributors would be assigned Project roles and responsibilities.
Currently, the group recommends to build:
- Facial Recognition
- Image Captioning
projects first, before initiating several more targets.
Right now, we are in the process to collect and annotate number plate images. Once this process is near completion we will initiate the ANPR project.
Current ANPR solutions are not robust in real-world situations. We aim to create a:
- robust ANPR end-to-end solution
- trained on our dataset
- trained on SSIG
- trained on UFPR-ALPR dataset
- capable of running at 45 fps, with
- accuracy target of 80%.
Current state of art runs at 78.33% at 35 FPS.
Our architecture is heavily inspired by a very recent paper (1 March 2018) by Rayson, Severo, Zanlorensi el al and we intend to beat it in performance and accuracy.
ANPR has three stages:
- Number Plate Detection
- Character Segmentation
- Character Recognition
Indian number plates are "hellish" as they follow no known pattern. There is some structure to number plates (NPs) of other countries, but in India, it is considered sort of a visiting card, where everyone want their own designs. This is the core reason why we need a strong dataset for Indian ANPR problem.
We also want to design a constraint-free ANPR solution, where we are not constrained on specific cameras or viewing angles, or specific backgrounds, good lighting conditions, fixed search regions or types of vehicles.
World's largest publicly available database contains just 4500 images. We intend to create a robust and wild database of up to 100,000 images, creating sort of a world record.
Our main contributions at the end of the project shall be:
- a new real-time end-to-end ANPR system using DSOD: Learning Deeply Supervised Object Detectors from Scratch
- A robust character segmentation and recognition in the wild created through strong real-life type image augmentation of synthetic as well as real number plates
- A public dataset for ANPR with 200,000 fully annotated images, as well as 500,000 "real-life" augmentation with features like motion blur, partial occlusion, colored background, extreme angles and dust-noise
- A public database for Vehicle Recognition with 500,000+ images, focused on Indian Vehicles
- A comparative evaluation with state-of-art solutions
Vehicle and NP Detection
We'll train two CNNs in this stage: one for vehicle detection in the input image and other for NP detection in the detected vehicle.
For speeding up the development time, we will use the convolution weights pretrained on DSOD, available at this link.
Since our Vehicle Database might not have number plates visible necessary (as our database contains images of vehicles from all angles), we will first train on Vehicle Recognition Database and then on Number Plate Recognition Database.
Once the NP has been detected, we employ a CNN for character segmentation and recognition.
Knowing that all Indian NPs have the same format we use 26 classes for letters, 10 classes for digits, 5 classes for backgrounds - White (Private NPs), Yellow (Taxis, Trucks, Commercial), Blue (Foreign Delegates, Embassies), Black (Private Vehicles for Commercial Purposes, Military), Red (President, Governers, Testing), 1 class for Arrow (Defence), 1 class for emblem, and 3 classes for Character Colors (White, Yellow and Black). In total we have (26 + 10 + 5 + 1 + 1 + 3 = 46) 46 classes.
Private Number Plate :
Taxi Number Plate : AP-19-AK-2804
Private Vehicle on Lease : MH.41.UB.8192
Embassy/Diplomat's Number PLates : 22 UN 14
Test Vehicles' Number Plate : KA 03 MT 9899
Since many characters might not be perfectly segmented, containing missing parts, and as each character is relatively small, that even one-pixel difference might impair the character's recognition.
Therefore, we evaluate different padding values (1-3 pixes) in the segmented characters to achieve higher recognition rates.
Indian NPs has a specific layout. With this knowledge, we know which characters are letters, symbols or numbers. We sort the segmented characters by their horizontal and vertical positions for cars and motorcycles, respectively. Each country would use a different approach to this step.
After performing the NP recognition on single frames, we'll explore the temporal redundancy information through the union of all frames belonging to the same vehicle. Thus, the final recognition is composed of the most frequently predicted character at each NP position (majority vote).
We'll post our results conducted on NVIDIA Titan X GPU. We'll consider as correct only the detections with IoU $\geqslant$ 0.5.
Current ANPR Systems
Recognition rates of previous works and commercial systems on SSIG dataset.
|ANPR System||$\geqslant$ 6 characters||All correct (vehicles)|
|Montazzolli and Jung||90.55%||63.18%|
|Gonc¸alves et al.||-||81.80%|
|Rayson et al.||99.38%||85.45%|
|Rayson et al. (with redundancy)||100%||93.53%|