NAV Navbar
  • Projects
  • Projects

    ANPR

    Character Recognition for NP

    NumberPlates

    Outside India Number Plates

    indian

    Wild Indian Number Plates

    DSOD

    DSOD Prediction layers

    white

    White NP - Private Vehicle

    Yellow

    Yellow NP - Taxis

    Blue

    Blue NP - Embassies/Foreign Dignitaries

    Black

    Black - Private Vehicles on lease as Taxis

    Red

    Red NP - Test Vehicles

    Red_Emb

    Red NP with Emblem - President of India/ Governers

    Our first community project is ANPR (Automatic Number Plate Recognition) for Indian number plates. As the group learns further, the contributors would be assigned Project roles and responsibilities.

    Currently, the group recommends to build:

    projects first, before initiating several more targets.

    ANPR

    Right now, we are in the process to collect and annotate number plate images. Once this process is near completion we will initiate the ANPR project.

    Project Expectations

    Current ANPR solutions are not robust in real-world situations. We aim to create a:

    Current state of art runs at 78.33% at 35 FPS.

    General Architecture

    Our architecture is heavily inspired by a very recent paper (1 March 2018) by Rayson, Severo, Zanlorensi el al and we intend to beat it in performance and accuracy.

    ANPR has three stages:

    1. Number Plate Detection
    2. Character Segmentation
    3. Character Recognition

    Indian number plates are "hellish" as they follow no known pattern. There is some structure to number plates (NPs) of other countries, but in India, it is considered sort of a visiting card, where everyone want their own designs. This is the core reason why we need a strong dataset for Indian ANPR problem.

    We also want to design a constraint-free ANPR solution, where we are not constrained on specific cameras or viewing angles, or specific backgrounds, good lighting conditions, fixed search regions or types of vehicles.

    World's largest publicly available database contains just 4500 images. We intend to create a robust and wild database of up to 100,000 images, creating sort of a world record.

    Our main contributions at the end of the project shall be:

    Vehicle and NP Detection

    We'll train two CNNs in this stage: one for vehicle detection in the input image and other for NP detection in the detected vehicle.

    For speeding up the development time, we will use the convolution weights pretrained on DSOD, available at this link.

    Since our Vehicle Database might not have number plates visible necessary (as our database contains images of vehicles from all angles), we will first train on Vehicle Recognition Database and then on Number Plate Recognition Database.

    Character Segmentation

    Once the NP has been detected, we employ a CNN for character segmentation and recognition.

    Knowing that all Indian NPs have the same format we use 26 classes for letters, 10 classes for digits, 5 classes for backgrounds - White (Private NPs), Yellow (Taxis, Trucks, Commercial), Blue (Foreign Delegates, Embassies), Black (Private Vehicles for Commercial Purposes, Military), Red (President, Governers, Testing), 1 class for Arrow (Defence), 1 class for emblem, and 3 classes for Character Colors (White, Yellow and Black). In total we have (26 + 10 + 5 + 1 + 1 + 3 = 46) 46 classes.

    Private Number Plate :  MH-15-AB-1234 

    Taxi Number Plate :  AP-19-AK-2804 

    Private Vehicle on Lease :  MH.41.UB.8192 

    Embassy/Diplomat's Number PLates : 22 UN 14 

    Test Vehicles' Number Plate : KA 03 MT 9899 

    Character Recognition

    Since many characters might not be perfectly segmented, containing missing parts, and as each character is relatively small, that even one-pixel difference might impair the character's recognition.

    Therefore, we evaluate different padding values (1-3 pixes) in the segmented characters to achieve higher recognition rates.

    Indian NPs has a specific layout. With this knowledge, we know which characters are letters, symbols or numbers. We sort the segmented characters by their horizontal and vertical positions for cars and motorcycles, respectively. Each country would use a different approach to this step.

    Temporal Redundancy

    After performing the NP recognition on single frames, we'll explore the temporal redundancy information through the union of all frames belonging to the same vehicle. Thus, the final recognition is composed of the most frequently predicted character at each NP position (majority vote).

    Experimental Results

    We'll post our results conducted on NVIDIA Titan X GPU. We'll consider as correct only the detections with IoU $\geqslant$ 0.5.

    Current ANPR Systems

    Recognition rates of previous works and commercial systems on SSIG dataset.

    ANPR System $\geqslant$ 6 characters All correct (vehicles)
    Montazzolli and Jung 90.55% 63.18%
    Sighthound 89.05% 73.13%
    Gonc¸alves et al. - 81.80%
    Rayson et al. 99.38% 85.45%
    OpenALPR 92.66% 87.44%
    Rayson et al. (with redundancy) 100% 93.53%