R-CNN Number Plate Detection

A complete, beginner-friendly implementation of a Region-based Convolutional Neural Network (R-CNN) for detecting vehicle number plates. Built from scratch using OpenCV and TensorFlow/Keras.

This project was developed for a Computer Vision (AI) university course and serves as an excellent learning resource for understanding how Object Detection pipelines work under the hood.

---

How It Works (The R-CNN Pipeline)

This project doesn't use pre-built detection APIs like YOLO or SSD. Instead, it builds the classic R-CNN pipeline step-by-step:

1. Selective Search (Region Proposals)
   The image is passed through OpenCV's Selective Search algorithm. Instead of scanning every single pixel, this groups adjacent pixels by color and texture to generate ~500 "candidate boxes" (Regions of Interest / ROIs) where an object might be.
2. CNN Binary Classification
   Every single ROI generated by Selective Search is resized to 128x128 and fed into a custom-trained Convolutional Neural Network (CNN). The CNN predicts a probability (0 to 1): "Is this box a number plate?"
3. Filtering by Confidence
   We discard any boxes where the CNN is less than 70% confident (> 0.7).
4. Non-Maximum Suppression (NMS)
   Multiple boxes often overlap the same number plate. We calculate the Intersection over Union (IoU) of these boxes and use NMS to keep only the single best bounding box for the plate.

---

Key Feature: Hard Mining (Positive & Negative)

A common issue in Object Detection is the model mistaking complex background objects (like car grills or headlights) for the target object, or missing the object if the bounding box isn't perfect.

To fix this, the data loader (Read_Annotation.py) uses Hard Mining. During training data generation, we use Selective Search to dynamically build a robust dataset:

• Hard Positive Mining: We extract multiple overlapping crops (IoU > 0.7) of the actual plate. This teaches the model to recognize the plate even if the proposed region isn't perfectly centered.
• Hard Negative Mining: We extract actual car parts (grills, headlights) that have an IoU of < 0.1 with the real number plate. By training the model on these "hard" background samples, we significantly reduce False Positives!

---

Project Structure

RCNN_NumberPlate_Detection/
│
├── numberplates/
│   ├── annotations.csv       # Bounding box coordinates for training
│   └── *.png                 # Training/testing images
│
├── Lab17_NMS_IoU.py          # Math logic for IoU and Non-Maximum Suppression
├── Read_Annotation.py        # Loads data + performs Hard Negative Mining
├── main.py                   # CNN training + R-CNN detection pipeline execution
└── requirements.txt          # Required Python libraries

---

Model Architecture

The custom CNN acts as a feature extractor and binary classifier:

Layer	Type	Details
Input	—	128 × 128 × 3 (RGB)
Block 1	Conv2D + MaxPool	32 filters, 3×3, ReLU
Block 2	Conv2D + MaxPool	64 filters, 3×3, ReLU
Block 3	Conv2D + MaxPool	128 filters, 3×3, ReLU
Block 4	Conv2D + MaxPool	256 filters, 3×3, ReLU
Flatten	—	—
FC 1	Dense	128 units, ReLU
Dropout	Dropout	0.5 (Prevents overfitting)
FC 2	Dense	64 units, ReLU
Output	Dense	1 unit, Sigmoid (Outputs 0 or 1)

• Loss Function: Binary Crossentropy
• Optimizer: Adam (learning rate = 0.001)

---

Installation & Setup

# Install dependencies (TensorFlow, OpenCV, Pandas, Matplotlib)
pip install -r requirements.txt

---

How to Run

Just run the main script! It handles both training and testing automatically:

python main.py

What the script does:

1. Parses annotations.csv and loads the images.
2. Generates positive (plate) and negative (background) training patches.
3. Normalizes the data and trains the CNN for 20 Epochs.
4. Saves the trained model as Base_model_keras.keras.
5. Loads a test image (Cars0.png) and runs the complete R-CNN detection pipeline on it.
6. Pops up a window showing the final detected number plate with a red bounding box!

Demo

Detection Result

---

Results Summary

Metric	Value
Training Epochs	20
Input Size	128 × 128
Target Accuracy	> 80%
Confidence Threshold	0.8
NMS Threshold	0.5
Selective Search ROIs evaluated	~500 per image

---

Author

Saqib Shah
BSCS (AI) — Semester 6
Pir Mehr Ali Shah Arid Agriculture University Rawalpindi