License plate recognition using YOLO and OCR

• yolo v8n - https://platform.ultralytics.com/ultralytics/yolov8/yolov8n
• license plate dataset - https://universe.roboflow.com/roboflow-universe-projects/license-plate-recognition-rxg4e

we would be using yolo v8n from ultralytics for this problem statement and since it is trained on COCO dataset, it does not contain license data as part of its training so we need to finetune it with a license plate dataset

steps:

1. yolo pretrained model
2. fine-tune on license plate data
3. use ocr with some specific constraints

• Finetuning

  • get annotated datatset of license plates
  • finetune a pretrained YOLO model
  • save the model weights
  • use the saved model

• Structure of the Dataset

  • train
    • images - images with car number plates
    • labels - txt file
  • valid
    • images
    • labels
  • test
    • images
    • labels
  • data.yaml

• 0 - class id since there is only 1 class number plate
• 7057 training images and 2048 validation images
• model - 130 layers with 3.01M params

• What is OCR?

  • OCR can be done through traditional and ML based methods
  • Traditional OCR ex- Tesseract OCR
  • DL based OCR - https://github.com/jaidedai/easyocr

  

  • inference

Issues:

• YOLO specific issue - flickering effect due to occlusion, since the position of the car changes in the video, the number plate maybe sometimes visible and sometimes not visible! (need a very well trained YOLO model)
• OCR specific issue - the OCR misreads some numbers or alphabets in the number plates when the image is pixelated, to solve this we use the following techniques:
  • note the number plate readings for 20-30 frames and take the one that has the maximum frequency using a deque.
  • once we get the bounding box of the license plate, increase brightness before passing to OCR.

Links:

202609051640