Training an OCR Model for Russian License Plate Recognition on PyTorch

TL;DR

The task is recognizing text from Russian license plate images (cropped plate regions).

CRNN architecture (CNN + BiLSTM + CTC) — a classic lightweight OCR approach, ~9M parameters.

Dataset: 37,775 training and 4,891 validation images. Alphabet — 22 characters (digits + Cyrillic).

Mixed Precision (AMP) on NVIDIA RTX 5090 provides ~2x training speedup.

Result: plate accuracy 95%+ after 50–100 epochs with warmup and cosine annealing.

Problem statement

Recognizing text from Russian license plate images — an OCR (Optical Character Recognition) task formulated as sequence-to-sequence prediction: input is a fixed-size plate image, output is a character sequence.

Where it's used: as part of the Video Analytics Platform for real-time license plate recognition in video streams.

Model architecture

Image (1x32x256)
    ↓
[CNN Backbone] — Feature Extraction
    ↓
Tensor (B, 512, 1, 63) → (B, 63, 512)
    ↓
[BiLSTM x2] — Sequence Modeling
    ↓
[Linear 512→23] — Classification
    ↓
[CTC Decoding] — Greedy Decode
    ↓
Plate text

CRNN (Shi et al., 2015) is a classic OCR architecture. Three blocks: CNN extracts features, BiLSTM models the sequence, CTC trains without character-level labels.

CNN backbone (Feature Extraction)

A convolutional neural network extracts feature sequences from the image. The architecture consists of 5 blocks with BatchNorm after each convolutional layer.

Key design decisions:

1) BatchNorm after each layer — stabilizes training and allows higher learning rates.

2) MaxPool(2,1) in blocks 3 and 4 — reduces height only, preserving width (horizontal resolution is critical for character sequences).

3) Final Conv with kernel=2 and no padding — collapses height to 1, preparing data for RNN.

The resulting tensor (B, 512, 1, 63) is reshaped to (B, 63, 512) — 63 time steps, each with 512 features.

BiLSTM (Sequence Modeling)

A bidirectional LSTM processes the feature sequence, capturing context in both directions. This is critical for understanding character boundaries.

nn.LSTM(
    input_size=512,
    hidden_size=256,
    num_layers=2,
    bidirectional=True,
    dropout=0.2,
)

Configuration: input_size=512, hidden_size=256, 2 layers, bidirectional=True, dropout=0.2.

Bidirectional means each time step receives information from both left and right. 2 layers provide sufficient depth, dropout 0.2 adds regularization.

CTC Loss and decoding

CTC (Connectionist Temporal Classification) is the key component for training without character-level annotation. We only know the final plate text, not which pixels correspond to which characters.

A blank token (index 0) is introduced for "no character". The model predicts a distribution over all 23 classes (22 chars + blank) for each of the 63 time steps.

CTC loss sums probabilities of all alignment paths leading to the target text. zero_infinity=True prevents NaN on early epochs.

Decoding (greedy): for each time step, the highest-probability character is selected, then duplicates and blanks are removed.

Data preparation

Dataset: 37,775 training and 4,891 validation images (~88.5% / 11.5%). Labels are extracted from filenames (e.g., A001AA50.png → A001AA50).

Preprocessing: OpenCV (BGR→RGB), grayscale (1 channel), resize to 32×256, normalization to [-1, 1] range.

Augmentations (train only)

To improve robustness to real-world capture conditions:

Training process

Adam optimizer with learning rate 1e-4. Warmup + Cosine Annealing: first 5 epochs LR ramps from 0 to 1e-4 (smooth start), then cosine decay to 1e-12.

Mixed Precision (AMP): forward pass in float16, GradScaler prevents gradient underflow. On RTX 5090 — ~2x speedup and larger batch size.

Gradient Clipping by norm max_norm=5.0 — prevents gradient explosion in LSTM, especially during early training.

Model is saved on validation loss improvement (not train) — prevents overfitting.

Hyperparameters

Metrics and results

Comparison with alternatives

Two approaches were tested during development:

Why PyTorch: TensorFlow dropped Windows GPU support from version 2.15. PyTorch with CUDA 12.8 fully utilizes the RTX 5090 — training is tens of times faster than on CPU.

Tech stack

Common OCR training mistakes

1) Training without augmentations — model fails on real-world conditions (noise, lighting, blur).

2) Using RGB instead of grayscale — plates don't need color info, and 3 channels increase model size.

3) No gradient clipping — LSTM is prone to gradient explosion during early training.

4) Saving model by train loss instead of val loss — leads to overfitting.

5) Starting with full LR without warmup — destructive weight updates early on.

6) Ignoring plate format (variable length) — without a proper collate function, CTC won't train.

Training monitoring

TensorBoard integration for visualization: train/val loss curves, plate accuracy, character accuracy, and learning rate changes.

Launch: tensorboard --logdir runs/v2_ocr

FAQ

Key Takeaways

1) CRNN + CTC is an efficient lightweight architecture for plate OCR (~9M parameters).

2) Mixed Precision + RTX 5090 provide significant training speedup.

3) Warmup + Cosine Annealing ensure stable convergence without oscillations.

4) Augmentations are critical for robustness to noise and lighting in real conditions.

5) PyTorch is currently the only option for GPU training on Windows.