[Feature] - Add region recognition

[ankandrew]

TODO:

• Extra head for region recognition
• Integrate with ONNX exporter, inference class, etc.

[ankandrew]

Will start re-training soon.

[andreasparas1]

I was wondering about how your current progress on this is going. I have been working on a standalone region classification model for USA states with decent success (97% validation acccuracy however the images in the dataset I have don't reflect real world conditions). I also tried taking the cct_s_relu_v1_global model and adding LayerNormalization -> SequencePooling -> Dense layers starting at the last transformer block. This approach got up to 90% validation accuracy with the model being fully fine tuned on my dataset. However this approach generally failed when trying to keep earlier layers frozen. Do you think its possible to have a separate training step, dedicated only to region recognition while preserving ocr accuracy, or does the model have to be trained for both tasks together?

[ankandrew]

@andreasparas1 I'm getting good numbers when training from scratch for dual objective (plate_text + region_cls), although I'm trying to add more diverse data to reflect US plate variations, which are quite different in color/texture between diff states. Btw, I'm not training for sub-region (i.e. state classification), although totally possible, I don't have enough data for that. The region recognition is at country level currently. I didn't see adding region classification hurting at all the plate text accuracy. If you want to freeze shared backbone and the plate text part of the model, you could just train or fine-tune the region classification part easily. Currently as I'm doing it, you can still train with single objective (only plate_text) and the region is optional.

[andreasparas1]

@andreasparas1 I'm getting good numbers when training from scratch for dual objective (plate_text + region_cls), although I'm trying to add more diverse data to reflect US plate variations, which are quite different in color/texture between diff states. Btw, I'm not training for sub-region (i.e. state classification), although totally possible, I don't have enough data for that. The region recognition is at country level currently. I didn't see adding region classification hurting at all the plate text accuracy. If you want to freeze shared backbone and the plate text part of the model, you could just train or fine-tune the region classification part easily. Currently as I'm doing it, you can still train with single objective (only plate_text) and the region is optional.

What does the model architecture look like for the region classification part currently? I tried keeping the backbone frozen and only train the classifier layers I added but saw severe underfitting.

[ankandrew]

What does the model architecture look like for the region classification part currently? I tried keeping the backbone frozen and only train the classifier layers I added but saw severe underfitting.

Currently looks like (very similar with what you mentioned)

fast-plate-ocr/fast_plate_ocr/train/model/model_builders.py

Lines 81 to 85 in 5b09fd1

	if not plate_cfg.plate_regions:
	raise ValueError("Region head requested, but no regions are defined in the plate config.")
	pooled_tokens = SequencePooling(name="region_seq_pool")(x)
	region_logits = layers.Dense(len(plate_cfg.plate_regions), activation="softmax", name="region")(pooled_tokens)
	outputs = {"plate": plate_logits, "region": region_logits}

I'd wait to fine-tune or re-run that experiment but with an actual model that has been trained on much data like the current cct models but with dual objective (once I complete this ticket). The current model you are trying to fine-tune has only seen plate text data, and freezing that might not be optimal for the currently untrained objective (region cls).

[ankandrew]

@andreasparas1 you can try now V2 CCT models!