CropResMoE-50: A Region-Aware Mixture-of-Experts Framework for Fine-Grained Vehicle Damage Detection and Semi-Automated Annotation

Abstract

Accurate detection of vehicle damages such as dents, scratches, and cracks is essential for improving the efficiency, consistency, and scalability of insurance claim assessment. Conventional inspection procedures rely heavily on manual evaluation, making them time-consuming, subjective, and costly. To address these limitations, this paper presents a three-stage progression of mixture-of-experts (MoE)–based classification models trained on the CarDD dataset, which contains 4,000 COCO-annotated vehicle damage images. The study begins with a baseline RawMoE model operating on flattened image representations, followed by ResMoE-50, which incorporates ResNet-50 for deep feature extraction. Building upon these foundations, we propose CropResMoE-50, a region-aware hybrid architecture that integrates object-level cropping with scale-specific analysis to enhance spatial localization and classification accuracy. Extensive experiments on the CarDD benchmark, together with comparisons against established baselines including ResNet-50, EfficientNet-B0, and Swin-T, demonstrate thatCropResMoE-50 achieves strong performance with favorable computational efficiency (24.8M parameters, 4.13 GFLOPs, and approximately 0.006 s inference latency). The model attains 89.30% test accuracy and average precision scores of 87.70%, 93.45%, and 98.12% for small, medium, and large objects, respectively. To extend practical applicability, a semi-automated labeling pipeline integrating ChromaDB is introduced to support retrieval-augmented learning and pseudo-labeling under uncertainty. Additional validation on a real-world insurance dataset from a real-world insurance company confirms robust generalization, achieving 83.33% accuracy. Overall, the proposed framework offers a scalable, interpretable, and deployment-ready solution for automated vehicle damage assessment.