MoVi-HER2: A MobileNetV3-ViT Fusion Network for HER2 Status Prediction in Gastroesophageal Adenocarcinoma from Tissue Microarray Images

Abstract

Accurate assessment of HER2 status is critical for therapeutic decision-making in gastroesophageal adenocarcinoma (GEA), yet manual evaluation of HER2 immunohistochemistry (IHC) remains labor-intensive and prone to interobserver variability. This research proposes MobileNetV3-ViT Fusion Network (MoVi-HER2), a deep learning model designed to automatically predict HER2 status and IHC scores from tissue microarray (TMA) images by integrating local and global feature representations. The model fuses MobileNetV3 as a convolutional backbone for fine-grained texture extraction with Vision Transformer (ViT) for contextual spatial reasoning, combined through concatenation and addition strategies followed by dense refinement layers for classification. Performance was evaluated on a public TMA dataset of GEA stained for HER2 expression, using macro F1-score, weighted F1-score, and balanced accuracy. MoVi-HER2 achieved superior performance across all metrics, obtaining a macro F1-score of 0.8531 and balanced accuracy of 0.9794 for HER2 status prediction, and 0.8119 and 0.9370 for IHC score prediction, surpassing MobileNetV3, ViT, Xception, EfficientNetV2, TinyViT, and EfficientFormerV2. Notably, these results were achieved without complex preprocessing or augmentation, indicating strong generalization and stain invariance. Fusion attention visualizations further confirmed that the model focuses on membrane-rich regions consistent with clinical HER2 scoring criteria, demonstrating its potential as an accurate, robust, and interpretable tool for AI-assisted digital pathology in GEA diagnosis.