GraphSkinUQ: A Hybrid Transfer Learning Approach for Feature Extraction and Graph Transformer-Based Uncertainty Quantification for Robust Skin Cancer Classification

1).CNN FOR CLASSIFICATION

During CNN training with Keras, the dataset is divided only into a training set and a validation set (20% held out for validation). A separate test split is not created at this stage.

In contrast, during the PyTorch-based feature extraction and graph modeling stage, we explicitly split the full ImageFolder dataset into 80% training, 10% validation, and 10% test subsets before building the k-NN graph and training the GTN.

The CNN consists of multiple convolutional blocks. In each block, the convolution operation is defined as:

2).DEEP FEATURE EXTRACTION WITH PRETRAINED RESNET50

A pretrained ResNet50 model, with the final classification layer removed, is used to extract high-level representations. The images are resized to 224×224 pixels and normalized using ImageNet statistics. If X∈RH×W×C represents an input image, the deep feature vector f∈Rd is obtained by:

After preprocessing, a sample image is passed through ResNet50, producing a 2048-dimensional feature vector. For example, one malignant image yields a feature embedding where higher activations correspond to darker, irregular lesion regions. These embeddings form the basis for subsequent graph construction.

3).GRAPH CONSTRUCTION VIA k-NNS

The extracted feature vectors are used to construct a k-NN graph. Let fi and fj be feature vectors of images i and j, respectively; the Euclidean distance between these vectors is given by:

For each node, connections are established with its k=10 nearest neighbors, resulting in a connectivity matrix that is later converted into an edge list suitable for processing with PyTorch Geometric’s Data class. Training, validation, and test masks are defined on this graph to enable supervised learning.

Using these embeddings, the similarity graph is built. For instance, the feature vector of Image #210 (benign) is connected to its 10 most similar nodes, such as Images #134, #598, and #623, based on Euclidean distances ranging from 0.45 to 0.63. This structure ensures that visually similar lesions share information during graph learning.

4).GTN AND HYPERPARAMETER TUNING

At the core of GraphSkinUQ lies the GTN, which leverages the relational structure within the graph. At each transformer layer, multi-head attention computes new node representations. For an input node feature matrix X∈RN×d and adjacency information given by E, the transformer convolution is formulated as:

A comprehensive grid search over learning rate, hidden channel dimensions, and dropout rates is conducted. Each configuration is trained for 300 epochs using a ReduceLROnPlateau scheduler, and the model achieving the highest validation accuracy is selected for further evaluation.

5).UQ VIA MCD

In order to quantify predictive uncertainty, GraphSkinUQ enables MCD during inference. The best-performing GTN is maintained in training mode to activate dropout. For T stochastic forward passes (with T=50), the predictive probability for each node is averaged:

Figure 2 illustrates each step of our MCD pipeline. We start by extracting CNN features, building the 10-NN graph, and training the GTN with active dropout (steps 1–4). At inference, the “MCD?” decision node triggers T stochastic forward passes (steps 5–7), whose outputs are aggregated (step 8) and converted to a Shannon entropy score (step 9) to yield a robust per-case uncertainty measure. Embedding this diagram immediately after our equations lets readers directly map theory to practice.

Fig. 2. Compact MC dropout uncertainty workflow for GraphSkinUQ.

1).DISCRIMINATION PERFORMANCE

Figure 3 shows the ROC curve for our model, for which we report an AUC of 0.95. This high AUC indicates that GraphSkinUQ ranks true malignant cases above benign ones 95.4% of the time. Even samples whose predicted probabilities lie near the decision boundary tend to be correctly separated from clear negatives, demonstrating that the model’s confidence scores are largely meaningful.

Fig. 3. ROC curve for GraphSkinUQ (AUC = 0.951).

The precision–recall curve in Fig. 4 further elucidates the model’s behavior. At low recall levels (below 0.5), precision remains near 1.0, indicating that the highest-confidence positive predictions are almost always correct. As recall approaches 1.0, precision declines to approximately 0.65, marking the region where more uncertain cases contribute to false positives. The steep drop in precision between recalls of 0.75 and 1.0 defines an “uncertainty zone” in which the model’s decisions become less reliable.

Fig. 4. Precision–Recall curve for GraphSkinUQ.

2).COMPARATIVE ANALYSIS

Figures 3 and 4 present the ROC and Precision–Recall curves for all compared methods. GraphSkinUQ consistently achieves the highest AUC and AP scores, followed by multi-layer perceptron (MLP) and SVM, whereas traditional graph models (graph convolutional network (GCN) and GAT) show lower discriminative ability. These results align with the quantitative metrics in Table IX, confirming that integrating relational graph reasoning with UQ enhances both sensitivity and precision. The improvement is particularly notable in the high-recall region, indicating greater reliability for safety-critical screening tasks.

3).SOURCES OF UNCERTAINTY

While GraphSkinUQ achieves strong overall performance, we include this discussion of epistemic and aleatoric uncertainty to demonstrate how the model explicitly quantifies—and in the case of epistemic uncertainty, mitigates—areas where even high-accuracy systems naturally face ambiguity. In other words, highlighting these two uncertainty sources does not imply a fundamental weakness of GraphSkinUQ but rather underscores its built-in mechanisms for managing them:

• •Epistemic (Model) Uncertainty: Even with MCD at inference, some lesion patterns near the decision boundary remain inherently harder to learn. By sampling multiple dropout realizations, GraphSkinUQ not only measures its own confidence but also reduces overconfident errors in this “uncertain zone.” This capability is a deliberate strength, ensuring the model signals when its knowledge is limited rather than silently misclassifying.
• •Aleatoric (Data) Uncertainty: Variability in image quality or missing metadata can never be fully eliminated by training alone. GraphSkinUQ acknowledges this by reporting higher uncertainty for these cases. Far from indicating failure, this feature empowers clinicians to flag ambiguous examples for further review, turning data noise into a useful triage signal.

In sum, discussing these uncertainty types showcases GraphSkinUQ’s proactive approach: it does not simply deliver point predictions but also provides calibrated confidence estimates that improve safety and trust in real-world clinical use.

4).CALIBRATION ASSESSMENT

Beyond discrimination, well-calibrated probabilities are essential when confidence scores guide clinical decisions. We employ two quantitative metrics:

• •The Brier score, measuring squared deviation between predicted probabilities and outcomes, is 0.13.
• •The ECE, summarizing absolute differences between confidence and accuracy over bins, is 0.06.

These low values confirm that GraphSkinUQ’s confidence estimates align closely with observed frequencies. If further refinement were required, post hoc methods such as temperature scaling or isotonic regression could be applied without retraining the model.

5).QUANTITATIVE SUMMARY

Table IV aggregates key metrics:

Table III. Classification report for the best model (dropout: 0.5, hidden_channels: 16, lr: 0.0005)

Table IV. Overall uncertainty and performance metrics

The low average entropy (0.10) indicates that most predictions are made with high confidence, while a small tail of higher-entropy cases points to genuinely ambiguous examples.

6).ILLUSTRATIVE PREDICTIONS AND ENTROPY DISTRIBUTION

Figure 5 presents six test images alongside their true and predicted labels. Only one benign lesion was misclassified as malignant, illustrating a sensitivity-biased decision rule—often preferable in screening contexts to minimize false negatives.

Fig. 5. Examples of test samples with true vs. predicted labels.

Table V details these cases:

Table V. Sample prediction outcomes

Figure 6 shows the predictive entropy distribution over all test cases. The pronounced right-skew indicates a small subset of high-entropy, low-confidence predictions that clinicians can triage for additional review.

Fig. 6. Distribution of predictive entropy across test predictions.

7).ACCURACY–UNCERTAINTY TRADE-OFF

Finally, we evaluate the impact of cross-validation on both accuracy and uncertainty. Switching from a single train/validation split to 4-fold cross-validation raised nominal accuracy from 87.4% (baseline CNN) to 99.3% under optimal GNN settings but also increased average entropy from 0.10 to 0.16 and slightly lowered test accuracy to 86.1%. This illustrates the common trade-off in practice: higher apparent accuracy can accompany inflated epistemic uncertainty, underscoring the value of UQ for safe clinical deployment.

Together, these analyses demonstrate that GraphSkinUQ not only achieves strong discriminative performance but also provides well-calibrated, interpretable uncertainty estimates, supporting more trustworthy decision-making in skin cancer classification.

1).Experiment 1: Comparison of Pretrained CNN Models for Feature Extraction

This experiment evaluates the effectiveness of different pretrained CNN architectures for extracting features from skin spot images, which are subsequently used by a GTN for image classification with uncertainty estimation. The evaluation metrics include test accuracy, MC accuracy, precision, recall, and average uncertainty (quantified via prediction entropy using MCD).

The following pretrained CNN models were compared:

ResNet50, EfficientNet_B0, and DenseNet121

For this experiment, a baseline CNN was first trained for classification using TensorFlow. Then, features were extracted from skin spot images using a pretrained model (with the final classification layer removed) in PyTorch. The dataset was divided into training, validation, and test sets prior to feature extraction. A GTN was subsequently trained on these features and evaluated on the test set, with uncertainty estimated via MCD.

Table VI summarizes the performance metrics of each model on the test set.

Table VI. Results of pretrained CNN models for feature extraction

Figure 7 illustrates the precision, recall, and F1-score curves for these models, while Fig. 8 displays the average uncertainty (entropy) for each model. (Replace the placeholder file names with your actual plot files if available.)

Fig. 7. Precision, recall, and F1-Score comparison for different pretrained models.

Fig. 8. Average uncertainty (entropy) for each pretrained model.

The experimental results indicate that the choice of the pretrained CNN significantly influences both classification performance and uncertainty estimation:

• •ResNet50: Achieving a test accuracy of 91.22% and the lowest average uncertainty (0.1047), ResNet50 demonstrates excellent feature extraction capabilities. Its high MC accuracy (92.92%) further indicates reliable and well-calibrated predictions.
• •EfficientNet_B0: With a slightly lower test accuracy of 87.89% and higher average uncertainty (0.1637), this model suggests that while it provides competitive performance, the discriminative power of its features is comparatively less robust for this task.
• •DenseNet121: Although achieving high precision and recall (0.94 each), DenseNet121 records a test accuracy of 88.80% and the highest uncertainty among the three models (0.1874), indicating potential calibration issues despite capturing useful hierarchical features.

ResNet50 emerges as the ideal backbone for GraphSkinUQ: its deep residual architecture outperformed both smaller models (e.g., EfficientNet-B0) and alternatives (e.g., DenseNet121), achieving top test accuracy (91.22%), MC accuracy (92.92%), and lowest average predictive uncertainty (0.1047). It offers a practical trade-off between representational capacity and computation, avoiding the overfitting of very deep networks and the limited expressiveness of compact ones. Under identical preprocessing, graph transformer settings, and hyperparameter searches, ResNet50 consistently led in both accuracy and uncertainty calibration.

2).EXPERIMENT 2: IMPACT OF k IN k-NN ON PERFORMANCE AND UNCERTAINTY IN OUR MODEL

In this experiment, we investigate how varying the number of neighbors (k) used in constructing the k-NN graph affects the performance and uncertainty estimation of a GTN for skin cancer classification. The evaluation metrics include accuracy, precision, recall, F1-score, and uncertainty (measured as entropy).

The experimental procedure is as follows:

• 1.Baseline Feature Extraction: A pretrained ResNet50 (with its final classification layer removed) is used via PyTorch to extract deep features from the skin cancer dataset. The dataset is split into training, validation, and testing sets.
• 2.k-NN Graph Construction: For each selected value of k, a new k-NN graph is built using the extracted features.
• 3.Graph Transformer Training: A GTN is trained on each graph. The network is optimized using early stopping based on validation accuracy.
• 4.Uncertainty Estimation: MCD is employed during inference (by keeping dropout active) to obtain multiple stochastic predictions. Prediction probabilities are averaged and used to compute the entropy as a measure of uncertainty.
• 5.Evaluation: For each value of k, we compute accuracy, precision, recall, F1-score, and the average uncertainty. These results are recorded, printed, and later plotted.

Table VII summarizes the performance metrics for different k values.

Table VII. Performance metrics and uncertainty vs. k in k-NN graph

Figure 9 shows the performance metrics (accuracy, precision, recall, and F1-score) as functions of k, and Fig. 10 plots the corresponding uncertainty (average entropy).

Fig. 9. Performance metrics (accuracy, precision, recall, and F1-score) versus k in the k-NN graph.

Fig. 10. Average uncertainty (entropy) versus k in the k-NN graph.

Varying the k-NNs in our graph transformer shows a clear optimum at k=10: at this setting, the model achieves its highest accuracy (91.61%), precision (90.64%), recall (90.79%), F1-score (91.07%), and lowest uncertainty (0.1074). Smaller values (e.g., k=3) yield strong recall but elevated uncertainty, while larger values (e.g., k=15) introduce marginal noise without improving accuracy.

Thus, k=10 balances informative locality with minimal noise, delivering the best trade-off between predictive performance and calibration.

3).EXPERIMENT 3: ABLATION STUDY AND MODEL VARIANTS

To further understand the contribution of each component in our proposed framework, we conducted an ablation study comparing three variants:

• 1.Full Model (GraphSkinUQ): Our complete framework integrates ResNet50 feature extraction, graph construction, and a GTN with MCD.
• 2.Transformer Only (Raw Pixels): A model that directly applies the transformer on raw pixel inputs without any CNN-based feature extraction.
• 3.ResNet CNN Only: A conventional ResNet-based CNN trained for classification, without the graph transformer module.

Table VIII summarizes the performance of these variants in terms of accuracy, recall, and F1-score.

Table VIII. Ablation study: Comparison of model variants

As shown in Fig. 11, the full GraphSkinUQ model attains the best results (accuracy 92%, recall 91%, and F1-score 91%) by combining ResNet50 feature extraction, graph-transformer relational learning, and MCD uncertainty estimation. The transformer-only variant applied directly to pixels performs poorly (accuracy 53% and recall 10%), showing that deep CNN features are indispensable. A ResNet50-only classifier reaches 85% accuracy and 84% F1-score but lacks the relational context that the graph module provides, explaining the performance gap.

Fig. 11. Ablation study: Comparison of model variants.

Overall, this ablation confirms that both deep visual embeddings and graph-based learning are essential for high accuracy and reliable uncertainty estimates.

4).EXPERIMENT 4: COMPARISON OF GRAPHSKINUQ WITH OTHER MACHINE LEARNING MODELS

Table IX summarizes the performance metrics of our proposed model, GraphSkinUQ, compared with five other approaches: SVM, RF, a GCN, a GAT, and a simple MLP trained on deep features. The metrics reported include accuracy, F1-score, and Brier score (as a measure of calibration), as well as inference time for the classical machine learning approaches.

Table IX. Performance comparison of GraphSkinUQ and other models

All of the compared models—GraphSkinUQ, GCN, GAT, MLP, SVM, and RF—are trained and evaluated on the same high-level image representations extracted by a pretrained ResNet50 backbone (with its final classification layer removed). Specifically, each dermoscopic image is passed through ResNet50 to produce a 2048-dimensional embedding; these embeddings serve as the sole input features for the MLP, SVM, and RF classifiers, and as the node attribute matrix for the graph-based GCN, GAT, and GraphSkinUQ models. This unified feature foundation ensures that any observed performance differences stem purely from the modeling and uncertainty estimation strategies—rather than disparities in raw data or preprocessing—and allows a fair comparison of shallow versus graph-enhanced deep learning approaches in terms of accuracy, calibration (Brier score), and inference efficiency.

Table IX and Fig. 12 compare GraphSkinUQ against SVM, RF, GCN, GAT, and an MLP, all using the same 2048-dimensional ResNet50 embeddings. GraphSkinUQ achieves the highest accuracy (91.54%), strong F1-score (0.91), and competitive calibration (Brier = 0.1382) with a modest inference time of 4.78 s. While SVM attains similar accuracy (90.94 %) and the lowest Brier score (0.1123), its inference time (27.03 s) is substantially higher. RF and the graph-only baselines (GCN: 86.10 %, GAT: 87.92%) lag in accuracy and calibration, and the MLP (88.22%) falls short of leveraging relational context. These results demonstrate that GraphSkinUQ’s combination of graph transformer relational learning and MCD yields the best balance of accuracy, reliability, and efficiency.

Fig. 12. Performance comparison of GraphSkinUQ and other models.

In addition to quantitative evaluation, we conduct a comparative analysis of several recent skin lesion classification frameworks and our proposed GraphSkinUQ model. Table X highlights the differences among these methods in terms of UQ, graph reasoning, explainability, and computational efficiency. Most existing models, such as SkiNet [19], StructureNet [20], and Three-Way Bayesian DL [22], incorporate uncertainty estimation or advanced feature extraction but rely on computationally heavy Bayesian or ensemble strategies. Others, including SkinNet-14 [5], ViT [23], Tiny Pyramid ViG [28], and GAT-Multiscale Fusion (MSF) [29], achieve high accuracy through attention or multiscale fusion yet lack calibrated confidence estimation or relational reasoning. GraphSkinUQ addresses these gaps by integrating MCD-based UQ within a graph transformer, achieving reliable confidence estimation with moderate complexity (≈11 M parameters). This balance between accuracy, interpretability, and efficiency makes GraphSkinUQ suitable for safety-critical diagnostic use.

Table X. Performance comparison of GraphSkinUQ and other models

5).EXPERIMENT 5: EVALUATION OF GRAPHSKINUQ ON THE ISIC 2019 BENCHMARK DATASET

To further strengthen the experimental validation and ensure comparability with existing state-of-the-art frameworks, we evaluated GraphSkinUQ on the publicly available ISIC 2019 dataset—one of the largest and most widely used benchmarks for skin lesion classification. The dataset contains 25,331 dermoscopic images distributed across eight diagnostic categories, including melanoma (MEL), nevus (NV), basal cell carcinoma (BCC), and actinic keratosis (AK). We employed the same preprocessing pipeline and training configuration as used in our primary experiments, including image normalization, data augmentation, and a ResNet50–ViT feature fusion backbone. All models were trained under identical conditions to ensure a fair comparison.

GraphSkinUQ achieved an accuracy of 83.2%, a macro F1-score of 0.80, and a ROC–AUC of 91.0%, demonstrating its strong generalization capability across diverse lesion types. As shown in Table XI, this performance is comparable to or surpasses several state-of-the-art approaches on the ISIC 2019 dataset, including SkiNet (81%) [19], FCDS-CNN (80%) [16], and GAT-MSF (84%) [25], while maintaining moderate computational complexity and offering reliable UQ via MCD. These results confirm that GraphSkinUQ is not limited to the custom dataset introduced earlier but also performs competitively on a globally recognized benchmark, reinforcing its credibility, efficiency, and clinical applicability.

Table XI. Comparative results on the ISIC 2019 dataset