Model | Dataset | Accuracy | Sensitivity | Specificity | PPV | NPV | AUC | 95% CI | aAUC (SD) |
---|---|---|---|---|---|---|---|---|---|
ResNet-18 | Internal Validation Set | 0.777 | 0.832 | 0.723 | 0.750 | 0.811 | 0.837 | 0.781–0.892 | 0.746 (0.018) |
Independent Test Set 1 | 0.674 | 0.767 | 0.652 | 0.340 | 0.923 | 0.726 | 0.644–0.807 |  | |
Independent Test Set 2 | 0.620 | 0.798 | 0.566 | 0.357 | 0.902 | 0.712 | 0.657–0.768 |  | |
ResNet-18a | Internal Validation Set | 0.767 | 0.861 | 0.673 | 0.725 | 0.829 | 0.813 | 0.753–0.874 | 0.754 (0.018) |
Independent Test Set 1 | 0.736 | 0.698 | 0.745 | 0.390 | 0.913 | 0.758 | 0.679–0.838 |  | |
Independent Test Set 2 | 0.700 | 0.677 | 0.706 | 0.411 | 0.878 | 0.728 | 0.673–0.782 |  | |
ResNet-34 | Internal Validation Set | 0.738 | 0.713 | 0.762 | 0.750 | 0.726 | 0.783 | 0.720–0.847 | 0.710 (0.019) |
Independent Test Set 1 | 0.740 | 0.581 | 0.777 | 0.379 | 0.888 | 0.680 | 0.586–0.775 |  | |
Independent Test Set 2 | 0.636 | 0.667 | 0.627 | 0.351 | 0.861 | 0.702 | 0.647–0.757 |  | |
ResNet-34a | Internal Validation Set | 0.777 | 0.871 | 0.683 | 0.733 | 0.841 | 0.813 | 0.752–0.874 | 0.746 (0.018) |
Independent Test Set 1 | 0.753 | 0.791 | 0.745 | 0.420 | 0.938 | 0.778 | 0.699–0.856 |  | |
Independent Test Set 2 | 0.638 | 0.788 | 0.593 | 0.370 | 0.902 | 0.725 | 0.672–0.778 |  | |
ResNet-50 | Internal Validation Set | 0.757 | 0.693 | 0.822 | 0.795 | 0.728 | 0.811 | 0.752–0.870 | 0.722 (0.019) |
Independent Test Set 1 | 0.599 | 0.814 | 0.549 | 0.297 | 0.927 | 0.698 | 0.606–0.790 |  | |
Independent Test Set 2 | 0.629 | 0.737 | 0.596 | 0.356 | 0.882 | 0.677 | 0.619–0.736 |  | |
ResNet-50a | Internal Validation Set | 0.733 | 0.911 | 0.554 | 0.672 | 0.862 | 0.794 | 0.732–0.856 | 0.733 (0.018) |
Independent Test Set 1 | 0.670 | 0.698 | 0.663 | 0.326 | 0.904 | 0.714 | 0.631–0.797 |  | |
Independent Test Set 2 | 0.660 | 0.747 | 0.633 | 0.381 | 0.892 | 0.738 | 0.687–0.790 |  | |
DenseNet-121 | Internal Validation Set | 0.743 | 0.911 | 0.574 | 0.681 | 0.866 | 0.806 | 0.747–0.866 | 0.674 (0.020) |
Independent Test Set 1 | 0.656 | 0.628 | 0.663 | 0.303 | 0.884 | 0.679 | 0.593–0.766 |  | |
Independent Test Set 2 | 0.540 | 0.687 | 0.495 | 0.292 | 0.839 | 0.607 | 0.543–0.671 |  | |
Swin Transformer | Internal Validation Set | 0.733 | 0.891 | 0.574 | 0.677 | 0.841 | 0.775 | 0.711–0.840 | 0.742 (0.018) |
Independent Test Set 1 | 0.722 | 0.628 | 0.745 | 0.365 | 0.895 | 0.722 | 0.643–0.800 |  | |
Independent Test Set 2 | 0.643 | 0.828 | 0.587 | 0.378 | 0.919 | 0.755 | 0.705–0.805 |  | |
Swin Transformera | Internal Validation Set | 0.733 | 0.832 | 0.634 | 0.694 | 0.790 | 0.788 | 0.725–0.850 | 0.737 (0.018) |
Independent Test Set 1 | 0.727 | 0.628 | 0.750 | 0.370 | 0.896 | 0.740 | 0.661–0.818 |  | |
Independent Test Set 2 | 0.646 | 0.818 | 0.593 | 0.379 | 0.915 | 0.729 | 0.677–0.781 |  |