Evaluation of AI-based decision support in infectious diseases

Abstract

Antibiotic resistance is a growing global health concern, driven in part by the misuse and overuse of antibiotics. Effective and rapid treatment decisions are important both for individual patient outcomes and for limiting resistance development. In recent years, the exploration of AI-based methods for diagnostics and decision support has increased. In Confidence-based prediction of antibiotic resistance at the patient level, Inda-Díaz et al. (2026) developed a transformer model for diagnostics using European data. This thesis focuses on evaluating the model for Escherichia coli infections and nine antibiotics using additional European data and data from The Public Health Agency of Sweden. A baseline evaluation applies the model to both datasets to compare performance. To address potential biases caused by differences in data distributions, three mitigation approaches are applied to adapt the model: decision threshold adjustment, fine-tuning, and retraining. Performance is assessed using standard metrics in the field, both overall and stratified by antibiotic, patient age, and gender. Fairness is evaluated through subgroup differences in error rates, inspired by the fairness criteria equalized odds and equal opportunity. The results indicate that, despite distributional differences, the baseline model surprisingly achieves better overall performance on Swedish data than on European data. Performance differences across gender and age are observed for both the baseline and adapted models, with the lowest performance seen for isolates from female patients and from the youngest age groups, compared to male patients and the oldest age groups. None of the adapted models reduce the error rate differences between genders, whereas all decrease the maximum pairwise difference across age groups. Stratifying by antibiotic shows only marginal differences in results for most antibiotics, but notable discrepancies for some. These findings suggest that future work toward fair and effective AI-based treatment decision support should combine fairness-aware bias mitigation with approaches that address complex, antibiotic-specific resistance mechanisms.