Where external validation has been done properly, the gains are modest. Machine-learning models improve on discrimination over well-built conventional models by a small margin. The gains are larger where the conventional model was poor, or the data unusually rich — full electronic records, imaging, wearables. Papers claiming that a model predicts death, or a disease, better than the standard tool now appear weekly, and a majority are trained and tested in a single dataset, with no external validation. There is almost no evidence that better prediction has changed an outcome. Prediction and action are different problems.
Detection is further along than prediction. Reading images is what the technology does best. The first randomised trials of AI-supported screening, mammography in Sweden most prominently, found more cancers at similar false-positive rates and with less radiologist time. The Swedish trial has since reported 12% fewer interval cancers, those found between screens. Whether that becomes fewer deaths is not yet known. Language models have entered clinical work as note-takers and document summarisers faster than any regulator has assessed them, on the strength of time saved rather than outcomes measured.
At population scale the evidence is thinner still. National statistical offices have begun testing machine-learning methods for coding causes of death, imputation and nowcasting, with mixed published results. The technology is real and the claims outrun the evidence. The studies that would settle it — external validation, impact on decisions, performance across populations — are rare.
