Whether machine learning predicts mortality better than the models we have

Papers claiming that a machine-learning model predicts death, or a disease, better than the standard tool now appear weekly, and a majority are trained and tested in a single dataset with no external validation. Where validation has been done properly, the pattern is consistent: modest gains in discrimination over well-built conventional models, larger gains where the conventional model was poor or the data unusually rich — full electronic records, imaging, wearables — and almost no evidence that better prediction has changed an outcome, because prediction and action are different problems. At population scale the picture is thinner still: national statistical offices have begun testing machine-learning methods for coding, imputation and nowcasting, with mixed published results. The honest state is that the technology is real, the claims outrun the evidence, and the studies that would settle it — external validation, impact on decisions, performance across populations — are rare.

Status

emerging

Direction

strengthening

Would change our view

An externally validated model showing a large, reproducible gain over the best conventional model on a population-level outcome; or a trial in which acting on a model's predictions changed mortality.

Trackers

Gaps

Validation across countries and ancestries; comparisons against strong rather than weak baselines; anything from the offices on operational use.