Primers
Short explanations of the methods behind the numbers.
How causes of death are coded, and why the coding matters
Every death certificate answers two questions: what happened, and what started it. The answer to the second is the underlying cause of death. That is the one that becomes a national statistic. How it is chosen, and how the choosing has changed, explains a surprising share of the trends in cause-specific mortality.
The certificate. A doctor records a chain: the immediate cause on the first line, then the conditions that led to it, down to the one that began the sequence. Pneumonia, due to immobility, due to a stroke: the stroke is the underlying cause. Other conditions that contributed without being in the chain go in a separate section. The chain is a clinical judgement made under time pressure, sometimes by a doctor who did not know the patient.
The rules. The International Classification of Diseases assigns a code to every condition, and sets rules for choosing the underlying cause where the certificate is ambiguous or implausible. It is now in its tenth revision in most countries, with the eleventh arriving slowly. The rules are detailed and occasionally counter-intuitive. They exist so that the same certificate produces the same code in Copenhagen and Chicago. They are revised, and each revision moves deaths between causes.
Automation. Most countries now assign the underlying cause by software that applies the rules to the text of the certificate — IRIS in much of Europe, the American system before it. Automation improved consistency. Changes to the software, and to the rules it applies, both move the results. When England and Wales updated their coding rules in 2011, deaths recorded as dementia rose sharply in a single year. This was not because more people died of dementia, but because many deaths previously coded to stroke were now coded to vascular dementia. A switch to new software in 2014 produced a second step. Any series that crosses a change in rules or software has a step in it.
Garbage codes. A share of deaths are coded to causes that cannot be underlying — "senility", "heart failure" without a reason, "unspecified". That share varies from a few per cent in the Nordic countries to a third or more elsewhere. The Global Burden of Disease project reassigns these by algorithm before comparing countries, which is one reason its figures differ from national ones.
The pandemic as a case study. COVID-19 got its own codes within weeks, and countries applied them differently. Some counted any death within 28 days of a positive test, some required the disease to be on the certificate, some required it to be the underlying cause. The reported death tolls diverged accordingly, which is why excess mortality became the comparison of choice.
Multiple causes. The underlying cause is one line of the certificate; the whole certificate is more informative. Multiple-cause data — every condition mentioned — show, for instance, how often diabetes or dementia is present at deaths attributed to something else. A growing number of countries publish them, and they change the picture of what people die with as against of.
Reading a cause-specific trend. Three questions bear on a reported rise or fall. Did the classification revision, the coding rules or the software change across the period? Did certification practice change, as it did for dementia when doctors were encouraged to record it? And is the trend visible in the underlying-cause series only, or in multiple-cause data too? A trend that survives those questions is probably real.
How to read a preprint
A preprint is a research paper posted publicly before peer review. medRxiv, the server this site reads, screens for obvious problems — plagiarism, patient identifiers, dangerous claims — and posts within days. Nobody has checked the methods. Much of the best work now appears first as a preprint, so that is not a reason to ignore them, but it is a reason to read them differently.
What the label means on this site. Items marked preprint are drawn from medRxiv and carry that source type until the paper is published. The item is then updated to say so, and to note whether the findings changed. The preprints page keeps the tally: how many were published, how many changed, how many were withdrawn. In practice most are published, most change little, and a few change a great deal.
What to check, in order.
The design. Is it a registry cohort, a trial, a cross-sectional survey, or a model? The design sets a ceiling on what the paper can claim, and no sample size lifts it. A large cross-sectional study cannot show that one thing caused another.
The population and the numbers. Who was studied, how many, and over what period. Then whether the headline number comes from the main analysis or from a subgroup. A finding in a subgroup of a subgroup is a hypothesis rather than a result, however it is worded.
Does the abstract match the tables? The abstract is written last, and written to be read. Where a result in the tables is weaker, not significant, or absent from the abstract, the tables are the paper.
Preregistration and data. Was the analysis plan filed before the data were seen? Are the data and the code available? Neither guarantees anything, but both make it harder for a result to be the product of many tries.
Who paid, and who wrote. Funding is declared, and the declaration is worth reading. Author affiliations show whether the people who made the data are the people analysing it.
Plausibility. Does the size of the effect fit what is already known? A new drug that halves all-cause mortality in an observational study is more likely to be confounded than miraculous.
What peer review would and would not have caught. Reviewers check reasoning, methods and interpretation, and sometimes catch errors. They rarely re-run analyses, seldom see raw data, and never remove the incentive to make a result sound bigger than it is. A published paper is a reviewed preprint, not a different kind of object.
When to wait. A preprint is filed, labelled, and given the new verdict rather than confirms or contradicts until it has been published or replicated. The site reports what it says, and waits for the evidence to go through more hands.
How vaccine effectiveness is measured
A vaccine that is 60 per cent effective does not protect 60 per cent of people. It reduces the rate of the outcome by 60 per cent in the vaccinated as compared to the unvaccinated. Most of the confusion about vaccine effectiveness comes from that sentence. Most of the rest comes from not asking which outcome.
Efficacy and effectiveness. In a randomised trial, participants are assigned to vaccine or placebo. The two groups then differ only by the vaccine, and the reduction in disease is the vaccine's efficacy. In the real world, people choose whether to be vaccinated, and those who do differ from those who do not: older, sicker, more likely to seek care, more cautious. Effectiveness is the estimate from that real world, and the work is in removing the differences that are not the vaccine.
The test-negative design. This is the design used in most influenza and COVID-19 effectiveness studies. Take everyone who arrives at a clinic or hospital with symptoms and is tested, then compare vaccination rates among those who test positive and those who test negative. Everyone in the study sought care, so the comparison is protected against the largest bias, that vaccinated people go to the doctor more. The test is the outcome, so misclassification is limited. It is not perfect: it assumes the vaccine does not affect the other illnesses that bring people in, and it estimates effectiveness against medically attended illness rather than against all infection.
Linked cohorts. Some countries have vaccination registers joined to hospital and death data. Those linked records allow whole populations to be followed, and effectiveness against hospitalisation and death to be estimated directly, adjusting for age, prior infection and comorbidity. The Nordic countries, the United Kingdom, Israel and Qatar produced most of the pandemic's estimates this way. Residual confounding remains, through the healthy vaccinee effect, where people well enough to be vaccinated are also less likely to die of anything.
Which outcome. Effectiveness against infection is lowest and wanes fastest. It is higher against symptomatic illness, higher still against hospitalisation, and highest and most durable against death. A report of effectiveness without an outcome is not a report. Influenza vaccines have typically shown a third to a half against medically attended illness, varying with the match between the vaccine and the circulating strains. The RSV vaccines for older adults showed high first-season effectiveness against hospitalisation, with the durability into later seasons the open question.
Waning and time since dose. Effectiveness measured in the month after vaccination is not the effectiveness six months later. Good studies report by interval since dose. Comparisons across studies that do not are comparisons of different things.
Reading an estimate. The things that matter are the design, the outcome, the population, the interval since dose, and the confidence interval, which for a single season's estimate can run from twenty per cent to sixty. A point estimate quoted alone has usually been stripped of the part that mattered.
Healthy life expectancy: what it measures and why the numbers disagree
Life expectancy counts the years lived. Healthy life expectancy tries to count the years lived in good health. It is the number behind the question of whether added years are good years, and it is measured in enough different ways that comparing two figures is often comparing two definitions.
The idea. A life table sets out survival by age. Each year of life is split into time spent in good health and time spent in poor health, using the proportion of people at each age who report themselves healthy. The healthy portions are added up to give healthy life expectancy: the years a person can expect to live in good health under current conditions. The arithmetic, known as the Sullivan method, is simple. Everything difficult is in the word "healthy".
The definitions. The Office for National Statistics publishes healthy life expectancy based on people's own rating of their general health as good or better, and separately disability-free life expectancy based on whether a long-standing condition limits their activities. Eurostat's healthy life years use a single question about activity limitation asked across the European Union. The Global Burden of Disease's health-adjusted life expectancy weights each condition by its severity from a disability schedule, rather than asking anyone how they feel. Japan's official measure asks about limitation in daily life. These are four different questions, answered by different surveys, and their results are not interchangeable. A country can have a high figure on one and a middling figure on another.
The problem with asking. Self-reported health depends on culture, expectation and the language of the question. Populations with better access to diagnosis report more conditions. Older people rate their health relative to their peers. Translation changes meaning. The measures are consistent within a country over time, which is what they are mostly used for. Across countries they carry a warning.
Compression or expansion. The question that matters is whether the healthy portion is growing as fast as the whole. If people live longer but the extra years are spent in poor health, morbidity is expanding. If illness is being squeezed into a shorter period at the end, it is compressing. In the United Kingdom, healthy life expectancy stopped rising around 2011 and has fallen for women since, so the years in poor health have grown. Other countries show compression on some measures and expansion on others, and the answer depends partly on which measure is chosen.
Inequality. Healthy life expectancy varies far more than life expectancy does across a country. In England the gap between the most and least deprived areas is of the order of two decades of healthy life against about a decade of life. That is why healthy life expectancy is the measure that appears in policy targets, and why its stalling is a policy story as well as a demographic one.
Reading a figure. Three things separate one figure from another. The first is which definition, which survey and which question lie behind it. The second is whether the comparison is across time within one series, which is usually sound, or across countries or definitions, which usually is not. The third is whether the change is in the health portion or in the life table beneath it, because a pandemic year lowers healthy life expectancy by lowering survival, not by changing anyone's health.
Age, period and cohort: reading the Lexis diagram
The diagram at the top of this page is a Lexis diagram. It is the most useful picture in demography. Calendar year runs along the bottom, age up the side. A person is a diagonal line. The line starts at the bottom in their birth year and moves up and to the right, one year of age for every calendar year, until it stops.
The diagram can be cut three ways. The three cuts correspond to three kinds of explanation for anything that happens to a population.
Age. A horizontal cut. Mortality at 80 is higher than at 40 because of what ageing does to bodies, whatever the year and whoever the person. Age effects are the most stable thing in mortality, and the reason age standardisation exists.
Period. A vertical cut. Something happens in a calendar year that affects everyone alive at the time: a pandemic, a heatwave, a new treatment reaching the whole population, a change in how deaths are coded. A period effect shows up as a vertical stripe across all ages at once. The pandemic is the clearest period effect in a century of data.
Cohort. A diagonal cut. People born in the same years share exposures that stay with them. The generation that took up smoking in the 1940s carried its lung cancer forward for fifty years. Children born in a famine carry its consequences into old age. The cohorts born after about 1960 in high-income countries carry higher early-onset cancer incidence at every age so far. Cohort effects move diagonally across the diagram, and they are the ones most often mistaken for something else.
Why they cannot be fully separated. Age equals period minus cohort. Any of the three is fixed by the other two, so a statistical model cannot estimate all three freely. Some assumption has to be made, and different assumptions give different answers. This is the age-period-cohort identification problem. It is why two competent papers can attribute the same rise in a disease to a cohort effect and to a period effect. Where a study claims to have separated them, the question is what it assumed.
Period life expectancy is not a forecast. The life expectancy quoted in the news — 79 for men, 83 for women, or whatever the latest year says — is a period measure. It is the length of life a hypothetical person would have if they experienced this year's death rates at every age for the rest of their life. Nobody does. A boy born this year will experience next year's rates at age one and the rates of the 2090s at age 70, and if mortality keeps improving those will be lower. Cohort life expectancy tries to account for that. It is higher, and it is a projection. Period life expectancy is a summary of the present. A fall in period life expectancy in a pandemic year does not mean anyone's life was shortened by that amount; it means that year's death rates were high.
What the diagram is for on this site. When an item reports a change, the first question is which cut it belongs to. A rise across all ages in one year is a period story. A rise that tracks birth year is a cohort story. A rise concentrated at older ages as a population ages may be no story at all. The diagram is the habit of asking.
How excess mortality is estimated
Excess mortality is the number of deaths in a period above the number expected without some event: a pandemic, a heatwave, a severe influenza season. It is the measure that made the pandemic comparable across countries, because it does not depend on who was tested or how a death was certified. It is also a measure that two careful analysts can calculate differently by a factor of two. The reasons are set out below.
The baseline. Expected deaths have to come from somewhere. The simplest baseline is the average of the previous five years. That was common in 2020, and it produces a specific bias: populations are ageing, so a flat average understates the deaths that would have happened anyway and overstates the excess. A trend-based baseline extends the pre-event pattern forward instead. The choice of baseline period matters too. Whether 2020 and 2021 sit inside the reference window changes every estimate for 2022 onwards, and analysts disagree about whether they should.
Age standardisation. A country with more old people has more deaths. Comparing raw counts across countries, or across years within an ageing country, confuses population structure with risk. Standardised estimates apply age-specific death rates to a fixed population, so a change means a change in risk rather than a change in the number of eighty-year-olds. The Office for National Statistics moved to an age-standardised, trend-based method in 2024 and revised its UK excess figures as a result. The two series are not comparable.
Registration. Deaths are counted when registered, not when they occur. In England and Wales, deaths referred to a coroner can register months late, so recent weeks are always incomplete and always revised upwards. Weekly series either wait, model the delay, or carry a known undercount in the latest weeks. Provisional figures from the United States are similar.
Denominators. Excess mortality needs a population to divide by. Between censuses, population estimates are themselves estimates. A census that finds fewer people than expected — as several did in 2021 and 2022 — rebases every rate calculated in the years before.
The models. Beyond the baseline and the standardisation, analysts choose a model for the expected count: a Poisson regression with seasonal terms, a Serfling-style cyclical fit, or the Farrington algorithm used in outbreak surveillance. These matter less than the choice of baseline, but they are not nothing. They explain why EuroMOMO's figures, the Short-Term Mortality Fluctuations series and a national office can all be right and all differ.
Why the estimates differed so much. For 2020 and 2021, the World Health Organization's global estimate was close to three times the count of deaths reported as COVID-19. The gap is not fraud on either side. Reported deaths depend on testing and certification. The WHO figure is modelled, with wide intervals, and includes countries with no reliable registration at all. The Economist's model and the Institute for Health Metrics and Evaluation's produced different global totals from similar inputs. Where national data are good, the estimates converge; where they are not, the model is doing the work.
Reading an excess figure. Four questions account for most of the difference between one estimate and another. What is the baseline period, and does it include the pandemic years? Is the figure age-standardised? Are the most recent weeks complete or provisional? And does the series revise its own history? Where it does, a figure published today may not be the figure in next year's series.