Do NHS records reflect patient ethnicity?

Routine recording of ethnicity is therefore part of the provision of fair and equitable health services. We can see where and why inequalities are occurring and – knowing this – unfairness or disadvantage can be removed.

But how we measure ethnicity matters. The NHS currently sorts individuals into one of sixteen ethnic categories (see table 1 below). While this simplifies measurement, it also simplifies the multidimensional nature of ethnicity. For example, shared culture or shared ancestry may be one element of how ethnicity is defined; language, national identity, religion and country of birth also play a part.

Table 1. Sixteen group classification of ethnicity currently used in the NHS, based on the 2001 UK Census classification

However, even if we accept the limitations of the approach currently used, we should still expect that the ethnic category assigned to an individual in their hospital record is a match to the ethnic group with which the individual self-identifies.

This may seem obvious but it is not always the case. Here, statistics and statisticians can help, by telling us whether, and how well, what is measured reflects what is actually true.

Specificity, sensitivity and positive predictive values

Let's start by looking at the four different possible combinations for what is measured and what is true using the example of medical tests to screen for or diagnose different diseases.

For medical tests we usually want to know how many of the people who truly report 'yes' are identified as such by the test. This is the 'sensitivity' of a test: referring to table 2, we would calculate this by dividing the number of people who report 'yes' and are identified by the test, a, by the total number of people overall who are 'true' yeses (a + b).

Table 2. What is measured and what is true? This table shows the four possible combinations of correct and incorrect measurement

Often, when statisticians are evaluating what is measured, they are not interested in the performance of the measure itself but only in how well it predicts what is actually happening. Symptoms as a predictor (or not) of cancer are a good example from clinical medicine. We want to know how well our measure (the symptom) predicts something much more serious which might be happening (possible cancer).

Here we would calculate the proportion of all people with the measured symptom who actually have cancer, a, divided by all people who have the symptom, whether or not they have cancer (a + c). This gives us a positive predictive value for a symptom. A positive predictive value of even a few percent might be enough to suggest that further investigation of a symptom is worthwhile.

Returning to the subject of ethnicity, we want to know how well the NHS 'measurement' of an individual's ethnic group reflects that person's true self identity. To be more specific, we want to know the answers to the following two questions:

1. If a patient self-identifies that they belong to a particular ethnic group, what is the probability that the hospital record will record the same (correct) ethnicity? This is the sensitivity of hospital-recorded ethnicity.
2. If a patient's hospital record states that they belong to a particular ethnic group, then what is the probability that this code has been recorded correctly, and that the patient self-identifies with this same ethnicity? This is the positive predictive value of the hospital-recorded ethnicity.

To answer these two questions we compared the self-reported ethnicity from survey responses from about 70,000 cancer patients to the ethnic group contained in their hospital records.¹

Figure 1. The sensitivity and positive predictive value of NHS hospital records for a person’s true self-identified ethnicity^{1, 2}

But for people from well-defined ethnic groups – Black African and Black Caribbean, Pakistani, Indian, Chinese or Bangladeshi groups – ethnicity is recorded correctly 60-80% of the time.

Irish ethnicity is usually correct if it is recorded in a person's hospital record (a high positive predictive value), but half of all people who self-identify as Irish will not have this recorded as their ethnicity (low sensitivity); mostly these people have White British ethnicity recorded instead.

If someone self-identifies as one of a number of mixed ethnic groups, this background will probably not be recorded in their hospital notes correctly, and if someone has a mixed background recorded in their notes it is probably wrong.

Discordance and concordance

Our analysis looked at whether each ethnic group in turn was recorded correctly or not. But we can also estimate an overall measure of agreement between NHS hospital records and self-identified ethnic group. A simple way to do this would be to estimate the percentage of people overall who have incorrectly recorded ethnicity – doing so, we see that only 4% of NHS records are discordant.

However, this low value doesn't reflect the differing sizes of the different ethnic groups. Cohen's Kappa is a different measure, that takes account of the fact that what is measured and what is true may be concordant by chance alone. Using this measure, the concordance of measured and true ethnicity is much lower, at only 0.64 (a long way from the maximum possible value of 1, which would indicate complete concordance).

So what did we find? Hospital-recorded ethnicity is concordant with a person's self-identified ethnic group as long as that person is from a White British background, but for people from ethnic minority groups often it is not.  Overall concordance is high, but this is mostly because the numbers of people from non-White British ethnic groups are a minority – it is not a reflection of good measurement of ethnicity overall.

This article was a joint runner-up in the 2014 Young Statisticians Writing Competition, organised by Significance and the Young Statisticians Section of the Royal Statistical Society. The winning entry, 'Does New York City really have as many rats as people?' by Jonathan Auerbach, was first published in the October 2014 print edition of Significance. Our other runner-up, 'Does Christmas really come earlier every year?' by Nathan Cunningham, was published online last month.

Footnotes

• 1. Saunders CL, Abel GA, El Turabi A et al. Accuracy of routinely recorded ethnic group information compared with self-reported ethnicity: evidence from the English Cancer Patient Experience survey. BMJ Open 2013; 3.
• 2. When using the 16-group ethnicity classification (table 1) in analyses, six broader groups are sometimes used: White (red labels), Mixed (blue), Asian or Asian British (green), Black or Black British (purple), Chinese (pink) and Other (black).