Last Friday November 2, The US Bureau of Labor Statistics (the Bureau) reported that the estimated unemployment rate for the month of October was 7.9%. If you go to the Bureau’s website you’ll see that their unemployment rate estimates have a margin of error of ±.19 percentage points. This means then that the unemployment rate could be anywhere from about 7.7% to 8.1% (rounding to one decimal place).
According to the Bureau, the unemployment rate for September was 7.8%, the figure that was also reported by political and economic commentators. But, considering the margin of error, this really means between about 7.6% and 8.0%. Now both the September and October estimates of 7.8% and 7.9% respectively, fall within the October margin of error of 7.7% to 8.1%. Thus, the October estimate could really be 7.8%, as was estimated, a little higher than that figure, or a little lower.
Pundits in the US typically do a lot of reporting on election polls and unemployment statistics. Even though these commentators frequently refer to the margin of error when covering elections, I’ve seen almost no mention of such errors when reporting unemployment and other labor market statistics.
When the October unemployment estimate came out last Friday, Romney criticized Obama because the unemployment rate had increased from 7.8% to 7.9%. Yet, given what I said above, it’s possible that Romney is correct in attacking Obama, but it’s also possible that he’s wrong. Once one considers the margin of error associated with unemployment rate estimates, it becomes clear that a change of 1/10 of a percentage point simply doesn’t mean much. That is, we have another example of a statistical dead heat.
Now it may be too much to ask politicians to be a little more careful regarding such matters since careful consideration of evidence may not be helpful for one’s campaign goals. But it seems to me that we should expect more from journalists and other supposedly objective commentators. But since even commentators may decide not to take up the task, I’ll offer a “quick and dirty” overview of why margins of error and reporting them matter.
An unemployment rate estimate is attempting to get at what proportion of people in the labor force is unemployed. An ideal way to obtain this figure would be to get hold of all people in the US labor force, those who are employed as well as those who are unemployed. One could then calculate the number of those unemployed divided by the number of employed plus the number of unemployed to obtain the unemployment rate. The problem with this approach is that it is very expensive to get a hold of everyone in the labor force. Thus, those who estimate unemployment rates rely on samples.
Statistical theory tells us three things about samples. One is how to select them so that what we find out about the sample is a good indicator of what’s going on in the population. Two is the reality that the result of any calculation we do for a sample is unlikely to equal the result of that same calculation if it were conducted on the entire population. The difference between the two is called sampling error. Three is how to account for sampling error in our estimates. Margins of error are the way we do this. The procedure is to calculate a statistic which serves as an estimate of some population value and then to follow the strictures of statistical theory to generate a margin of error around that estimate.
Margins of error are typically 90% or 95% margins. For example, the one reported on the Bureau’s website is a 90% margin of error. Assuming one is considering the classical, as opposed to Bayesian, tradition in statistical theory such a margin is interpreted as follows. If repeated samples from the same population were selected, we calculated an unemployment rate for each one, and created a margin of error for each of them, 90% of those intervals or margins would contain the true population unemployment rate. We can think of a margin of error as a statisticians’ way of saying “this is our best guess, at this point, of what’s going on in the population but we’ll include this margin just to hedge our bet a bit.” It’s a prudent procedure, in my view, and one that pundits and, perhaps, even politicians might do well to pay more attention to.
