As a result, the expression ‘hot hand fallacy’ was coined – the belief that a momentum effect exists when it doesn’t. But a recent paper provides what I think is rare and persuasive evidence of a ‘hot hand’ effect in a sporting event.
The event at the centre of this analysis is the NBA Three-Point Contest. It has been held annually since 1986 (with the exception of 1999), as part of the NBA All-Star Game festivities. A pair of economists, Joshua Miller and Adam Sanjurjo, found video recordings of those contests and analyzed the results1.
In the contest, players shoot 25 attempts per round. To get to 100 attempts, and be included in the study, a shooter has to play at least four rounds in his career. Here's a video of the 2013 competition.
They found that players were significantly more likely to make a shot after a series of three hits than otherwise. Among the 33 shooters who had at least 100 shots in their careers, the average player hit 54% overall, but 58%2 after three consecutive hits.
After the authors corrected for biases (which I discuss further on my blog), they noted, the ‘hot hand’ effect jumped from four percentage points to six. That's a big effect, a difference of six percentage points.
Maybe it's easier to picture it another way
Of the 33 players, 25 of them shot better after three consecutive hits than their overall rate. In other words, the ‘hot hand’ beat the ‘mean-reverting hand’ with a win-loss record of 25-8. With the adjustments included, the hot hand jumps to 28-5.
To a minor extent, this could be due to selective sampling of players. But (again as I discuss on my blog), any such effect is probably very minor. One reason I believe the effect is real is that it makes much more intuitive sense to expect a hot hand in this kind of competition than in normal NBA play.
In each round of the contest, players shoot five consecutive balls from the same spot on the court, in immediate succession. That seems like the kind of task that would easily show an effect. It seems to me that a large part of this would be muscle memory – once you figure out the shot, you just want to do exactly the same thing four more times (or however many balls you have left once you figure it out).
After those five balls, you move to another spot on the arc for another five balls, and so on, and the round ends after you've thrown five balls from each of five locations. However, even though the locations move, the distances are not that much different. So some of the experience gained earlier might extend to the next set of five, making the hot hand even more pronounced.
There's one piece of the evidence that offers support for the ‘muscle memory’ hypothesis. It turns out that the first two shots in each round were awful. The authors report that the first shot was made only 26% of the time, and the second shot only 39%. For the remaining twenty-three shots, the average success rate was 56%.
I bet you'd find a hot hand if you tried the equivalent contest yourself. Position a waste basket somewhere in the room, a few feet away. Then, stay in one spot, and try to throw scrunched up bits of paper into the basket. I'm guessing your first one will miss, and you'll adjust your shot, and then you'll get a bit better. Eventually, you'll be sinking 80 to 90% of them, which means you have a ‘hot hand’. Once you get the hang of it, you'll be able to just repeat what you learned, so hits will tend to follow hits.
So, does this study actually refute the hot hand fallacy?
Well, it refutes it in its strongest form, which is the position that there NEVER exists a hot hand of ANY magnitude, in ANY situation. That's obviously wrong. You can prove it with the scrunched paper and waste basket example.
The real ‘fallacy’, as I see it, is not the idea that a hot hand exists at all, but the idea that it's a significant factor in predicting what's going to happen next. In most aspects of sports, the hot hand (when it does exist) is so small as to have almost no predictive value.
Suppose a player has two kinds of days, equal and random – ‘on’, where he hits 60% of shots, and ‘off’ where he only hits 50%. That would give rise to a hot hand, obviously, but how big a hot hand? What should you predict as the chance of the player making his next shot?
Before the game, you'd guess 55% – maybe he's on, or maybe he's off. But, now, he hits three straight shots. He has a hot hand! What do you expect now? If my math is right, you should now expect him to shoot… 56.3%. Not much different!
The ‘50/60 on/off’ actually represents a huge variation in talent. The problem is that streaks are a weak indicator of whether the player is actually ‘on’, versus whether he just had a lucky three shots. In real life, it's even weaker than a 1.3% indicator, because for one thing, how do you know how long a player is ‘on’ and how long he's ‘off’? I assumed a full game, but that's wildly unrealistic.
The hot hand issue reminds me of the ‘clutch hitting’ issue in baseball. Both effects probably exist, but are so tiny that they're pretty much useless for any practical purposes. Academic studies fail to find statistically significant evidence, and imply that ‘absence of evidence’ implies that no effect exists. We sabermetricians cheat a little bit, saving effort by saying there's ‘no effect’, instead of ‘no effect big enough to measure’.
So ‘no effect’ becomes the consensus. Then, someone comes up with a finding that actually measures an effect – like this study for the hot hand. And those who never disbelieved it jump on the news, and say, ‘Aha! See, I told you it exists!’ But they still ignore effect size.
People will still declare that their favorite baseball hitter is certainly creating at least a win or two by driving in runs when it really counts. But now, they can add, ‘because, the hot hand exists, it's been proven!’ In reality, there's still no way of knowing who the best clutch hitters are, and even if you could, you'd find their clutch contribution to be marginal.
And now, I suspect, when the Yankees win five games in a row, the sportscasters will still say, ‘they have momentum! They're probably going to win tonight!’ But now, they can add, ‘because, the hot hand exists, it's been proven!’ In reality, the effect is so attenuated that their ‘hotness’ probably makes them a 0.501 expectation instead of 0.500 and, probably, even that one point is an exaggeration.
My bet is that the ‘hot hand’ narrative won't change, but now it will claim to have science on its side.
This article is an edited version of a post that first appeared on Phil's Sabermetric Research blog.
Footnotes
1. ‘Is It a Fallacy to Believe in the Hot Hand in the NBA Three-Point Contest?’
2. The 58 percent figure is approximate: the study reports an increase of four percentage points after three consecutive hits than after other sequences. Because the authors left out some of the shots in some of their calculations (as discussed in my original blog post), it might be more like 59% vs. 55%, or some such. None of the discussion above depends on the exact number.)
