How well do FIFA’s ratings predict World Cup success?

Two WCs (1994 and 1998) were played under the original 1993 system, two (2002 and 2006) under the 1999 system while the 2010 WC was played under the 2006 system, as will the upcoming 2014 WC. Are there other rating systems that might better serve FIFA for the men’s WC? As in life, men should look to women for the answer.

How the FIFA World Ranking system works

The three FIFA systems have the same general form. Match points are calculated for each game using five factors, Match points are summed over a period of years.

match points = result x opponent strength x importance x continent x ageing

Table 1 below shows the history of those five factors, along with a system I suggested in a letter to FIFA in 1996, which is quite close to the eventual 2006 FIFA system. FIFA combined my suggestion with others into a serious of options given to their consultants. In 1993, FIFA’s result points were based on 2,1,0, for wins, draws and losses respectively, even though FIFA mandated 3,1,0 points for all domestic and international matches. Those points were assigned to each team in some mysterious way that they would not publish, based on relative opponent strength.

In 1996, I suggested 3,1,0 points multiplied by a step-down table, so that progressively fewer points are gained when defeating or tying progressively worse-ranked opponents. In 2006, FIFA changed to 3,1,0 multiplied by 100 times a step-down equation, 2 – rank/100, which is more precise than the table I had suggested and which (coincidentally) went through the points in my 1996 step-down table.

In 1996 I suggested making a WC match count at least 3x that of a friendly, while FIFA undervalued a WC match as being only 1.5x as important as a friendly. They now count a WC match 4x as much as a friendly. The continental factors have been adjusted only slightly.

In 1993, FIFA used a six year window. I suggested using a more logical four-year WC cycle in 1996. FIFA moved to an unusually large eight-year window in 1999, which appears to have resulted in worse predictive accuracy, and then to a four year window.

Table 2 below shows the predictive success of the three FIFA systems over the last five WCs, based on pre-WC ratings. Ties were not counted in the group phase to facilitate comparison with the knockout phase and with other systems. The 1993 system was 69.2% successful in predicting winners in the 1994 and 1998 WCs. The 1999 system only predicted 64.1% of the 2002 and 2006 matches correctly, with an abysmal 37.5% accuracy in the 2006 knockout phase. On the other hand, the 2006 system resulted in 72.5% success in 2010 with an excellent 87.5% success in the knockout phase. The 2006 system is headed in the right direction, but might other systems be a better choice for FIFA?

Some better rating systems

World-wide attention focuses on the rugby and cricket WCs too; but, the rating systems are more predictive. The International Rugby Board, IRB, and International Cricket Council, ICC, use the same basic scheme for adjusting ratings after each match.

New rating = old rating + K [result – prediction]

The IRB and ICC both score a result 1, 0.5, 0 for a win, tie and loss respectively. The prediction (the probability of winning) is also limited to a 1-0 basis depending on rating difference, using a simple equation (0.5 + 0.5 diff /10 for rugby and 0.5 + 0.5 diff /50 for the David Kendix system used in cricket). K depends on importance and score difference in rugby and on importance and games played in cricket. The rugby system includes an adjustment for home advantage. The higher-rated team won an average of 83% of all matches in the 2007 and 2011 rugby WCs, with 90% success in 2011. David Kendix’ system was 79% predictive in the 2007 and 2011 One Day International WCs and 76% predictive in the 2014 T20 WC. Both systems outperformed FIFA’s men’s system.

FIFA need only look to their women’s Elo system for improvement. The form is the same as for rugby and cricket. FIFA cleverly converts game score to a 1-0 scale, instead of making every win the same and every loss the same. The prediction (again the probability of winning) is calculated from rating difference using the well-known Elo equation from chess ratings. The Elo system, which adjusts for home advantage, was 82% successful for the women’s WCs of 2007 and 2011. The rugby, cricket and women’s FIFA systems more precisely adjust after each win, tie or loss and each is more strongly weighted toward pre-WC strength. FIFA would do well to use the women’s Elo system for men’s competition as well.

The four top-ranked teams

Table 3 contains the success of the four top-ranked teams in the last five men’s football WCs using pre-WC rankings. Also shown are the top four as of 8 May, 2014. One more set of ratings will be published prior to the 2014 WC. Of the 20 teams, 75% reached the knockout phase, meaning that 25% of those supposedly dominant teams were eliminated in the group phase. 55% reached the quarter finals and 30% (6) reached the semi-finals. Three of those six won the WC and three were second. No top rated team won, presenting a challenge for Spain in 2014. The second-rated teams won twice, good news for Germany. The fourth-rated teams were second twice, with host nation Brazil currently being fourth. Brazil is particularly well placed to win, given a host-nation advantage. 18th ranked France won as the host nation in 1998. The average winner was ranked 8th, four positions worse than Brazil.

Please excuse me as I return to gnashing my teeth. Landon Donavan was not chosen for the US team. Silly me: I thought that loyalty, dedication and experience were reasons to be selected. Let the serious gnashing now begin.