Overachievers and underachievers at the last three Rugby World Cups

These official ratings are calculated by a system created on behalf of The International Rugby Board (now World Rugby). A team’s rating is adjusted after each match on a 0-1 scale, where a loss = 0, a tie = 0.5 and a win = 1, compared to the likelihood of winning, also on a 0-1 scale, based, in turn, on rating difference adjusted for home advantage. A rating goes up after a win, depending on the rating of the defeated team. Beating a weak team gains much less than defeating a strong team. Losing to a weak team reduces a rating much more than losing to a strong team. This common-sense system justified itself over the last three Rugby World Cups.

In each World Cup, four five-team groups played a group-wise round-robin total of 40 matches. The top two teams from each group went on to complete the eight-match knockout phase, resulting in a total of 48 matches per tournament. In 2007, the higher rated teams won 36 of the 47 non-tied matches (76.6%). That accuracy was exceeded in 2011 when the higher rated teams won 43 of the 47 non-tied matches (91.5%), and in 2015 when higher-rated teams won 42 of the 48 matches (87.5%). Only two of 144 matches were tied, and both ties were between Canada and Japan. Overall, higher-rated teams won 85.2% of non-tied matches in all three World Cups – an excellent result, speaking well of the rating system. Among 13 official sports federation rating systems for team sports having world championships or world cups, the World Rugby system was equal first in predictability with a similar system developed by David Kendix for ICC ODI cricket, ICC T20 cricket and IFNA netball.¹

Overachievement and underachievement by nations
To determine which teams outperformed and underperformed expectations, we now analyze the 21 upset wins and 21 upset losses, as summarized in Table 1 – an ‘upset’ being any match in which a lower-ranked team beats a higher-ranked opponent. Only 22 teams were needed to fill the 3×20 slots for the three World Cups. Russia and Uruguay did not compete in 2007 as was true of Portugal and Uruguay in 2011 and of Portugal and Russia in 2015. The 22 teams are listed by upset wins minus upset losses. The remaining matches went as expected. The two tied matches were not included.

Table 1. Distribution of the upset wins and upset losses among the 22 teams taking part in the 2007, 2011 and 2015 Rugby World Cups.
 

The five teams in the top panel earned a positive 'upset win minus upset loss' differential. Argentina scored five upset wins, two more than Tonga, the next closest team with upset wins. With no upset losses, Argentina came to play. In fact, the Pumas were expected to win eight and lose 11; but actually had a win-loss record of 13-6. Tonga had three upset wins, balanced somewhat by two upset losses. Fiji, Japan and Georgia had upset wins and no upset losses. All five of the teams that most over-achieved were expected to have losing records based on pre-WC ratings; yet, the five proved to be competitive enough to pull off 13 upsets in total, nearly two thirds of all upsets.

The nine teams in the middle panel balanced upset wins (if any) with an equal number of upset losses. The first five listed teams in the middle panel had no upsets at all, for 100% predictability. However, Namibia, Portugal, Russia and Uruguay accomplished that feat by losing all 24 matches collectively. Romania was more noteworthy in that they had a 2-10 record, losing but also winning as expected. Four teams had both upset wins and losses, in particular England with two of each. Unfortunately, England had both upset losses in 2015 at home (to both Wales and Australia, the latter based on adding England’s home advantage to their rating). At least Wales, Scotland and England achieved winning records as expected. The remaining six teams earned losing records as expected.

The bottom eight teams in the table may be the most interesting. Six of those teams had a balance of -1 for upset wins minus losses, four of which had excellent winning records, and achieved very close to demanding expectations. Among them, the All Blacks performed quite well, considering that their 18-1 record was only one win less than 100% predictability. It’s hard to compensate an upset loss with an upset win if a team is not expected to lose. Similarly, South Africa (16-3), Australia (15-4) and Ireland (10-4) achieved winning records one win less than demanding expectations. Italy and the USA were within one win of much less demanding records. Samoa should have had a record of 6-6 but fell to 4-8 due to two upset losses and a -2 balance. Finally, France was by far the greatest underachiever with four upset losses and one upset win for a -3 differential. They should have had a record of 14-5; but instead, had an 11- 8 record, which was well below expectations entering the World Cup.

Overachievement and underachievement by continents
Table 2 contains the data from Table 1, combined by continent. The most upset wins, seven, were earned by European teams which also had the most upset losses, 11. Oceania had the next most upset wins and losses, six and seven respectively. The biggest overachiever was South America (thanks to Argentina). The biggest underachiever was Europe (thanks to France). The most predictable continent was Africa (South Africa and Namibia) where 30 of 31 matches (96.8%) went as expected. The least predicable continent was South America (Argentina and Uruguay) where 18 of 23 matches (78.3%) met expectations.

Table 2. Distribution of the Upset Wins and Upset Losses among the continental federations for the 2007, 2011 and 2015 Rugby World Cups.
 

The Oceania nations, large and small, had the best win-loss record at 47-28, one win less than expected. The Southern Hemisphere had a win-loss record of 60-38, overachieving by four matches. All four semifinalists in 2015 were from the Southern Hemisphere. At the opposite extreme, the Northern Hemisphere had a losing record of 82-104, underachieving by four.

Summing up
The World Rugby rating system proved to be very accurate in that the higher-rated teams, pre-World Cup, won 85.2% of 142 non-tied matches. Nations with losing records were able to pull off upset wins (top panel of Table 1) while dominant nations still had a few upset losses (bottom panel of Table 2). Argentina rose to the demands of the Rugby World Cup by pulling off five upset wins with no upset losses. France was the biggest underachiever with four upset losses and only one upset win. The African content was the most predicable. Namibia, Portugal, Russia and Uruguay met expectations by losing all 24 matches, combined. New Zealand, South Africa, Australia and Ireland also lived up to expectations. The Southern Hemisphere dominated over the Northern Hemisphere in wins versus losses and in overachievement. Let’s reconvene this study in Japan in 2019 where the Brave Blossoms will be the hosts. Look to the World Rugby ratings pre-World Cup to see who is likely to win matches.

• Ray Stefani is an emeritus professor of engineering at the California State University, Long Beach

Reference

1. Stefani, R.T (2015) The Relative Competitive Balance of Male and Female Teams in World Championship Competition and the Relative Predictability of Official International Sports Rating Systems, Proceedings, Mathsport International, Mathsport 2015, Loughborough England, 29 June-1 July, 2015.