Six Nations rugby: increased predictability (despite England upsets)

Rugby match

With the Six Nations rugby championship kicking off this weekend, we have two years of results to add to our understanding of the competition, following my 2017 article, “6 Nations Rugby – who’s the biggest overachiever?”. The six nations involved are England, France, Ireland, Italy, Scotland and Wales. We will look for recent changes in the ratings-based predictability of the teams in general and for recent changes in the distribution of upsets in particular, when ratings-based predictions proved to be wrong.

A rating system is used to account for home advantage and to determine the team most likely to win each game. The International Rugby Board, now called World Rugby, created the system, and it adjusts the ratings of each team after each match so as to optimize future predictive accuracy.

For a given game, the probability that the home team will win is estimated to be 0.50 plus (home team’s rating +3, less the visiting team’s rating) /20. Suppose that Ireland, with a rating of 85, plays host to France, which has a rating of 87. Ireland would gain three rating points for playing at home and thus would have a higher adjusted rating by a margin of 88-87, establishing Ireland as the ratings-based favourite. Ireland would be predicted to win with a probability of 0.5 + (85 + 3 – 87) / 20 or 0.55.

The three-point home advantage accounts for the physical advantage due to the visitors’ possibly experiencing travel fatigue, the psychological advantage of having the vociferous support of the home fans, and the tactical advantage of familiarity with the venue, which could aid kicking accuracy and enable more consistent performance. If both teams have the same rating, the home team is therefore expected to win 0.50 + 3/20, or 65% of the games.

For each rating point a team gains or loses, the probability of winning increases or decreases by 1/20 or 5%. After each game, ratings are adjusted depending on game importance and the difference between the actual result (on a 0-1 basis) and the predicted probability. The team that did better than expected gains rating points and the team that performed worse than expected loses the same number of rating points, forming a point-exchange system.

We begin by testing this use of three rating points for home advantage. We ask: does the average home nation actually win about 65% of Six Nations games?

Home team advantage

In each Six Nations competition, 15 games are played. Each team plays the other five opponents, but obviously not with the same number of home and away games. The following year, the home/away schedule flips. Therefore, over a two-year period, a balanced double round-robin occurs for each team, in which each of the five opponents is played once at home and once away.

The “Home win” column in Table 1 summarizes the fraction of games won by home teams in games that were not tied. In 2011 the home teams won 60% of non-tied games, and in 2012 they won 57.1%, a small differential of 2.9 percentage points. In 2015, visiting teams were particularly strong, leaving home teams with only 46.7% of the wins. When those strong teams played at home the following year, 2016, they won 71.4% of the non-tied games, creating a 24.7-point differential. The home teams had exactly the same success rate in 2017 and 2018: 73%.

Home team advantage is much better behaved when pairs of years are combined. Over several pairs, we expect the average home-team rating to approach the average visitor rating, thus creating a meaningful test of the three-point home advantage. Over all eight years, with ties omitted, the home nation won very close to the expected 65% figure (winning 64.1%). Our conclusion is therefore that the three-point home advantage correction is indeed quite accurate.

TABLE 1 Success of the home team for Six Nations rugby, 2011-2018.

G = games played, W = wins, L = losses, T = ties
Year	G	W	L	T	Home win (no ties)	Combined years	G	W	L	T	Home win (no ties)
2011	15	9	6	0	0.600
2012	15	8	6	1	0.571	2011-12	30	17	12	1	0.586
2013	15	9	5	1	0.643
2014	15	10	5	0	0.667	2013-14	30	19	10	1	0.655
2015	15	7	8	0	0.467
2016	15	10	4	1	0.714	2015-16	30	17	12	1	0.586
2017	15	11	4	0	0.733
2018	15	11	4	0	0.733	2017-18	30	22	8	0	0.733
All	120	75	42	3	0.641

Accuracy of predictions

We now turn our attention to the accuracy of predictions made by World Rugby’s ratings system. Recall that the system allows us to calculate a win probability for each team in a two-team match up, with the favourite being the team with the highest win probability.

Table 2 reports how often the favourite won each Six Nations game over eight years of competitions. Accuracy rates (in the “Favourite won” column) are calculated for non-tied games only, as was done for home advantage.

Successive years have exhibited as much as a 43.8-point differential (2013-2014) in accuracy rates, but the two-year combined data are much better behaved. Notice that the last three two-year combinations have shown monotonically increasing predictive accuracy, from 65.5% to 79.3% to 83.3%. In fact, 2017 (with 86.7% accuracy) is tied with 2014 for most predictable year, while 2018 (with 80.0% accuracy) is tied for third most predictable year.

TABLE 2 Success of the favourite for Six Nations rugby, 2011-2018.

G = games played, W = wins, L = losses, T = ties
Year	G	W	L	T	Favourite won (no ties)	Combined years	G	W	L	T	Favourite won (no ties)
2011	15	10	5	0	0.667
2012	15	10	4	1	0.714	2011-12	30	20	9	1	0.690
2013	15	6	8	1	0.429
2014	15	13	2	0	0.867	2013-14	30	19	10	1	0.655
2015	15	12	3	0	0.800
2016	15	11	3	1	0.786	2015-16	30	23	6	1	0.793
2017	15	13	2	0	0.867
2018	15	12	3	0	0.800	2017-18	30	25	5	0	0.833
All	120	87	30	3	0.743

Upset winners and losers

Over eight years and 120 games, 87 matches were won by the favourite, three were tied, and 30 could be considered “upset” games – that is, the favourite lost. But which nations have been involved in these upsets?

The left portion of Table 3 summarizes head-to-head records of actual game outcomes, rather than ratings-based predictions. In Table 3, England, Wales and Ireland had the most successful records – winning more games than they lost or tied; for France, Scotland and Italy, it is a different story.

Turning to the 30 “upset” games in the second half of the table, we see that Wales has been involved in 13 such games, with 11 upset wins against 2 upset losses. Welsh teams have come to play and left with a positive 9-game differential. Italy is the only other team with more upset wins than losses, but almost every win by Italy has been an upset win as Italy has almost always been predicted to lose its matches. Italy was the sixth-ranked team every year except 2013, when they were ranked fifth.

TABLE 3 Team overall wins, losses and ties, 2011-2018, along with upset wins and losses.

G = games played, W = wins, L = losses, T = ties
Nation	G	W	L	T	Fraction won (no ties)	Total upset wins	Total upset losses	Upset wins 2017-18	Upset losses 2017-18
England	40	31	9	0	0.775	4	7	0	4
Wales	40	27	12	1	0.692	11	2	0	0
Ireland	40	24	13	3	0.649	5	7	2	1
France	40	18	20	2	0.474	3	8	2	0
Scotland	40	12	28	0	0.300	3	5	1	0
Italy	40	5	35	0	0.125	4	1	0	0

Looking at the five most recent upsets, for the years 2017 and 2018, we see that Wales and Italy were involved in no upsets. Scotland, France and Ireland were involved in one, two and three upsets, respectively, admirably scoring more upset wins than losses. Then we come to England, and its uncharacteristically poor play in 2017-18, leading to four upset losses. England placed fifth in 2018. Conclusion: in 2017-18, teams have been more consistently predictable based on pre-competition ratings, except for England.

Now, what can we say about the 2019 competition?

The 2019 Six Nations competition

Here are the ordinal rankings and World Rugby ratings of the six teams, after the last pre-competition adjustment on 6 January 2019.

Ireland (91.17)
Wales (87.24)
England (86.22)
Scotland (81.84)
France (77.33)
Italy (72.75)

Ireland and Wales are ranked quite well, considering that Ireland has not been ranked first in the last eight years, while Wales were ranked lower than second for six of the last eight years. Conversely, England had ranked first for five of the last eight years, but dropped to third for 2019, partially due to the four Six Nations upset losses in 2017-18.

After adding three rating points to each home team rating, the predicted wins are: Ireland (5), Wales (4), England (3), Scotland (2), France (1) and Italy (0). The champion may be determined in round 5, when Wales (rated 90.24 including home advantage) plays host to Ireland (91.17). The World Rugby System favours Ireland with a probability of 54.6 %. Considering that Wales have earned 11 upset wins in eight years and are playing at home, Wales could go top in what ought to be an exciting and well-played game; so, well worth watching. England must score upset wins (as per World Rugby ratings) to become competitive.

About the author

Ray Stefani is an emeritus professor of engineering at the California State University, Long Beach.

Tags: