The first same-sex marriages in the UK have not been universally accepted. So says a BBC Radio 5 Live survey of 1,007 people, which leads with the headline 'Gay weddings: Fifth of Britons would turn down invitation'. My question is, who are the fifth?

Let's ignore the issue of whether the respondents are representative of the population as a whole. Certainly we would expect the type of people who respond to a BBC Radio 5 survey to not adequately represent the whole UK population both in age and gender, as well as the range of political and religious beliefs. I also want to ignore the form of the questions, how they were asked, and how many questions were on the complete survey.

These are important issues for deciding whether the survey results have any national importance, and a strong argument can be made that one thousand people, even when re-weighted, do not speak for the whole UK population of 63 million.

What is most interesting is trying to understand what type of people disagree with or would not attend a same-sex marriage. Within the article a range of statistics are reported, but as you will see they might not give you all the information you want – not enough detail is reported.

The key concept is something called cross-tabulation, arranging the responses (the yes or no to attending) by multiple features. So you do not report percentages just by age or sex, but by all combinations of age and sex together.

The survey of 1,007 people reports that '22% said they would spurn an invitation to attend a same-sex wedding ceremony'. This gives us one summary number (aka a statistic) called a marginal percentage, let's see why when we reconstruct the following table.

We know the total number of people surveyed and the overall percentage that would not attend. This is in the margin of the table on the left, hence a marginal percentage:

 Sex: Female Male Total Attend: Yes No Total 1,007 Sex: Female Male Total Attend: Yes 78% No 22% Total 1,007 Sex: Female Male Total Attend: Yes 785 No 222 Total 1,007

We are not told what percentage of responders were men directly, so we could guess it was 50% – if the survey was representing the whole population.

In fact we do not need to guess at the percentage of responders since we're also given the percentages of people who would not attend by sex, '29% of men asked said they would not attend, compared with 16% of women'. Using this we can work out what the table might look like (this might not exactly match the real data, since we've had to round to whole numbers):

 Sex: Female Male Total Attend: Yes 455 330 785 No 87 135 222 Total 542 465 1,007

Hence 46% (465/1007) of the responders were men. Although it has taken some work, I have been able to reconstruct the table so far and see the actual numbers.

It was also reported that 'younger people were more likely to support same-sex marriage, with 80% of 18 to 34-year-olds backing it, compared with 44% of over-65s'. Importantly these percentages are for a different question, younger people supporting same-sex marriage does not automatically lead to younger people attending a ceremony, nor do we know the relationship between supporting and attending. Why now switch to reporting a different question? And what about us 35 to 64-year-olds?

At this point I have no hope of reconstructing the table of who would not attend by sex and age, there is not enough information about the 1,007 surveyed. So how many older men or women would not attend is left a mystery. To better understand this survey we need the actual numbers that are missing from the table below.

 Sex: Female Male Total Age: 18-34 35-64 65+ 18-34 35-64 65+ Attend: Yes 785 No 222 Total 1,007

We have an unclear picture of who would not attend, and an even vaguer idea of who supports same-sex marriage. By who, we mean people counted by age and by sex. This is important for anyone who wants to understand the changing attitudes of the population over time, and campaigners and policy makers trying to influence public opinion. As is commonly noted, older people are more likely to vote than younger people.

I will highlight a single comment by Robert Pigott in the article, 'Most religious groups remain opposed – and Radio 5 Live's poll shows they are not alone in deeply held objections to gay marriage'. About 75% of the UK population reported belonging to a religion, so is that different to belonging to a religious group? Can we further divide people by being in a religious group, having a religion but not being in a religious group, and not having a religion – how does that change with age and sex? If the fifth of people who would not attend are also members of a religious group, then they are alone in their objections to same-sex marriage – we cannot say from the survey as reported.

It is clear that one thousand people are really not enough to reflect the views of the nation, but really we want to know how different groups of people feel. In our last table we have divided everyone into twelve groups: by male/female, by 18-34/35-64/65+, and by would they attend a same-sex wedding or not.

Perhaps as few as 100 people are deciding how certain groups of the population – and by that we mean you, the reader – feel about same-sex marriage, and we're no closer to finding out how many older men and women who are religious (but not part of a religious group) would not attend!

Significance Magazine