My son is an Arsenal fan.

Now, when I say "Arsenal fan", I need to explain a few things. First, he's never actually been to see Arsenal play, either home or away. Indeed, he's never actually been to a professional football game in his entire life. I doubt he could name you more than one member of the squad and if you asked him where they were in the Premiership at the moment he wouldn't have a clue.

If you wanted to find out what Arsenal fans thought of the club at the moment, you wouldn't be looking for my son to give you an insightful view of what the average supporter in the terraces thinks. The only reason my son is an Arsenal fan is because at school his friendship circle includes some much more avid football fanatics, so he decided he'd say he'd make Arsenal his team to fit in. If he had to pick a team he'd pick Arsenal, but really there's no reason why he didn't pick Manchester City or Liverpool instead.

Of course, he didn't pick Manchester United because no-one picks Manchester United these days.

The Ozanne Foundation publicised a repeat survey by YouGov looking at the opinions of a sample of 5,000 people on same-sex marriage. As part of the survey (a normal YouGov internet panel) the participants were asked what religion they associate with, and unsurprisingly around a fifth stated that they were "Church of England", "Anglican" or "Episcopalian". That last answer indicates that the survey included some people in Scotland where the local Anglican denomination is the "Scottish Episcopal Church". The 1,100 or so people who said they were "Anglican" could therefore be assessed for their response to the "do you think same-sex marriage is a good thing" question and a headline therefrom created.

And a headline was created. "Anglicans now in favour of same-sex marriage" crowed the Ozanne Foundation and this story was picked up by mainstream media and religious news sites, despite the fact that once again (because this is the third poll of its kind), the truth is actually slightly more complex. There are in fact a number of issues with the poll and the way it has been reported, issues which have been identified and raised before, but which seem to have been ignored yet again by the revisionist lobby keen to use the results as a political lever for change in the Church of England.

The first issue is whether these "Anglicans" are in any way representative of the men and women (and children) who worship in the pews week after week. With one in five of the sample of 5,000 saying they were Anglican, if these were people who were actually active in their local CofE churches it would indicate around 10 million people worshipping every week. Revival!!!

Of course the truth is different. When people tick the "Anglican" box in this survey they are simply expressing a cultural connection. These are not religiously or spiritually active Christians.  Rather the likelihood is that the overwhelming majority are nominal Christians at best, people who grace our parish churches with their presence at funerals, weddings, baptisms and the occasional Sundays, but who are not on the Electoral Roll, aren't financially supporting the mission of the Church and in all probability don't believe the faith "handed down".

Now there are some very easy ways to find out the views of the people in the pews. The first is to ask a second question of everyone, namely something like "How often do you attend a place of worship for a service?" In this way you could see the difference in support for same-sex marriage between those who attend every week, every month and those who are hardly ever seen on a Sunday. It would be very easy to ask this second question, but despite the fact this has been twice pointed out to Jayne Ozanne, she refused to do so this time.

I wonder why?

The second way to find out the views of the people who actually make up the active membership of the Church of England is to use a specialist religious panel like that delivered by Savanta ComRes. Unlike the general panel that YouGov uses, a specialist religious panel is made up of people who have been qualified for active religious participation. The use of such panels is more expensive (by their nature, trying to find a subset of people with a high level of religious activity takes more effort and is more expensive) but it is a much more effective way of finding out the opinion of the people who really matter.

So on the first issue – representing the people that the press release claims are represented – the poll fails. The official press release uses language like "Church of England supporters" (no attempt is made in the poll to see if the "Anglicans" support the Church of England) and "Church of England members" (the official record of active membership of the Church of England is the Electoral Roll, yet the poll does not ask if "Anglicans" are on it and even if you stretched membership to mean baptised in the Church of England, again "Anglicans" in the poll are not asked if they were baptised in a Church of England parish) but these are deliberately misleading terms.

On a second issue, statistical claims, the poll does not support the confident statements in the press release. The press release talks about a "marked increase" in Anglicans supporting same-sex marriage, but the reality is there is no statistical difference between the 2016 and 2020 figures for supporting same-sex marriage. The difference between 45% (2016) and 48% (2020) is well within the margin of error for the small sub-sample (1171), meaning the increase of 3% might just be random variation based on the particular sample used. This is a basic statistical error and demonstrates that the authors of the press release do not understand how to handle the data that YouGov has generated for them. In fact, a dispassionate observer might note how surprising it is that overall views don't appear to have shifted by any statistically significant amount.

On a third issue, qualification of terms, the poll question and the press release fail. The question asked is "Do you think same-sex marriage is right or wrong", but nowhere is an explanation of what "right" or "wrong" might mean. Right politically? Right morally? Right socially? Is it right because it confers legal guarantees to those who seek them but still morally wrong (but in a liberal society as individuals we accept that some people engage in activities we believe are morally wrong)?

The press release jumps from "English" to "British" without a consideration of the difference in the terms. Indeed, the claim that the poll is about "Church of England supporters" is questionable since the inclusion of "Episcopalian", a term not used in the Church of England but actively used in Scotland, shows that the "Anglicans" in question may actually have affiliations outside of the Church of England. Neither does the poll attempt to qualify those who may attend a Church of England church but have a different religious affiliation.

The press release includes a quote that "To pretend that this is an issue on which many have not yet formed a view is to misunderstand the reality of what is happening in our pews". Again, this is a misrepresentation of who the poll has been conducted on as no-one in the sample has been qualified as to whether they sit in the pews in Church of England churches or elsewhere. Furthermore, when the level of "don't knows" is examined, the percentage for Anglicans (18%, MoE 2.8 on sample of 1259) is statistically significantly different to that of those with no religion (13%, MoE 1.9 on sample of 2667), leading us to be confident in the statement that the "Anglicans" in this survey are more uncertain about what they think compared to those who claim no religious affiliation.

I could go on, but the point is clear – the poll does not represent what the press release claims it does. It is not a reflection of Church of England members in the pews, it does not show any change in support for same-sex marriage in the past four years and it uses terms with little or no qualification in a manner that misleads the reader as to the meaning of the poll. That most of these issues have been pointed out on a previous occasion but have been ignored by the authors demonstrates a deliberate choice to perpetuate these errors for the sake of a political cause.

I close with a challenge to Jayne Ozanne and her self-referential Foundation. As described above, one very easy way to correct these errors would be to ask at least one extra question around church attendance. If Jayne Ozanne were to repeat the exercise, I would happily fund the asking of this extra question, the wording of which would be determined by a neutral third party to the agreement of both parties. My hypothesis is that by looking at church attendance statistics you would see that (a) the majority of these "Anglicans" are not active church members at all and (b) the active church members would hold statistically significantly different views on the subject to the non-church-attending respondents. In fact, this kind of work has been done before, by Mark Regnerus in the States. What he found was that nominal, non-church-attending respondents were indistinguishable from the general population, not only on this issue but on sexual morality more broadly, whilst it was active, church-attending members who held views on all these issues quite out of step with the wider culture. Were the Ozanne Foundation poll to make this kind of enquiry, and find something similar, then it would be significant—but rather awkward.

Proper academic inquiry, including in the area of quantitative study, is open to further information and to clarification and stratification in this manner. It adds to the body of human knowledge, it helps to deepen our understanding of sociological issues. There is no good reason why the Ozanne Foundation should refuse such an offer, unless they were afraid that the results such an extra question would generate would undermine their position, but in the area of academic research that is not a good enough reason not to explore a subject in greater detail.

The challenge is clearly there – the issues with the poll have been on numerous occasions and now a cost free option exists to correct them.

My son is an Arsenal fan, but there are Arsenal fans and there are Arsenal fans. I wouldn't go to my son to find out what the fans in the seats of the Emirates think about Alexandre Lacazette's goal scoring abilities and I wouldn't look to this Ozanne Foundation poll to tell me what the people in the pews of your local parish think about same-sex marriage.

Rev Peter Ould is a Church of England priest based in Canterbury. He works in the field of statistical research and application and writes and broadcasts on issues around the Church, sex and statistics. This article was originally published on the Psephizo blog of Rev Dr Ian Paul and appears here with permission.