Homosexuality acted as a "lightning conductor" in the power struggle between evangelicals and liberals for dominance in the Church of England, according to the Archbishop of Canterbury Justin Welby.

Some of the quotes are drawn by Atherstone from a conversation recorded a decade ago when the Archbishop was a canon at Coventry Cathedral but were only made public today. Others come from an earlier speech at a New Wine conference. According to the biography, the future Archbishop asked: "How can we go around the world trying to talk about reconciliation...when we don't live it out in our own community?"

Some at the cathedral did not want to invite evangelicals to preach, "because they're homophobic European versions of the Taliban," he is quoted as saying in a new biography out this week. Others refused liberals, "because they don't preach the gospel."

Welby was of the opinion that "we're going to have to take some risks if the cathedral community is going to find a safe place to work out its issues in a reconciled way, not with conflict."

In the recording of the conversation, he made his own position clear, that "sexual practice is for marriage, and marriage is between men and women, and that's the biblical position." Such a view was pastorally difficult, he admitted, "but it's what the Bible says."

Archbishop Welby was a canon at Coventry cathedral, a pioneer in the work of reconciliation and conflict resolution, from 2002 to 2007.

The exclusive account was of a "shared conversation" between Archbishop Welby and Canon Adrian Daffern, his friend and then precentor of Coventry cathedral, in 2004.

Details are published for the first time this week in Andrew Atherstone's new biography, Archbishop Justin Welby: Risk-taker and Reconciler (DLT).

The debate models the "free and friendly" exchange of views the Archbishop has in mind as the Church of England published a new paper on dealing with the conflict over sexuality in advance of next month's General Synod in York, according to Atherstone.

The recording only came to light during the writing of the new biography. Atherstone admits the Archbishop says some "remarkable things".

It came soon after Dr Jeffrey John, a vocal advocate for the blessing of same-sex unions, was nominated in May 2003 as the new Bishop of Reading. He was then forced to stand down by the then Archbishop of Canterbury Dr Rowan Williams after protests from evangelicals.

Atherstone writes: "The Jeffrey John Affair of 2003 created ripples at the cathedral, where a significant number of the congregation were gay. The appointment was greeted with furore. Protest letters rained down, bishops broke ranks and the secular press had a field day. After seven weeks of mounting pressure from across the Anglican Communion, and amidst mutual recriminations, John was forced by his friend Archbishop Rowan Williams to withdraw his acceptance of the position. "

In the recording, Archbishop Welby says the question of right and wrong in the Jeffrey John affair "matters enormously...truth is essential."

Nonetheless, he was perturbed at the manner in which John's nomination as a bishop was debated by the Church: "The public arguing through the columns of the Times, the Telegraph and over the BBC has not helped evangelism... I'm not saying that the issue isn't important, it's just not the right way of doing it."

He said that whatever people might think about the principles at stake, "it cannot be right that the secular press is a substitute for dialogue between Christians, a vitriolic go-between that makes our communication with other people who follow Christ more difficult not more easy."

The "open conversation" took place during a Sunday evening service in January 2004.

The future Archbishop said that in his former parish, Southam, he had only preached once about sex in seven years, "so it's not an obsession... Can the church actually talk about anything else?... There are plenty of other sins that matter... There are a lot of serious issues which, in the words of the New Testament, quench the Holy Spirit of God, which we tend to overlook."

Speaking from "an evangelical tradition", the key issue for Welby was the authority of Scripture. He acknowledged that sometimes the Church had been captive to non-Christian views, foolishly trying to defend slavery or apartheid from the Bible, for example. Nevertheless, he observed that in the great periods of renewal in Christian history, the call was always "back to the Bible".

He said: "The Bible is actually clear that homosexual practice is not permitted, is against the will of God."

Daffern gave the example of a gay couple, friends whom he had introduced to each other, one of them an Anglican clergyman in a parish in south London which was "on any measure blessed by God, a most remarkable ministry, with a thriving church, with amazing things going with young people and the community, and with a passionate, passionate commitment to evangelism, to the gospel of Jesus, and yet they are gay."

Canon Welby was not convinced by this argument from fruitfulness, and replied that he was increasingly conscious that "if the success of my ministry depended on my own good behaviour, I would be a total and complete failure from beginning to end, and nothing would have ever happened at any time in my ministry. The fact is that God's grace works despite human failure and that therefore, it seems to me, to look at a ministry and say 'that's very successful', tells us wonderful things about God's grace but not necessarily wonderful things about the ministers – it may very often, but it doesn't necessarily. It doesn't follow that because the ministry's a success, the ministers are good."

According to Atherstone, this was the inverse of his subsequent controversial aphorism as Archbishop of Canterbury that "where you have a good vicar you will find growing churches." A good vicar might mean a growing church, but according to Canon Welby a growing church did not necessarily mean a good vicar.

He continued: "Pastorally I would love to say, 'It's fine'.... I know gay Christian couples who I respect hugely for their spirituality, and in many ways consider infinitely better people than I am – but what I've also discovered in my life since I was a Christian, despite some real failures of the most desperate kind, is that God knows best... God speaks through Scripture, and if we follow what Scripture says that is the best love that there is. And although we constantly fail to do that, if we at any point say, 'Well, in this particular area, actually because of more recent understandings or whatever, we're going to do something different', that pastorally, however we feel, however hard it is – and I can't begin to describe how difficult it is to say this – however hard it is, we will be letting down the people that we are dealing with pastorally, because God knows best and the Scripture in the end is clear."

He spoke not only of the pain of gay people within the Coventry Cathedral community, but also the pain experienced by conservative Anglicans in Africa. For example he had attended an ordination service in Nigeria in the summer of 2003 at which the bishop described developments among Anglicans in North America and the United Kingdom. Canon Welby remembered: "I've never heard 2,000 people groan in pain before. There was a gasp." Speaking to his own cathedral congregation he concluded: "I see and I feel the pain of fellow members of this community. But I come back to the fact that I believe at the bottom of my heart that what the Bible says is true...genital sex between people of the same gender is not permitted by Scripture."

Canon Daffern said: "It's a wonderful thing in this cathedral that we have a team of clergy who are passionate about their love for Jesus, and their longing to see this place be alive for God, and yet within the team we have our differences on some issues. And I think that makes us stronger not weaker, because we stay together, we pray together, we study Scripture together, we love each other... It's amazing what God is doing in us."