"Don't shove your religion down my throat." We've probably all heard this comment, or similar, when discussing our faith with others. In terms of inter-personal relationships, it's obviously a good idea not to try to force ideas on anyone.

Yet, when this thinking gets translated into the public square, it becomes strangely muddled. The argument goes something like this, "We all believe different things. Therefore it's best for religion to stay out of politics and public life and it must be restricted to private life."

There was a perfect example this week. With the UK Government trying to deregulate Sunday trading times, I wrote a strongly worded piece about why I (and churches, trade unions, small shopkeepers and others) opposed the changes. In response, I had various tweets saying, broadly, that the church needed to mind its own business.

"Respectfully, I find it equally insane and outdated when religion tries to influence business law," said one tweet. "This feels like imposition of a religious rule on secular society," said another.

A whole response blog was penned, in this case by a conservative Christian, who claimed those of us who want to maintain a small element of Sunday being special were guilty of patronising nonsense; caricaturing my position as "We must make people realise that they cannot be trusted to know what is in their best interests; only we selfless Christians can enlighten them."

The theme running through all these responses is one of the most pervasive canards of contemporary life: the myth of secular neutrality.

Basically, goes the argument, religious views aren't based on facts, they're divisive, they're reactionary, they're dangerous. Therefore, the answer is to have a completely secularised state, political life and public sphere.

It's a superficially sound argument. We live in a world where radical Islamist regimes like Saudi Arabia systematically deny human rights, where Hindu nationalists in India persecute minorities and where the deified dictators of North Korea allow no deviation from their diktats.

Secularism developed during the Enlightenment in response to the abusive excesses of the theocrats of the day. The collusion of the Church with repressive states across Europe meant that there was a strong argument for some secularisation.

Secularism, though, isn't a panacea. You only have to look at the route that France went down in the aftermath of extreme secularisation to realise that. The Reign of Terror which followed the Revolution was as savage as anything the Church had sanctioned.

The problem with secularism being seen as 'neutral' is that there's no such thing as neutral in public life. This viewpoint assumes we're all born as clean slates and that those who manage to avoid 'undue religious influence' are free of taint, but those of us who have faith commitments are uniquely compromised.

Of course my faith informs my views. How could it not? Along with my nationality, life experiences, upbringing, class, place of birth and many other factors, it plays a role in determining how I think about socio-economic and political issues. But the myth of secular neutrality suggests that all the other factors which play into my views are valid as determiners, but faith is not.

The myth claims religious belief is uniquely divisive; that someone who makes a political decision informed by their faith is doing some uniquely irrational and this must be excluded from the political process. Why?! Why is religious belief uniquely divisive? We'd never dream of excluding public school educated politicians' views, simply on the basis that they paid for their education. Yet such an upbringing will undoubtedly have had an impact on that politician's views.

Similarly, at least in theory, we don't discriminate on the basis of age, race, class, gender, disability, sexual orientation or other forms of difference in public life. Though these core parts of our identities have a vast influence on what we bring to the discussion in the public sphere, we cannot and must not leave them at the door. Why, then, are we sometimes told that faith must be left at the door?

Let's be clear about what I'm not arguing for here. I don't think we should have some kind of Reconstructionalist theocracy (directly taking laws out of the Hebrew Bible into 21st century politics). Human rights should be universal – and the rule of law is non-negotiable.

Second, I'm not suggesting that just because someone's viewpoint is 'religious' it needs to be 'protected' or not scrutinised. People of faith need to be subject to the same scrutiny as others. Indeed there are some 'religions' which need a vastly increased level of scrutiny (yes Scientology, I'm looking at you.)

Third, it should go without saying that people of faith need to make their arguments in terms of the common good, not just blindly assert 'because the Bible says so'. So, in the example of Sunday trading, I'm not arguing we should restrict it because I want to force everyone to go to church. We should restrict it because Sabbath is a common good concept – a shared day of rest benefits the whole of society – believers, and nonbelievers, Christians and anti-theists.

Our aim shouldn't be to seek power for the church, instead, we should use our influence to seek justice for all.

We have to get away from the idea that faith is inherently divisive and the only answer is strict secularism. That's just the imposition of another worldview (secular humanism) on everyone else. The risk is we end up with a situation like in France where freedom of expression is now denied to religious people. So much for secular tolerance. Instead we should work for a mature, pluralist democracy where expression of faith is fully allowed, but there's complete freedom not to believe.

The Church and other faith groups need to engage in a grown-up, responsible and democratic way. But when we do, we mustn't be told that our viewpoint isn't valid. In the UK and the US, the Church remains the biggest civil society movement. For all its faults, that means that we deserve a voice in the public square – neither privileged nor private – in the words of one important enquiry into this subject. The sooner we get over the idea that faith is uniquely divisive and the only option is enforced and deadening hyper-secularism, the sooner we can get on with marshaling the forces of the church and other faith groups for the common good.

Follow Andy Walton on Twitter @waltonandy