When I pray, what does God do? It's a question of inspired simplicity that forms the title of a book by David Wilkinson (Monarch, £8.99).

If we're honest, we have all asked it at one time or another. We wonder exactly what prayer achieves. Perhaps we've heard prayers for fine weather for the church picnic and thought, how exactly does that work? Will God adjust the path of the jetstream for us? And what about everyone who's really hoping for a bit of rain on their allotments?

Or there's the super-spiritual person who regularly prays for a parking space and finds that God never lets her down. We aren't sure how that works, either – does God just mark it with a sort of spiritual bollard?

But these examples are on the fringe of our worries about prayer. There are plenty of others that are more serious. Week by week we pray for peace, in Iraq or Afghanistan, Syria or Sudan. Things just get worse. Or it comes very close to home: a spouse or a child is ill or in pain. We pray for them to get better and they don't. Healing is perhaps the most intractable of all these problems. The Bible is full of it, it's an article of faith for many churches that God heals, and we really want to believe it, but the actual evidence that it happens in real life is painfully thin.

David Wilkinson is the principal of St John's College, Durham and in a former life was an astrophysicist. That's not irrelevant, because science is based on the idea that you can explain the world. It's ordered and orderly. But if God can overturn the rules of the world he's made, there's a problem. If he can and he chooses not to, there's another problem – and Wilkinson has experienced this in his own life as his wife Alison, a Methodist minister, suffers from painful and debilitating rheumatoid arthritis.

In his book he's honest about his own struggles with prayer and has a refreshing chapter on the 'everyday myths' that we ought to beware of. He warns against the 'slot machine of faith' idea, where if we just pray hard enough God will do what we ask, and the 'health and wealth' heresy that God wants us to be rich.

He also talk about the idea that prayer is just about changing the person who prays, rather than expecting God to do anything. It's this view that stems from the notion that miracles, in which God intervenes in the world to break the natural laws that he has created, are just impossible. The world goes like a clock, and there are no good reasons for expecting God to 'correct the mechanism' through miracles. Wilkinson cites influential philosophers like Voltaire, Spinoza and Hume, whose scepticism led to a 'de-miraclizing' of the New Testament.

So for example, the 'miracle' of the loaves and fishes was really a miracle of sharing, because the example of the little boy who brought Jesus his food inspired others to do the same. And Jesus didn't really walk on water, it was just a sandbank and the disciples were a bit confused.

So, are we just to accept that God doesn't intervene in the world at all, whether through 'miraculous' events or in any other way?

Fascinatingly, Wilkinson says that the problem with this view is that it's based on out of date science – a mechanistic, Newtonian view of the world in which cause and effect can be plotted exactly.

But we now know that the world doesn't operate like that at all. Quantum theory tells us that the small-scale structure of the world is, in the words of Christian physicist John Polkinghorne, "radically random": "By that he means it is unpredictable and nothing like a mechanical clock," says Wilkinson. "It is a world that is unpicturable, uncertain, and in which the cause of events cannot be fully specified."

So, suggests Wilkinson, there's plenty of room for God to act, because the system isn't closed at all. He can "push" electrons here and there and alter the course of events in the world without breaking any of the laws of nature. The problem is that too many theologians simply don't know enough about physics and are stuck with out-of-date science. Quantum theory doesn't answer all our questions, Wilkinson says cautiously, but it "may be one dimension of how God works in the world".

He also writes about 'chaos theory'. Again, this undermines Newton's idea that the world is predictable. It says that most systems in the world – like the weather – are very sensitive to change, so that a small disturbance will make them act very differently. Effectively that means they're unpredictable, a principle known as the 'butterfly effect' after meteorologist Edward Lorenz, who asked in 1979, "Does the flap of a butterfly's wings in Brazil set off a tornado in Texas?"

But could we predict systems if we had a big enough computer? No, says Wilkinson. In a fascinating illustration, he suggests a thought experiment in which a cue ball is struck against the rest of the snooker balls. Predicting where, in the absence of friction, they'd all end up after one minute should be simple enough. But actually the system is chaotic. To predict their positions accurately we'd need to take into account effects as small as the gravitational attraction of an election on the very edge of our galaxy – and a computer bigger than the universe.

This means that at the macro level as well as the micro, the idea that the world is fixed and predictable is just wrong, and that arguments against an interventionist God don't work. So, Wilkinson says, chaos might give "space for God to work in unusual and specific ways within the scientific description of the world". Again, he quotes John Polkinghorne, who says that chaos means that the world is open to the future: "This means that we can pray and God responds by working in the openness of a chaotic system."

His own view is that claiming science rules out miracles is folly.

Prayer is still mysterious. We're still left with questions about unanswered prayers, and with our struggles against laziness and carelessness in prayer. But those are ordinary problems that everyone faces, and there are rich resources to help us in the Bible and in our fellowship with other Christians. Books like this help us to have confidence that whatever prayer is, it isn't a waste of time: God can act in the world and he is able to answer our prayers.

David Wilkinson's book When I Pray, What Does God Do? is available here.

Follow @RevMarkWoods on Twitter.