When Pope Francis was elected, few of us outside the Roman Catholic Church had heard of him. But from the moment he stepped out onto the balcony to receive the acclaim – and request the prayers – of the faithful gathered in St Peter's Square, he has been the darling of much of the media.

There have been repeated instances of the Pope surprising and wrong-footing reporters, from the profound – washing the feet of Muslim women – to the silly – popping out for pizza.

The previous image of the Papacy as stuffy and distant from ordinary people may not have been deserved. Many of Francis' predecessors were comfortable among the crowds and made interacting with normal people in the pews a high priority. Yet Francis seemingly has a gift – his way with words, his ease with people and his penchant for grand gestures, which indicate more than words ever could.

There has been a group of sceptics from day one, though. Prelates, commentators and theologians from the conservative wing of the Church are wary of Francis and his agenda. They appear to be to be gaining some traction and their voices are becoming louder. While few, if any, are actual recusants, there are an increasing flow of criticisms being published.

Spectator writer Damian Thompson, a long-time Francis sceptic and vocal conservative, has a piece out today. "Why more and more priests can't stand Pope Francis," it declares. The Pontiff's conservative sceptics come from within the Church hierarchy, too. Cardinal Burke, the former Archbishop of St. Louis, has said that the cardinals would issue a "formal act of correction", adding: "It is the duty in such cases, and historically it has happened, of cardinals and bishops to make clear that the Pope is teaching error and to ask him to correct it."

New York Times columnist Ross Douthat has chronicled the Pope's actions in the last few years and has also come out as unconvinced. He writes of Francis' "attention-grabbing breaks with papal protocol, the interventions in global politics, the reopening of moral issues that his predecessors had deemed settled" before observing, "the blend of public humility and skillful exploitation—including the cashiering of opponents—of the papal office and its powers."

These critiques from conservatives around the world are reaching a crescendo because of the document Amoris Laetitia published in April 2016. It's an apostolic exhortation by Pope Francis which focuses on divorce, sexual issues and other topics which were discussed at the Synod on the Family. Late last year, four Cardinals asked the Pope to clarify what he meant in the document, leading to battle lines being drawn – and it's into that uneasy situation that the latest articles are pitched.

But for every negative piece written about Pope, there are many more from secular media which praise him. Liberal press around the world has fallen under his spell, at least to an extent. One example was the Guardian's recent characterisation of him as "a champion of humanity". There are now data which show the Pope's positive impact on the perception of the Church. It isn't just secular liberals who are impressed though. Protestants of many stripes seem to love the Pope.

Here's the real bind that Francis finds himself in. While he seems instinctively to enjoy and thrive on this wide appeal to the progressive wings of his own Church, other Christians and liberal commentators, there is disquiet in his own ranks. The conservative flank of the Roman Catholic Church seems to be increasingly uncomfortable with his leadership.

He can't keep them all happy, so can the Pope continue to walk a tightrope or will he choose to disappoint one of these factions? Paul Vallely, the Pope's biographer and long-time Vatican-watcher, says he hopes Pope Francis can continue to steer "back towards the centre". He adds: "This pope is playing a long game... quietly moving to restore a more moderate tone to the church".

We don't know how long the Pope has left in the job. He is 80 and may decide to follow the precedent of his predecessor and retire. He may, however, feel that there is a way of steering through the arguments and carrying out the reforms he deems necessary, setting a different tone and direction for the Church for generations to come. Those on all sides of the debate, and those inside and outside the Church, wait and wonder.

Follow Andy Walton on Twitter @waltonandy.