A retired Catholic bishop has confessed to a "terrible sense of shame and disgrace" over a failure to act more quickly on allegations of abuse against his priests and nuns.

Bishop Brian Heenan, who led Rockhampton diocese in Queensland, Australia from 1991 until he retired in 2013, admitted to the Royal Commission into Institutional Responses to Child Sexual Abuse that he had failed the children of the Neerkol orphanage.

Former residents had earlier described the terrible sex abuse, along with physical and psychological torture, that they endured from the nuns and priests at the orphanage, which was open as recently as 1978.

When the victims first approached the church for redress, he wrote in a pastoral letter condemning their stories as "scandalous" and "scurrilous". In addition, he did not stop the main culprit from working in the church, even alongside children. Bishop Heenan formally apologised to abuse victims. He has admitted he was concerned to protect the reputation of the Church.

He told the commission that hearing the stories again had brought home to him again what "dreadful" things they had suffered.

"My heart goes out to you ... and I just hope and pray that the experience for you this week has brought you some relief."

Last week, he admitted to failing to protect the children from paedophile priest Reginald Durham.