The average indie rock album doesn't start with an exploration of faith, doubt and loss. But then Craig Finn doesn't make average indie rock albums. For the best part of 15 years, he's made music which demands a close listening ear. There's some straight up good time rock and roll for sure, but there's so much more than that. His characters often experience the full range of human emotions in a four-minute song.

In a music scene where the default belief systems are atheism or a vague spirituality, Finn stands out (along with the likes of Sufjan Stevens) for his use of Christian symbolism in his music – something drawn from his own upbringing in churches.

As the front man of the acclaimed band The Hold Steady, he's developed a great relationship with fans. At the end of most gigs he tells the crowd that there is, "So. Much. Joy!" in what they do – and you believe him. This isn't a marketing trick – it's a man revelling in what he was born to do. The tight knit community that's formed with the band was illustrated a couple of years ago, when fans rallied round the family of one of the their number who'd died young and worked together with the band to raise funds to support them.

Finn's second solo record begins with that song exploring faith, doubt and loss called Maggie, I've Been Searching For Our Son. In it he muses, "If you're down and out/If you still have any doubts/If you doubt that there's a savior yet to come/Stop doing what you're doing/Close your eyes and keep on breathing/And slowly turn your face up to the sun."

This is typical of Finn's work – both his solo recordings and his six albums with The Hold Steady. He's no evangelist, or preacher for that matter. He deals with the edges of life. People who are down on their luck. People who have seen the dark side of the American Dream, but have sometimes caught glimpses of hope and faith.

His first solo album Clear Eyes Full Heart came out in 2012 and had a loose, southern groove. After working on The Hold Steady's latest album, 2014's Teeth Dreams, he returns this year with his second solo offering. I sat down to talk to him about the new album, the songwriting process, and why he spends a lot of time thinking about Jesus...

What's the idea behind the new record?
It's called Faith in the Future. When I looked back, a lot of the songs were written in a period – my mother passed away in April 2013 and when I got back from Minneapolis, where she was (I was with her through all of it) I got back to Brooklyn where I live and in my grief I was having a hard time getting motivated; I wasn't really doing much and I got concerned. So I thought, 'I need something to work on' so I just started writing every day, really on a schedule and trying to make songs every day. None of them address the loss of my mother in any way but when I got the whole batch of songs, and started to look back at what I'd written in that month or so, I realised a lot of the songs were about people persevering after change or tragedy – that seemed to be a recurring theme on the record. That's the idea behind the title – Faith in the Future.

Is that how you usually work, you don't have a grand vision, but you look back in hindsight and see a theme has developed?
It's about going to work: sitting down and writing the songs and then looking back and seeing what it reveals to you. I think Joan Didion said that we write to find out what we're thinking about and I believe that's very much true... The goal of this record really was to keep the narratives up front and not interfere with them so we made a lot of decisions to keep the songs fairly unadorned and try to just let the stories come through.

Stories have always been primary in your work. Are you happy for us to decipher our own meanings too?
Absolutely. I always think it's like a short story, it's [John] Cheever, [Raymond] Carver – the tensions they create with the details they give you and what they leave out. I think it's important for a song to have some space – even if you're telling a story – for the listener to insert their own hopes and dreams... With the solo stuff it's a little quieter and I think it allows me to be a bit more vulnerable and a little more personal, so I can write about maybe more mundane things, smaller moments, and let it get through. If I write about sitting at a table drinking coffee, talking to someone, that's just not going to make a very good Hold Steady song.

One of the main themes of your writing is Christian faith. How does that work its way into your lyrics?
Being raised in a Catholic church... [my] parents [left] some of the big questions about morality... to the Church – big concepts of right and wrong and big concepts like forgiveness – kind of deferring to the Church, within the family. So when I think of some of these things from a Catholic childhood, I immediately go there [to Christian imagery]. I think of Christ and the way he would treat people, and where that works in the songs. A lot of times I tend to write about desperate characters and that's obviously who Christ spent a lot of time around. I've just always been fascinated by that.

It seems like you're living with that upbringing, like even if you wanted to, you can't get rid of that imagery?
Yeah. My adult life has not been all super-Christian or super-holy so, it's kind-of going back to that always, and it's almost like an ongoing struggle to remain comfortable with having it there and still live the life I want to live.

Would you self-describe as a Christian?
I do, I do. I think about Christ and how I want to treat people and my politics, things like that. I'm a Catholic and when I go to church, which is about half of Sundays, obviously there's things I have problems with but I think most of those problems have to do with the humans involved rather than the actual text.

It's an interesting time with Pope Francis about to visit the US...
Yeah, what I really like about him is that, it seems that what we can do, the Catholic Church can really make an impact on is poverty – to minister to those people. He seems committed to that, and that's important. Way more important than making a lot of weird rules that people have a hard time following. So that's exciting to me. I also have to say that I was excited that Benedict actually stepped down – that was kind of amazing, the first guy to step down in quite some time.

Themes of grace and forgiveness aren't exclusive to Christians, but artists from a Christian background, maybe particularly a Catholic background – Graham Greene, Bruce Springsteen – there seems to be a rich vein that you're all mining.
Yeah, I think some of it's the fire and brimstone, the pomp and circumstance of the Catholic Mass. In this ritual, there's beauty. I was talking to my priest the other day, back in Minneapolis at my nephew's baptism, and he was talking about how much literary inspiration there is in the stories of the saints. You can really blow your mind by opening up a book about saints and reading! It's so rich. For instance, on this record, I was reading when Saint Peter, Simon Peter was crucified he requested, out of respect for Christ to be crucified upside down. I thought, "that's interesting" and that turned into Saint Peter Upside Down, a song on the record. There's a literary richness to it all that you draw on and is fascinating – it's heavy and it's attractive in its heaviness, to me.

Do you see writing about faith as a career-long exploration for you or do you sometimes just want to write songs about baseball?
I feel like there will always be a struggle – there always has been – you write and you figure out what you're thinking. I'm pretty sure, whether I like it or not this is what I'll be thinking about 'til my grave. So probably as long as I write songs it will appear in there. Some days you just want to write a song about baseball, but it's part of my internal thing.

Aging is another theme in your music and mortality as well...
Yeah and in some ways it's even more literal than that. I have a song on the record about 9/11 about being on my friend's roof, watching the towers go down, thinking, this is all pretty awful. Some years later I meet the girl that I fell in love with... and she was on the 34th floor of the tower and went out against orders – they told her to stay where she was and she left. So I'm watching the towers go down and what I don't know is that somewhere there's something way better waiting for me... It's not necessarily about death but it's about thinking things are better up ahead.

The record's called Faith in the Future. Someone might pick it up and think "This is going to be a positive thing" but then the opening line is, "There's a darkness in my body..."
I think I'm always playing with putting dark against light. So even the album cover is kind of a dark image of a guy walking through a liquor store parking lot. I think I've always played with that – putting a chorus that's maybe a little sad against big major key changes.

The songs that stay with us aren't just straightforwardly happy are they?
They at least acknowledge the darkness. The Hold Steady song Stay Positive says, "You gotta Stay Positive" but the only time you say "stay positive" is when things are going kind of [bad]. That's kind of like Faith in the Future in some way – the times you need to have faith in the future is when things are not happening.

When you put a record out like this do you have goals in mind?
I certainly want people to connect with the songs – that's the whole point. As an audience member, a listener, a reader when I have a book or band or song or movie, and I say "Hey, I've felt that same way – I know what this guy's talking about." Making a connection with the audience is ultimately the goal.

I saw recently that you had a guy propose to his girlfriend during a show...
That was really cool. He really did it right. I knew her and he got her phone and stole my number out of her phone and contacted me. It was in Chicago, it wasn't where she lives in Ohio, so we hid him. It was really well executed and very fun... It was very sweet and it was very pure – I think I'm gonna have to go to the wedding!

Faith in the Future is out now on Partisan Records.