At the height of TLC's hit show about the Duggar family, "19 Kids and Counting", it seemed like nothing could split them apart.  But a rift has opened up with daughter Jill Dillard and she's not sure when it might heal. 

Jill and her husband Derek Dillard have been going their separate ways from the family for some time now, and speaking to People magazine, Jill says it was because they couldn't live with the limits on their freedom. 

"Our control to choose what jobs we were allowed to accept and even where we were allowed to live was taken away from us," Jill told People.

Revealing he shares the frustration, Derek said it got to the point where they felt like the restrictions on their lives were stopping them from doing what they really wanted to do. 

Derek said: "The first few years of our marriage, we spent time and money working towards opportunities only to hit a dead end when we'd be told, 'Well, you're not allowed to do that.'"

When they decided to walk away from the show, it "didn't go over very well with anyone", Jill said. But that didn't deter them from quitting. 

"By that point, we'd had enough," she said.

"We knew we had to pull out completely to reevaluate and get our bearings."

Jill and Derek are parents to sons Israel David, 5, and Samuel Scott, 3. 

Since walking away from the show, Jill has embraced her freedom to break some of the Duggar family rules by wearing pants and dresses above the knee, and drinking the occasional alcoholic beverage.  She even has a nose ring. 

She made it clear to People magazine that she doesn't care if the family disapprove of her choices. 

"Sometimes, it's a good thing being okay with other people not being okay," she said. 

As for healing the divisions with her family, she said she was just taking it one day at a time for now. 

"I never expected this to happen or for it to get to this point," she said.

"But I'm realizing I can't put a timeline on healing. I love my family and they love me. I really just have to follow God's lead and take it one day at a time."

Jill's parents, Jim Bob and Michelle Duggar, said they were praying for healing in a statement responding to the Dillards' comments.

"Every family has differences of opinion and perspective at times, but families work things out. We all love Jill, Derick, and their boys very much," they said.

"It is our prayer that our relationship is healed and fully restored quickly!"