Christian conservative Bryan Fischer thinks Ben Carson's presidential aspirations are on shaky ground after he apologized for saying that prison can make people gay.

Appearing Wednesday on CNN's New Day, Carson was asked if being gay was a choice.

“Absolutely,” Carson answered. “Because a lot of people go into prison straight and when they come out they're gay.”

(Related: Ben Carson opposed to marriage equality because some inmates turn gay in prison.)

Later in the day, Carson apologized for the remark and vowed he would no longer talk about gay rights.

Fischer told his radio listeners that Carson was sending a troubling message.

“Dr. Carson seems to be not considering the possibility that some people could be offended because he apologized,” Fischer said on Thursday. “They're disappointed in him because he apologized.”

He added that Carson was listening to “the wrong people.”

“He may have done fatal damage to his campaign with his apology last night,” he said. “The bottom line is that on homosexuality and maybe on everything else, Ben Carson needs to decide right now whether he is going to listen to God or his handlers. His career is in the balance.”