Lawyers defending Proposition 8, California's gay marriage ban, want the ruling of former U.S. District Judge Vaughn Walker scrapped because he's gay.

Last August, Walker ruled that the 2008 voter-approved law was unconstitutional after presiding over a 13-day trial in San Francisco.

Earlier this month – three months into his retirement from the bench – Walker acknowledged that he's gay and in a ten-year relationship with a physician, confirming for the first time a San Francisco Chronicle report published in February. He told reporters that he never considered his sexual orientation a good reason to recuse himself from the case.

But lawyers for Protect Marriage, the coalition of socially conservative groups defending the law, disagree. In a motion filed on Monday, the group said Walker should have disqualified himself because he is in a same-sex relationship.

“If at any time while this case was pending before him, Chief Judge Walker and his partner determined that they desired, or might desire, to marry, Chief Judge Walker plainly had an interest that could be substantially affected by the outcome of the proceeding,” lawyers wrote.

Andrew Pugno, general counsel for Protect Marriage, also railed against Walker in a statement.

“The American people have a right to a fair judicial process, free from even the appearance of bias or prejudice. Judge Walker's ten-year-long same-sex relationship creates the unavoidable impression that he was not the impartial judge the law requires. He was obligated to either recuse himself or provide full disclosure of this relationship at the outset of the case. These circumstances demand setting aside his decision.”

Chad Griffin, president of the American Foundation for Equal Rights, the group formed specifically to support the case, called the claim “desperate and absurd.”

“They're attacking the judge because they disagree with his decision,” Griffin told