Northumberland County Football Association official John Cummings has had his appeal against a four-month ban for discriminatory language dismissed.
An FA independent regulatory commission gave the original verdict after Cummings told a female referee "a woman's place is in the kitchen".
He appealed, citing the case of ex-referee David Elleray, who avoided serious censure for offensive remarks.
But an appeal board upheld the ban, and ordered Cummings to pay £1,500 costs.
Initially the independent regulatory commission banned Cummings from all football activities for four months, fined him £250 and ordered him to attend an education programme.
Prior to his appeal Cummings said: "Does it seem fair that I get a four-month ban and a £250 fine and [Elleray] gets a slap on the wrist?
"All I want is a fair punishment."
Cummings, vice-president of the Northumberland County FA, told referee development officer Lucy May at a workshop event in March that "a woman's place is in the kitchen and not on a football field".
The 77-year-old went on to say that "all the time I'm alive, a woman will never referee in my league".
|Northumberland FA's equality policy|
|"The Northumberland FA, in all its activities, will not discriminate, nor treat anyone less favourably, on grounds of gender, sexual orientation, marital status, race, nationality, ethnic origin, religion or belief, ability or disability. The Northumberland FA will ensure that it treats people fairly and with respect and will provide access and opportunities for all members of the community to take part in, and enjoy, its activities."|
Speaking to BBC Radio 5 live's Richard Conway, Cummings said he made the comments to 24-year-old May "after congratulating her on an excellent performance".
"In a conversation that followed I referred to something I said 20 years ago," he said. "I said the words - I'm not going to deny them. I accept that I said it and I was totally wrong.
"I will apologise. To be frank, though, the comments have been used in jest on many occasions. It's been a standing joke for over 20 years - that 'a woman's place is in the home'."
Cummings, who must attend an education programme before he returns to his position, appealed based on the "severity of the judgement in relation to other high-profile cases", specifically that of former Premier League referee Elleray.
Now chairman of the FA's referees' committee, the 60-year-old told Robert McCarthy, a black non-league referee coaching manager, he "looked rather tanned" and asked him "have you been down a coal mine?"
McCarthy did not make a complaint and Elleray subsequently made an unreserved apology.
The FA ruled Elleray breached the FA Council code of conduct, reminded him of his responsibilities and ordered him to attend an equality and diversity training course.