Conservatives criticise 'two day a week' Baker
Norman Baker left office with a few passing shots at Theresa May's lack of collegiality, saying he was treated like a "cuckoo in the nest" at the Home Office where it was "like walking through mud".
He said he was resigning to take a break from all this so he could spend some more time with his family and his music.
Not surprisingly some Conservatives have not been slow in painting a different picture.
They suggest that the former transport minister resigned to defend his fragile 7,000 majority in Lewes, a battle which they allege had been claiming more of his time than the Home Office.
One Tory source said: "He was never at the Home Office. He was only there two days a week. He spent most of Mondays in his constituency and then he was away by Thursday morning to prepare for his local radio show."
Mr Baker presents a two hour music programme on Seahaven FM every Thursday evening.
The show is called: "Anything Goes." The station's website says: "When Norman is not presenting on Seahaven FM, he is the Lib Dem MP for the Lewes constituency and a Home Office minister."
The Tory source said that despite spending a limited time at the Home Office each week, Mr Baker still wanted to see everything that was passing through the department, not just the papers relating to his own ministerial responsibilities.
"That put huge pressure on his private office," the source added. "And inevitably that meant he did not see some things. This is the Home Office. Stuff happens all the time and when it happens it is big.
"It is not same as Transport. And he never got on with Theresa. I think he was one of those people who have trouble working for a powerful woman. He did not report to her. He came in and announced he was effectively the Lib Dem home secretary. Not surprisingly, he was told where to go."
The source added: "This is all about him fearing he is going to lose his seat. He may have a majority of 7,000 but that was when the Lib Dems were polling 23%.
"Now they are just creeping above 10%. So that's half his vote gone. And he is close enough to Brighton have some students, they will all have gone. All he needs is a small Labour revival and he could be in real trouble."
'Fall in crime'
The Lib Dems, naturally, dispute this account. They accept that Mr Baker is keen to spend more time in his constituency but they strongly defend his ministerial record.
They point out that as crime prevention minister he has presided over a substantial fall in crime and the drugs report that he outlined last week has begun a genuine debate that could ultimately transform the way the state deals with offenders.
"For one year in the Home Office, that is not a bad record," one Lib Dem source said.
And while few Tories hail his time in the Home Office, some are much more willing to praise his stint as a transport minister.
One Conservative said: "He was good, he had a good grip on policy. He was much more competitive than he seems externally and he could make the system work for him."