Barry Ferguson has told BBC Scotland his V-sign gesture during a Scotland match against Iceland was targeted at specific individuals in the Scottish Football Association.
"You could put it that way, yes. It annoys me that people say it was to the fans," said Ferguson.
"If I could turn back the clock and do it over, I would never have done it."
However, the former Scotland captain regrets the incident, but feels he was right to end his international career.
The SFA acted after Ferguson and his then-Rangers team-mate, keeper Allan McGregor were dropped to the bench following a drinking session after a defeat by the Netherlands and then made V-signs on the bench in the subsequent match against Iceland.
Ferguson's then-club Rangers stripped him of the captaincy and suspended both him and McGregor for two weeks without pay.
"My proudest moments in my career were the Rangers captaincy and the Scotland captaincy," said Ferguson. "Probably the lowest was that night.
"I knew as soon as I did it, I was in deep trouble.
"I've looked at myself in the mirror a hundred times, 200 times and just wondered why I did it.
"It was a reaction to what happened on the Saturday night after the game in Holland.
"We came back on the Sunday and got in trouble for having a few drinks with a few of the boys. I was disappointed in the way they handled it and I've been open about it.
"The boundaries were that we were allowed to go and have a few drinks - probably, by me being captain, I should have turned round to the boys and held my hands up and said, come on, let's get to bed - but they're old enough and wise enough to make their own decisions.
"That wasn't the problem - the problem was the Wednesday night.
The Blackpool midfielder insists the gesture was never intended to be directed at the fans or the media, and when pressed on the matter, admitted it was aimed at individuals inside the SFA.
"It certainly wasn't the fans," said Ferguson.
"I can't blame the media for doing what they did after what happened on the Sunday for getting into trouble with the Cameron House drinking session - that's a media man's job, to get stories.
"They make some stories up, but it certainly wasn't aimed at the fans for what people had said.
"The simple fact is, if it was to the fans, I would be doing it to 10 or 15 of my mates who go to the games.
"But I can't turn the clock back. That's what happened. Some things happen for a reason - there's anger at others, but I'm more angry at myself for letting myself doing something stupid like that and give people to write about that aren't true."
When Craig Levein took over after George Burley was sacked as manager, the door was opened for Ferguson to return to the national side, but despite several conversations with the new manager, he decided not to return to international competition.
"I was delighted that Craig wanted to speak to me," said Ferguson.
"I spoke to him and Nick Oliver and we had a quick chat. I'm always one to go and ask guys who I respect in the game, who are experienced like Artur Numan and Craig Moore, or my brother and my dad.
"At the end of the day, the decision has to come from me and I just didn't feel it was right.
"Craig came in and has a core of young players coming through, and I think I made the right decision to go away from it and let Scotland rebuild."