The common use of terms like "tax haven" harms Guernsey's image, the island's finance chief executive said.
It follows criticism after it was revealed the profits from UK NHS hospital PFI schemes were being handled by a Guernsey-based investment bank.
Peter Niven said his group worked tirelessly to promote the island's image despite negative publicity.
He said: "We try to talk to the people making the noise... but we're always working against these kind of terms."
HSBC set up a firm called HICL in Guernsey in a legal tax loophole to handle profits from NHS private finance initiatives. A former MP described it as "scandalous".
But Mr Niven said: "These are investment funds that are being invested in by UK pension funds, UK insurance companies looking for good investments that provide good returns so that they in turn can provide the UK individuals with good pensions and good returns on their insurance policies."
He said: "What Guernsey does is facilitate that. We are tax neutral, that means we are not levying an additional burden on that investment fund.
"So we're not taking away some of the profits, we're allowing it all to flow back into those large investors."
Mr Niven said: "This is the latest in a series of storms and it obviously doesn't do us any good.
"We're out there talking to people to try to get the message [across], there are people with agendas who will never be changed in their views, but we have to reinforce the message time and time again.
"To make sure that people know we are doing good business here, that we're doing legitimate business and that overall we are adding value to the island and, I believe, to the UK economy."