Traders are being asked to vote on whether to adopt a business improvement district (BID) after a previous proposal was rejected six years ago.
Critics say they do not want the extra cost and claim the levy would allow councils to pass the cost of discretionary extras such as Christmas lights to the BID in Taunton.
But the Taunton BID steering group said the money could be used for marketing and events to attract visitors.
The ballot will run until 30 March.
The steering group said any BID spending would be in addition to existing budgets or discretionary spending by councils.
But a No Taunton BID spokesman said: "A BID survey came out last November and it found that that only 27% of councils across the UK were actually adhering to that.
"They are not legally binding, they are a guide and can be changed."
The first BID covered 800 businesses and was in place from 2007 to 2012, but the new BID zone has shrunk to cover 440 businesses.
Concerns have also been raised about whether councils should have a vote in the plan, which is designed for businesses.
Chairman of the Taunton BID steering group, Colin Barrell, said Somerset West and Taunton Council was "a £40m plus business with one of the biggest revenues".
"They are contributing a big amount of money and they do stand to benefit," he said.
"They should have a vote because we really want to fill those car parks - we want more people to come to town."
He also said national chains would contribute more money as the tax was dependent on a trader's rateable value.
In effect, Sainsbury's could expect to pay £15,000, but for a smaller trader this would be between £300-£500 a year.