Two and three quarter million people could be refunded as much as £2.7bn for being mis-sold Payment Protection Insurance (PPI).
The Financial Services Authority (FSA) has given banks and other lenders until 1 December to adopt new rules for dealing with PPI complaints.
The FSA said that over five years it had found "wide and deep evidence of weaknesses in PPI sales".
PPI insures people's loan re-payments if they fall ill or lose their jobs.
The FSA expects its new rules to force the financial services industry to deal with about 550,000 complaints a year for the next five years.
Average compensation will vary from £900 for those who were mis-sold about regular-premium PPI policies to £1,800 for those mis-sold single-premium policies.
However, a law firm which advises financial companies on regulation said the rules had "no sense of proportionality".
"Firms are receiving thousands of bogus complaints and their right to robustly defend those complaints is now being challenged by FSA," said Paul Edmondson of CMS Cameron McKenna.
A long running campaign by consumer groups such as Citizens Advice and Which? has accused the sellers of PPI in engaging in a widespread "protection racket".
They have accused lenders and others of selling the insurance alongside loans when it was unnecessary, without telling the borrower they were even paying for a policy, or of selling policies on which the borrower could not in fact claim.
The financial services industry has been engaged in a behind-the-scenes campaign to deter the regulator from bringing in the new sanctions, arguing that they are either unnecessary or disproportionate.
But the FSA has finally decided to act.
"Today is the culmination of months of hard work and now, with these measures, we look forward to consumers being treated fairly whether they are buying or complaining about PPI," said Dan Waters of the FSA.
"Since we took over the regulation of PPI we've carried out 24 investigations and three thematic reviews, issued warnings, halted the selling of single premium PPI with unsecured personal loans, visited over 200 firms, and handed out some very significant fines," he added.
There has been a dramatic increase in the number of PPI complaints in the past two to three years, alongside highly critical investigations by the Office of Fair Trading (OFT) and the Competition Commission.
However, the FSA explained that firms had been turning down almost half of the PPI complaints they received, and that some had rejected nearly all their complaints.
About 30% of those people had turned to the Financial Ombudsman Service (FOS) for help, where about 80% of the complaints were then upheld.
"Where complaints are referred to the FOS, the FOS continues to overturn in favour of the consumer the great majority of firms' decisions rejecting the complaints," the FSA said.
"These trends suggest that there is an even greater extent of mis-selling and potential consumer detriment than we had assumed in  and that makes the need and case for an effective approach to addressing such detriment stronger, not weaker," it added.
The key features of the new rules are that if someone complains:
- they should be reimbursed their PPI premiums, plus interest, if the firm decides the customer would not have bought the policy in the first place
- where the premium was a single payment up-front, if the firm concludes the customer would have bought a regular premium policy instead, he or she should be put back in the position they would have been had they done so.
The FOS revealed that it had dealt with more than 100,000 PPI complaints already, with just over 48,000 arriving in the past financial year and an extra 21,000 coming in since 1 April this year.
"We hope to see complaint numbers fall over time as firms deal with them better in the first instance," the FOS said.
The FSA's estimate of how much will have to be reimbursed in total is based on an assumption that some lenders will contact people who may not have complained, but who might still have been sold a policy wrongly.
Last year, firms were told to re-open 185,000 old complaints they had previously rejected.
"For years, the industry has handled poorly thousands of PPI complaints so it's important that the FSA is able to force firms to review old cases," said Which? chief executive, Peter Vicary-Smith.
"We want the government to act swiftly and activate the FSA's power to force lenders to review rejected PPI cases so consumers whose complaints were wrongly dismissed can get the redress they are due," he added.
The Competition Commission is expected shortly to announce its own rules to prohibit the sale of PPI policies at the point when someone is granted a loan.
Last month, Lloyds became the first bank to break ranks with the rest of the industry when it decided to would stop selling PPI to its own borrowers.
HSBC stopped selling PPI polices in November 2007 "across all the HSBC group of companies", a spokesman said.