The Financial Conduct Authority (FCA) watchdog has warned it will "take out" payday lenders who do not follow stricter new rules.
The FCA is taking over regulation of credit providers and debt management firms from the Office of Fair Trading.
It wants to deal with extortionate charges, and the danger of borrowers being pushed into a cycle of debt.
It is also taking over regulation of credit cards, hire purchase, debt management firms and debt advisers.
"Our processes will probably force about a quarter of the firms out of the industry, and that's a good thing, as those are the ones that have poor practices," Martin Wheatley, chief executive of the FCA, told BBC Radio 5 live.
But driving out payday lenders will not stop demand, Russell Hamblin-Boone from the Consumer Finance Association, the trade body for payday loan firms, told the BBC.
"The demand is there," he said. Without such lenders, people would be forced to take loans from "less reputable" sources, he added.
"With all the scrutiny the industry has been under for 18 months people are continuing to take out these loans and pay them back and not have a problem.
"If we continue to drive up standards this can only be good for business," Mr Hamblin-Boone said.
There have been concerns that payday lenders have been making too much of their profits from people who were struggling to pay the money back.
Typically the lenders offer loans of hundreds of pounds for a few weeks, but at very high annual rates of interest and with high penalties for failing to repay.
The new regime is designed to force them to lend only to those who can afford it.
The FCA faced criticism last week after it released details of an investigation into the insurance industry early, sending shares in insurance firms down. The details were revealed in the Daily Telegraph.
Commons Treasury committee chairman Andrew Tyrie said the leak may have "damaged consumer confidence".
The inquiry will look at pensions, endowments, investment bonds and life assurance policies sold in the UK between the 1970s and 2000.
Mr Wheatley said the "market responded in a way nobody expected".
'Get them to change'
The FCA now has tougher powers than the Office of Fair Trading, including unlimited fines, ordering refunds and banning misleading advertisements.
The FCA has made clear that payday lenders will be closed down if they do not comply with its new rules.
These regulations include limiting the number of times a customer can rollover a loan, improving affordability checks, and controlling attempts to extract repayments from borrowers' bank accounts.
"If we don't like the practices in a firm, the first instance is to get them to change it," said Mr Wheatley.
"If they won't change it, then we take away their permission to operate."
"What we are concerned about are the people who frankly shouldn't be lent to, who can't afford the loans and who then get rolled over and get pushed ever further into a debt cycle," said Mr Wheatley.
The FCA and the Prudential Regulation Authority replaced the Financial Services Authority in regulating the City last year.