UK banks have been given until November to agree to a code of practice that they will meet their tax obligations.
Chancellor George Osborne told the BBC that just four of the 15 major banks had so far signed the agreement which was introduced last year.
All banks would be required to agree to "pay what is due both in the spirit of the law as well as to the letter of the law", he said.
The names of those which have signed the code have not been released.
The British Bankers' Association (BBA) said that its members "would continue to work with HM Revenue and Customs on issues relating to tax".
Mr Osborne also said he would introduce legislation this week to bring in a permanent bank levy - which he hopes will raise £2.5bn.
The purpose of the levy is to encourage the banks to take fewer risks in how they fund themselves and will be placed on that part of a bank's balance sheet, which regulators and HMRCbelieve poses a systemic risk.
This levy differs from the previous Chancellor's one-off bonus tax and from a transaction tax which is being looked at separately by the G20 group of nations.
The Labour government introduced the voluntary tax code - stipulating minimum standards of tax compliance - in 2009.
It followed reports of banks participating in large scale tax avoidance schemes using a series of complex transactions and financial instruments.
The code calls on banks to ensure that their tax and the tax obligations of their customers are observed - and that they do not go out of their way to avoid tax for themselves or clients.
"At the best of times tax evasion is unacceptable. At a time like this it is immoral," Mr Osborne told BBC One's Andrew Marr Show.
And the chancellor added funding would be increased for HMRC officers to ensure rich people and companies were not illegally dodging tax.
On the same programme, shadow chancellor Alan Johnson said he believed that a tax on banks should "play a bigger role" in reducing the UK's budget deficit, adding it was "perverse" that more money would be raised from child benefit cuts than from the banks.
He also suggested that Labour would look at increasing capital gains tax.
Labour will unveil its plans for the economy on Monday.