The voice records of five million taxpayers are being deleted by the UK's tax authority, as the way they were collected broke privacy rules.
HM Revenue and Customs (HMRC) failed to gain explicit consent from individuals before signing them up to the voice ID system for telephone enquiries.
Campaigners accused HMRC of creating "biometric ID cards by the back door".
The Information Commissioner's Office (ICO) said there had been a "significant" breach of data laws.
"Innovative digital services help make our lives easier but it must not be at the expense of people's fundamental right to privacy," said Steve Wood, deputy commissioner at the ICO.
Now the BBC has learned that the tax authority's system will continue despite the deletions.
Sir Jon Thompson, HMRC chief executive, said: "I am satisfied that HMRC should continue to use voice ID."
"It is popular with our customers, is a more secure way of protecting customer data, and enables us to get callers through to an adviser faster," he said in a letter to HMRC's data protection officer.
How does the system work?
In a bid to speed up the much-criticised HMRC helpline, people were invited to use the voice recognition system, rather than the normal security checks.
The scheme, launched in 2017, asks callers to repeat the phrase "my voice is my password" to register.
Once this task is complete, they can use the phrase to confirm their identity when managing their taxes. HMRC passes the voice through an algorithm to instantly confirm their ID.
Similar projects have been launched by banks and other providers, although they have not always been entirely successful, as this BBC investigation found.
What went wrong?
Privacy campaigner Big Brother Watch complained about the audio signatures system, claiming users were "railroaded" into using it as they were not given the choice to opt out.
The General Data Protection Regulation (GDPR), which came into force across the European Union in May last year, requires organisations to obtain explicit consent before they use biometric data to identify someone, including voice recordings.
HMRC has been told by the UK's Information Commissioner's Office (ICO) that it was not adhering to the data protection rules. In effect, it had automatically pushed people into the system without explicit consent.
The commissioner is issuing the first enforcement notice of its kind to HMRC, under GDPR rules, to ensure the data is deleted. As a result, no fine will be levied.
What happens now?
The tax authority changed the way it sought permission for voice ID in October. Some 1.5 million people have called HMRC since then, and said they wanted to continue using the service. Their records have been retained.
But HMRC has started to delete the voice records of the remaining five million who enrolled into the system before October and who have not called or used the service since.
It said the records would be deleted "well before" the Information Commissioner's deadline of 5 June.
Silkie Carlo, director of Big Brother Watch, said: "This is a massive success for Big Brother Watch, restoring data rights for millions of ordinary people around the country.
"To our knowledge, this is the biggest ever deletion of biometric IDs from a state-held database. This sets a vital precedent for biometrics collection and the database state, showing that campaigners and the ICO have real teeth and no government department is above the law."
Mr Wood, deputy commissioner at the ICO, said: "We welcome HMRC's prompt action to begin deleting personal data that it obtained unlawfully. Our investigation exposed a significant breach of data protection law. HMRC appears to have given little or no consideration to it with regard to its voice ID service."