Kenyans tortured by British colonial forces during the Mau Mau uprising will receive payouts totalling £20m, Foreign Secretary William Hague has announced.
He said the UK government recognised Kenyans were tortured and it "sincerely regrets" the abuses that took place.
A lawyer for the victims said they "at last have the recognition and justice they have sought for many years".
Thousands of people were killed during the Mau Mau revolt against British rule in Kenya in the 1950s.
Mr Hague also announced plans to support the construction of a permanent memorial to the victims in the Kenyan capital, Nairobi.
'Pain and grievance'
"I would like to make clear now, and for the first time, on behalf of Her Majesty's government, that we understand the pain and grievance felt by those who were involved in the events of the emergency in Kenya," he told the Commons.
"The British government recognises that Kenyans were subject to torture and other forms of ill-treatment at the hands of the colonial administration.
"The British government sincerely regrets that these abuses took place and that they marred Kenya's progress towards independence."
Mr Hague said 5,228 victims would receive payments totalling £19.9m following an agreement with lawyers acting for the victims, who have been fighting for compensation for a number of years.
The compensation amounts to about £3,000 per victim and applies only to the living survivors of the abuses that took place.
Mr Hague said Britain still did not accept it was legally liable for the actions of what was a colonial administration in Kenya.
Christian Turner, the British High Commissioner to Kenya, also made a statement on the settlement to members of the Mau Mau War Veterans' Association in Nairobi.
Gitu wa Kahengeri, secretary-general of the association, said it was the "beginning of reconciliation between the Mau Mau freedom fighters of Kenya and the British government".
But BBC East Africa correspondent Gabriel Gatehouse said the reaction in Kenya was "muted".
He said it may have been because the announcement had been expected or because it was marred by the UK government continuing to deny liability for some of the abuses.
That said, it was a "big milestone" for the Mau Mau veterans, said our correspondent.
In a statement, Martyn Day, of law firm Leigh Day, said it took "courage to publically acknowledge for the first time the terrible nature of Britain's past in Kenya".
"The elderly victims of torture now at last have the recognition and justice they have sought for many years. For them the significance of this moment cannot be over-emphasised," he said.
Archbishop Desmond Tutu - who backed the case and last year wrote to Prime Minister David Cameron accusing Britain of trying to evade its legal responsibility to the victims - said the settlement was a "balm" for both the victims and perpetrators.
"It sends a signal to the world that no matter how badly human beings behave towards one another, goodness ultimately prevails," said the South African Nobel Peace Prize laureate.
But Bryan Cox, who is representing Tandem Law, said there were "thousands" of further claims that remain unresolved and "the matter was far from over".
He said the law firm was working with more than 8,000 Kenyans who were still awaiting an agreement with the UK government.
The UK has argued that all liabilities for the torture by colonial authorities was transferred to the Kenyan Republic upon independence in 1963 and that it could not be held liable now.
But in 2011, the High Court in London ruled that four claimants did have "arguable cases in law".
Their lawyers allege Paulo Muoka Nzili was castrated, Wambuga Wa Nyingi was severely beaten and Jane Muthoni Mara was subjected to appalling sexual abuse in detention camps during the rebellion. A fourth claimant, Ndiku Mutwiwa Mutua, died last year.
After the ruling, the case went back to the High Court to consider a claim by the UK Foreign and Commonwealth Office (FCO) that the actions had been brought outside the legal time limit.
The FCO said it had faced "irredeemable difficulties" in relation to the availability of witnesses and documents.
But in October last year, the court ruled the victims had established a proper case and allowed their claims to proceed to trial despite the time elapsed.
At the time, victims' lawyer Mr Day said he would be pressing for a trial "as quickly as possible" but would also be pushing for the government to reach an out-of-court settlement.
The Mau Mau, a guerrilla group, began a violent campaign against white settlers in 1952, but the uprising was eventually put down by the British colonial government.
The Kenya Human Rights Commission says 90,000 Kenyans were executed, tortured or maimed, and 160,000 people were detained in appalling conditions, although a number of historians believe the figure is lower.