Most of the inmates at Oxfordshire's Campsfield House immigration removal centre have ended their protest, the UK Border Agency (UKBA) said.
Almost 140 detainees began refusing meals began on Monday. They said their demonstration was due to people being detained for long periods.
The agency said inmates had access to food from vending machines and that there was also an on-site shop.
On Thursday, it confirmed about five people were still refusing sustenance.
Jonathan Sedgwick, the UKBA's deputy chief executive, said: "The vast majority of detainees who were refusing meals have now ended their protest, however a small number continue to decline food prepared by staff.
"The welfare of detainees remains our priority, and we will monitor the small number who are still refusing meals closely.
"The protest has been a peaceful one at all times, and our staff are working with detainees and listening to their concerns."
Mr Sedgwick added that the inmates "continue to have access to legal representation and 24hr medical care".
An earlier statement from the inmates said some people had been detained for more than three years at immigration removal centres across the country with "no prospect of removal or any evidence of future release".
They added that on-site immigration had "hidden away and refused to engage in any communication or dialogue".