The only specialist unit for spine injuries in Wales is to be closed by next April, with care moved to other hospitals.
Rookwood Hospital in Cardiff has 69 rehabilitation beds for patients needing long-term treatment after strokes, spine and brain injuries.
Staff have been told that in the future care will be provided at the Llandough and University Hospital of Wales sites.
Cardiff and Vale Health Board said it hoped better service would be provided.
The overall number of beds is likely to decrease in the the move, with some patients offered care within their communities.
A decision is yet to be taken about the other facilities on the Rookwood site in Llandaff, including a 20-bed centre for geriatric patients.
The health board admits that it is under substantial pressure to save money and that building a new hospital isn't affordable.
But they insist that the changes are part of a wider plan to improve the quality of care.
They say parts of the hospital, which dates back to 1886, are in a poor physical state, but finances are not available to build a new centre.
The news came after hospital staff attended a special meeting to hear plans for the future of rehabilitation care.
Patients say the unit is a valuable resource, providing care for people for many months or even years.
Sarah Phipps, 20, from New Inn, near Pontypool, Torfaen, spent five months recovering at the hospital following a bike accident that nearly killed her.
She said: "I couldn't walk, I couldn't talk, I couldn't eat. They helped me to learn to walk - they had to toilet train me because I'd lost all those muscles.
"There was a bungalow there that helped me a lot - it was like being back home with my family.
"If I hadn't gone there I wouldn't have got this well this quickly. It would have been a struggle and it would have taken a lot longer and it wouldn't have been as good.
"So it was basically everything in my recovery."
Cardiff and Vale University Health Board, like every part of the NHS, is under pressure to make savings.
In a statement released before the meeting, it said: "The service currently provides excellent care for patients with extremely difficult conditions.
"Today's announcement will make sure that staff are able to continue and build on that work.
"Both patients and clinical staff within the service have spoken about things they would like to see changed, including the poor physical state of the hospital and gaps in the service.
"Today's meeting will be the start of a process aimed at developing a better service providing more support for patients and their families, not just in the short term but for many years to come."