The number of council-run day care centres in Devon could be cut by nearly 75%, under plans being considered.
Devon County Council said attendances have dropped significantly since 2005, so it is proposing to close some centres, merge others and use more private sector provision.
It insists that everyone who is eligible for council support will receive it.
The proposals are under consultation and no decision has yet been made.
If approved, the number of centres will be reduced from 35 to nine.
The council said the 66% fall in attendance and the reduction in government funding had led it to review how future services were delivered.
In 2012/13, the Oasis Centre in Barnstaple - the council's only dedicated centre for people with physical disabilities - had an occupancy rate of 78% at a cost of £48 per person per day, according to the council.
There had also been no new referrals for three years and so it is one of the centres the council has earmarked for closure.
But Martin Moss, who attends Oasis twice a week, said if it closed it would leave "a big hole" in his life.
"You can talk to people with other disabilities without so-called normal people staring at you and we can be ourselves," the 53-year-old told BBC News.
"It's part of your extended family... it would be absolutely devastating if it closed."
Iris Stoneman, 86, from Hartland, said she had been "depressed and in tears" since learning the Springfield Day Centre in Bideford may also close.
"It used to be such a happy place... but when I walked in last week, I thought 'ye Gods' I could have come to a wake," she said.
The council claims more people are pursuing other hobbies or attending places such as luncheon clubs, while an increasing number were using direct payments from the council to pay for the services they wanted.
The average independent sector costs for day care ranged from £27 to £43 per day, per person, the council said.
Councillor Stuart Barker, cabinet member for adult social care, said nothing was "set in stone", but, with rising costs and lower attendance, the council had to be "realistic".
"We should be prepared to keep centres in places where there is demand and insufficient alternative. But, where there are good private or voluntary sector-run alternatives, we should consider using them more," he said.