Three million new social homes must be built in England over 20 years to solve the "housing crisis", a report says.
Housing charity Shelter said 1.2 million homes are needed for younger families who cannot afford to buy and "face a lifetime in expensive and insecure private renting".
Existing schemes like Help to Buy are a less effective use of taxpayers' money, the report claims.
The government said providing fair social housing was a priority.
It plans to build 250,000 homes by 2022, including homes for social rent.
Shelter said the report was partially inspired by the Grenfell Tower fire and the belief "the disaster must mark a turning point in how we talk about social housing".
Social rent properties are currently set aside for those in most need and are let on long-term tenancies.
Shelter's report claims 1.3 million homes are needed to house those in greatest need - including homeless people and those living with a disability.
The charity estimates 277,000 people are homeless in England, most commonly because they have lost their privately rented homes.
A lack of readily available social housing has led to a "drop in the numbers of young families moving into ownership, the rise of pensioners in insecure unaffordable private rentals, and the homelessness that scars our society", the report found.
Kirsty, 26, lives with her daughters aged five and two. She split from their dad a year ago, and is struggling to afford her privately rented flat in Harlow.
"The first property we moved into was in terrible condition. We could never keep the place warm as it was single-glazed and the doors had holes in."
She and her children spent one month over Christmas without heating.
"My little girl had an operation during that time and she had to come home from hospital to a flat with no heating", she said.
"The landlord just wasn't willing to fix anything. In the end we had to move, which cost me so much money I've now only got £6 in my bank account.
"The kids get the food they need and I eat the leftovers or whatever I can afford.
"I'd love to work but I just can't afford the childcare. It feels like whatever avenue I take I lose at something.
"I'm on the waiting list for social housing but I'm so low down - I just don't know how I'm ever going to be better off.'
Emma and her partner lived in Grenfell Tower for five years, until they lost their home in the fire of June 2017.
"On the night of the fire, we were lucky to get out. The lift stopped at random floors on the way down, picking people up. After we got out at the ground floor, I don't think that lift ever went back up.
"It was only when we got outside that I saw how bad it was."
After seven months in temporary accommodation, Emma is finally in social housing.
"Moving in was a struggle as the flat flooded almost as soon as we got here - trying to get the management to fix that was difficult", she said.
"As social housing tenants, I just feel like we're not listened to. Someone should be held accountable. I want them to understand that it was our lives they were playing with.'
Shandor, 48, became homeless after falling behind on his rent following an accident at work.
After six months in a homeless shelter, he and his eight-year-old son Billy moved into a permanent social home.
"The night before I was evicted, I was given a place in a homeless shelter. There were people there with serious mental health issues", he said.
"I had my son Billy every weekend and it was not a good place for him to be.
"We now have a wonderful little one-bedroom flat, which is our home. The sheer satisfaction of hanging pictures on the wall with picture hooks rather than Blu Tack is incredible.
"Billy is more settled and is starting to make new friends."
The report was authored by 16 independent commissioners, including former Labour leader Ed Miliband, Baroness Sayeeda Warsi, Baroness Doreen Lawrence, TV architect George Clarke and Grenfell survivor Ed Daffarn.
Mr Miliband told BBC Breakfast the proposal would "transform the fabric of the country, the lives of millions of people in the country".
"It is a big sum of money but the problem is, the last labour government didn't do enough, and neither is the current government doing enough.
"In the years after the Second World War, governments - Labour and Conservative - built about 120,000 social homes and council houses every year.
"In the last 20 years or so - we've built 20,000 social homes per year - and that is one of the biggest causes of the housing crisis."
Building 3.1 million new social homes would cost an average of £10.7bn a year, according to analysis by research group Capital Economics on behalf of Shelter.
But Shelter claims the government would save £60bn over 30 years if it can make renting cheaper.
The charity said the housing benefit system was being kept unnecessarily expensive because of more people renting privately at higher costs.
It said having more social housing would lower rents.
It claims the scheme has enabled people on relatively high incomes to buy expensive homes.
Former chair of the Conservative Party, Baroness Warsi, said: "Social mobility has been decimated by decades of political failure to address our worsening housing crisis.
"We simply cannot afford not to act."
The Ministry of Housing, Communities and Local Government said a further £2bn had been committed as part of a 10-year home building programme through to 2028.
"Councils have been given extra freedom to build the social homes their communities need and expect," a government spokesman added.