Ken Clarke has promised to end the rise in prisoner numbers and "break the cycle" of crime by tackling the causes of reoffending.
The justice secretary plans more help to deal with inmates' drug and alcohol addiction problems and mental illness.
His Green Paper on sentencing in England and Wales aims to cut the 85,000 inmate population by 3,000.
But Labour said ministers were trying to reduce prisoner numbers "in order to cut costs".
The government wants to reduce the £4bn prison and probation budget by 20% over four years.
The Green Paper puts the emphasis on making prisons in England and Wales more purposeful and toughening up community sentences.
Launching the paper in the Commons, Mr Clarke said the present criminal justice system "falls short of what is required" - with around half of those released from prison going on to reoffend within a year.
'Fewer crimes, fewer victims'
The Green Paper marked a change in policy putting "more emphasis on reducing reoffending without reducing the punishment of offenders", he said.
"By reforming criminals and turning them away from a life of crime we will break the cycle. This will mean fewer crimes, fewer victims and safer communities."
The BBC home affairs correspondent Danny Shaw said the biggest reduction in prison numbers was likely to come from a proposal to halve sentences for those who plead guilty as early as possible.
Other plans to reduce prison numbers include allowing some foreign criminals to leave Britain for good instead of going to prison and to give judges more discretion over sentencing in murder cases - although it says there is no intention to scrap the mandatory life sentences.
Mr Clarke said current legislation attempted to analyse "each and every sort of murder largely by the method used by the murder". "We're going to have to put a simpler, more sensible alternative in its place. The motive of the murderer should have some bearing on how long he gets," he said.
He said prison should be "first and foremost" a place of punishment - but it could not continue to be "simply an expensive way of giving communities a break".
But he was challenged by fellow Tory MP Edward Leigh who told him: "Communities deserve a break - they deserve a break from being burgled."
Mr Clarke told him no-one was suggesting burglars should not get prison terms.
Other proposals include a full working week for prison inmates, more "demanding" community sentences and tougher curfews, diverting more criminals with drug and alcohol problems into community treatment schemes and releasing more defendants on bail if they are unlikely to be jailed for their crime.
Mr Clarke said there would be a "revolutionary shift in the way rehabilitation is financed and delivered" - including at least six new projects over the next two years, which would pay the companies running them "by results".
Controversial indefinite sentences for the public protection - where people are sentenced to a minimum jail term but must satisfy authorities they are fit for release - would be restricted to only the most serious offenders - those who would have received at least a ten-year sentence.
Sue Hall, chair of the Probation Chiefs Association, said: "The best way to protect the public and prevent the next victim is to focus on rehabilitating offenders. We know this works and will break the cycle of reoffending for many of the 85,000 people in prison now."
And Juliet Lyon, Director of the Prison Reform Trust, said: "The green paper is a blueprint for moderate and sensible reform and should mark the end of sterile debate on toughness or softness on crime. Rather than settling for policy-making on the hoof or enduring a crisis-driven justice system, Ken Clarke has opened a proper consultation on sentencing and rehabilitation based on evidence of what works."
Plans in the Conservative manifesto for prisoners to "earn" their release, as part of a so-called "Min-Max" sentencing regime, have been scrapped, as has the pledge that anyone convicted of carrying a knife should get a jail term.
Mr Clarke said he wanted to leave it to judges to decide on sentences although he told MPs he would be "shocked" if someone convicted of stabbing someone did not get a prison sentence.
But Shadow Justice Secretary Sadiq Khan said it was a "humiliating U-turn" by the Conservatives and meant pre-election promises to be tough on crime had been abandoned.
He added: "Sentencing policy should be about dealing with offenders in the right way to protect the public. But this review has been about trying to reduce the prison population in order to cut costs."