The Ministry of Justice is considering banning prison sentences of less than six months in England and Wales.
Ministers argue that short jail terms are less effective at cutting reoffending than community penalties.
Prisons minister Rory Stewart told the Daily Telegraph that such sentences were "long enough to damage you and not long enough to heal you".
If such jail sentences were to be scrapped, it is thought it could free up thousands of prison places.
Some 30,000 criminals, including burglars and most shoplifters, could be spared jail every year under the proposals, the Telegraph reports.
'Loss of reputation'
Offenders could face sanctions such as community sentences but no alternative penalties have been confirmed.
Overall, the measure could reduce the prison population by about 3,500, although it would require legislation.
The Ministry of Justice is now considering preventing courts from imposing prison terms of less than six months unless the sentence is for a violent crime or a sexual offence.
In Scotland, a presumption against prison sentences of less than three months is already in place and is due to be extended to 12 months.
Arguing for the need for reform, Mr Stewart told the Daily Telegraph Magazine: "You bring somebody in for three or four weeks, they lose their house, their job, their family, their reputation.
"They come (into prison), they meet a lot of interesting characters (to put it politely) and then you whap them on to the streets again.
"The public are safer if we have a good community sentence... and it will relieve a lot of pressure on prisons."
Since they took up their posts last year, Justice Secretary David Gauke and Mr Stewart have both made it clear they want to reduce the use of short prison sentences.
The prison population has doubled in England and Wales since the early 1990s, rising from about 40,000 to more than 80,000 in 2018, official figures show.
Almost two-thirds of prisoners released after sentences of less than 12 months reoffend within a year.
More than half of the 86,275 offenders sentenced to immediate custody in England and Wales in 2017 were handed sentences of six months or less, according to a Parliamentary response from Mr Stewart last month.
Mr Stewart said he realised that the proposals could provoke a backlash against "soft justice" by people in his own party and the public, but said it was "a debate I have to win".
The Prison Reform Trust, which has previously called for a presumption against short prison sentences, welcomed the new proposals.
Peter Dawson, the charity's director, told the Telegraph: "Ministers should be congratulated for having the political courage to start the debate."
He told BBC News that while the law required community penalties to contain an element of punishment, they can include other things as well such as community work, treatment for addiction and help for mental health conditions.
An MoJ spokesperson said: "As we have said previously, short sentences are too often ineffective, provide little opportunity to rehabilitate offenders and lead to unacceptably high rates of reoffending.
"That's why we are exploring potential alternatives, but this work is ongoing and we have reached no conclusions at this time."