Ken Clarke has said there is "no dissent" among ministers about proposed reforms to sentencing and penal policy designed to cut reoffending rates.
The justice secretary has been attacked by sections of the media for pledging to curb the rise in prisoner numbers.
Mr Clarke told MPs it was "right" to imprison serious criminals and he was not at odds with Home Secretary Theresa May over whether prison worked.
He insisted he had never called for an end to short sentences.
Ministers want to cut the 85,000 inmate population in England and Wales by 3,000, reducing the £4bn prison and probation budget by 20% over four years.
Mr Clarke has pledged to "break the cycle" of crime by tackling the causes of reoffending, pledging more help to deal with inmates' drug and alcohol addiction problems and mental illness.
Launching a green paper on sentencing in England and Wales earlier this month, he also called for more emphasis on making prisons more purposeful and toughening up community sentences.
But amid reports in several newspapers that he had been forced to back down over sentencing changes, and following comments by the Home Secretary that prison worked, Mr Clarke said there was no difference of opinion over the matter.
"I had heard this morning that the home secretary had apparently disagreed with me and I looked at what she said and she said exactly the same things," he told the Justice Select Committee.
"Prison is the right punishment for serious criminals... but prison, at the moment, is not succeeding in getting reoffending rates down."
Mr Clarke said sections of the press had "flared up" over sentences for certain crimes but there was no disagreement with ministers over their overall approach, which he said was "steady and consistent".
"I don't mind people attacking me from outside but there isn't any disagreement on the government's key policy," he said, adding it had been discussed in cabinet and approved by the prime minister.
Mr Clarke insisted there was no general target for reducing prison numbers.
Efforts to reduce the number of people reoffending within a year of being freed from 50% to 40% would be "quite challenging", he added.