Liam Fox has said the Ministry of Defence must become "leaner" to help tackle the "dangerous deficit".
The defence secretary indicated there would be fewer civil servants and senior officers, to ensure enough money was devoted to the front line.
He did not say what jobs or equipment would go, but said cutting the deficit would be "difficult and painful".
For Labour, Bob Ainsworth said Dr Fox was just "softening us up" for the cuts he had previously opposed.
The MoD is looking at cuts to its budget of between 10% and 20%. A strategic defence and security review, expected to bring about cuts in equipment and personnel, is due to report in October.
In a speech at the Royal Institution of Chartered Surveyors, Dr Fox said it had to be the "defence review that puts the Cold War to bed", and that it would "shape Britain for the 21st Century".
And he blamed Labour for what he called an "unfunded liability" of £37bn defence spending plans over 10 years - £20bn for equipment and support programmes Dr Fox said were ordered "without ever having an idea whether the budget would be able to afford it".
He outlined plans to reorganise the MoD into three areas - policy and strategy, the armed forces and procurement and estates - and for a cultural shift "which will see a leaner and less centralised organisation".
In opposition, Dr Fox had said he wanted to cut the department's running costs by 25% and criticised the fact it had 85,000 civil servants while the Army had just 100,000 soldiers.
But asked how many civil service jobs he expected to go, he would not put a figure on it, saying there would be "root and branch reform", and that it was "much more complex than that".
"I don't pretend that it will be painless, but I want, at the end of this period of office, for us to look back and say we took the necessary changes for the long-term view."
He said a visit to Afghanistan this week had reminded him that "the prime purpose of what we are doing is to make sure that our armed forces on the front line have everything they need... to carry out their mission successfully and safely".
"That means that the backroom sometimes has to do without to make sure that the front line gets what it wants."
He also said "fundamental assumptions" about tour lengths and intervals for armed personnel had to be challenged, "taking into account the varying pressures on our personnel resulting from widely varying missions".
While he said he was not intending to merge the armed forces, he would "consider whether the current senior rank structure across the services is appropriate".
"We cannot demand efficiency from the lower ranks while exempting those at the top."
The defence secretary announced that a defence reform unit was being set up under Lord Levene to guide the strategic defence and security review.
The new unit will work with the permanent secretary, chief of the defence staff and the service chiefs to find ways of devolving more responsibility for the running of the armed services themselves.
Shadow defence secretary Bob Ainsworth said Dr Fox had said nothing of substance: "What he's effectively doing is softening us up for cuts that he opposed in opposition."
He told the BBC he was concerned that not enough time had been put aside for a proper defence review - and it appeared to be "Treasury-led".
"He can't even tell us whether the MoD budget is going to have to pay for the full costs... of the Trident nuclear deterrent."
Previously the Trident renewal costs - estimated at £20bn - were funded directly by the Treasury - but Chancellor George Osborne said last month the full costs must be met from the MoD's budget. Dr Fox said how that would happen was "a conversation that is constantly ongoing with the Treasury".
Mr Ainsworth agreed that tour lengths should be looked at but he warned against cutting civilian staff too quickly as it could mean their jobs have to be "backfilled" by uniformed personnel.
Asked about the £37bn funding gap, Mr Ainsworth said: "There are some pressures on the [MoD] budget but there always have been. It's just not true to say this is a new creation, that this wasn't there in the past or these difficulties don't exist in other countries, because they do."
The SNP said the coalition government should abandon plans to renew the Trident nuclear missile system and not undermine the "conventional capabilities of our forces" in Afghanistan.
"Absorbing Trident into the core defence budget is unsustainable and would have a devastating impact for spending on conventional forces, which are already overstretched," said the SNP's Westminster spokesman Angus Robertson.