The Labour group is calling for Swindon Borough Council to ditch a developer over its progress in delivering a huge regeneration project.
The Kimmerfields contract with Muse was announced in 2008, but much of the land around Fleming Way remains derelict.
Labour said the council needed a new partner as "nothing's happened".
Muse said the council was now in the process of acquiring the remaining land necessary to be able to bring forward development.
Tory cabinet member Garry Perkins said: "We need to take stock of where we are and look at the options rather than rush off into new directions."
Both Conservative MPs for Swindon, Justin Tomlinson and Robert Buckland, have also expressed concerns about the Kimmerfields project.
Lack of investment
When Muse signed the deal, it promised private investment to build 450 new residential properties, offices, a new hotel and shops.
Mr Perkins said the first phase, a new multi-storey car park, health centre and an assisted living residential home, all funded via the taxpayer, was now complete.
A spokesman for Muse said: "Major regeneration projects like this are extremely complex and involve a number of third parties and landowners to complete land assembly so can take several years to come to fruition."
The council has spent the past two and a half years buying up land, using compulsory purchase orders, which it only completed two months ago.
The delay has also been blamed on the recession, which resulted in a lack of investment in regeneration projects.
Labour councillor Jim Robbins said: "We would get rid of Muse, we're looking at 10 years that they've been there and nothing's happened.
"They're clearly not the right people for us to be working with, the sensible thing for the council to do is get rid of that contract and find a development partner that can actually work and deliver."