A campaign group that took on the running of Dudley Hippodrome must hand it back to the council after "failing" to restore the dilapidated building.
Dudley Metropolitan Borough Council has voted to take legal action against Black Country Hippodrome Ltd (BCHL) who took on a five-year lease in 2016.
The authority said BCHL "failed to achieve keystones" and that it was time to "be realistic".
BCHL said it did not accept it was the end and vowed to fight on.
In its heyday, the 1930's Art Deco building hosted stars including Marlene Dietrich and Laurel and Hardy.
It had previously been owned by the council, who planned to demolish it in 2013. It is not yet known whether it will now look to demolish the venue again.
"The building has a wonderful history and we are very proud of our past," the council said.
"But now is the time that we need to be realistic and accept that."
The cabinet approved recommendations for the authority's legal team to start procedures "leading to forfeiture of the lease".
The group had failed to meet four out of five "keystones" set out as conditions of the lease, including a funding strategy and a proposed scheme for the building, the council said.
But Dr Paul Collins, chairman of BCHL said they had secured the site after vandalism inside, got it insured and made watertight and were about to go inside to tidy up.
He said they disagreed with some of the council's findings and hoped to be allowed to restore the historic building.