A motion of no confidence in Labour leader Jeremy Corbyn has been passed by the party's MPs.
The 172-40 vote, which is not binding, follows resignations from the shadow cabinet and calls on Mr Corbyn to quit.
Mr Corbyn said the ballot had "no constitutional legitimacy" and said he would not "betray" the members who voted for him by resigning.
The leader's allies have told his critics to trigger a formal leadership contest if they want to challenge him.
Opponents of Mr Corbyn are meeting to decide what to do next and whether to rally round a single candidate to put up against him, with names in the frame including former frontbenchers Angela Eagle and Yvette Cooper.
The BBC's political correspondent Iain Watson said they still hoped Mr Corbyn would step down voluntarily.
As Mr Corbyn's future was being fought over, thousands of demonstrators - many of them young people - gathered outside Parliament to show their support for the EU and to protest at the outcome of the Brexit vote.
David Cameron is currently meeting EU leaders for the first time since the outcome of the referendum - with senior EU officials saying the process of starting formal exit talks cannot be delayed indefinitely.
The result of Tuesday's no-confidence ballot - in which there were four abstentions - has led to further resignations from Labour's frontbench and more calls for Mr Corbyn to make way.
Labour's leader in Scotland, Kezia Dugdale, suggested Mr Corbyn's position was untenable, telling the BBC: "If I had lost the support of 80% of my MSPs I could not do my job."
Analysis by Iain Watson, BBC political correspondent
I am told its unlikely - though these days the unlikely has a habit of happening - that we will get a formal challenge to Jeremy Corbyn tonight.
As one opponent put it "he should stew in his own juice" and see how uncomfortable it is for him in the Commons chamber to have so few MPs on his side. The hope is that he ignores the advice of his advisers and thrown in the towel.
I hear strong rumours of a meeting between key union supporters of Labour and a likely attempt will be made to get Jeremy Corbyn to go.
Under this scenario Tom Watson would become interim leader and either lead the party into a genuinely snap election or oversee a proper leadership contest to a more relaxed timescale open to all comers rather than an Eagle/Corbyn battle.
As things stand if Jeremy Corbyn still digs in his heels and MPs must unite behind one alternative, Angela Eagle looks the most likely… but don't rule Tom Watson out just yet.
Labour MP and Corbyn critic Wes Streeting said the vote was "unprecedented", adding: "I think Jeremy just has to accept now that his leadership is untenable."
But following the result the leader issued a statement saying the government was "in disarray" following the vote to leave the EU, adding: "Labour has the responsibility to give a lead where the government will not."
He added: "I was democratically elected leader of our party for a new kind of politics by 60% of Labour members and supporters, and I will not betray them by resigning. Today's vote by MPs has no constitutional legitimacy.
"We are a democratic party, with a clear constitution. Our people need Labour Party members, trade unionists and MPs to unite behind my leadership at a critical time for our country."
The most recent resignations from Mr Corbyn's top team include shadow justice minister Andy Slaughter, shadow culture minister Clive Efford and shadow local government minister Liz McInnes who backed Mr Corbyn in the ballot but said the overwhelming result left him with no option but to stand down.
Dave Sparks, a Labour councillor in Dudley and a former chair of the Local Government Association, warned that if Mr Corbyn stays, the party was looking at its support disappearing in England as it has melted away in Scotland.
Mr Corbyn faced calls to resign at a stormy meeting in the House of Commons on Monday after more than 20 members of his shadow cabinet and a similar number of junior ministers walked out, questioning his performance during the EU referendum and ability to lead the party into what they believe could be a snap election.
But one of Mr Corbyn's allies, newly promoted shadow health secretary Diane Abbott, said the no confidence vote "has no meaning".
"MPs don't choose the leader of the Labour Party, the party does," she told Today.
"I think it is really sad that colleagues have chosen to stage this three-ring circus because they don't want to have a leadership election because they are not certain of winning a leadership election. "The way to resolve this is to have a leadership election."
The new shadow cabinet line-up includes:
- Shadow foreign secretary - Emily Thornberry
- Shadow health secretary - Diane Abbott
- Shadow education secretary - Pat Glass
- Shadow transport secretary - Andy McDonald
- Shadow defence secretary - Clive Lewis
- Shadow chief secretary to the Treasury - Rebecca Long-Bailey
- Shadow international development secretary - Kate Osamor
- Shadow environment food and rural affairs secretary - Rachel Maskell
- Shadow voter engagement and youth affairs - Cat Smith
- Shadow Northern Ireland secretary - Dave Anderson
On Monday, Mr Corbyn announced a reshaped shadow cabinet to replace those that had walked out but several positions in his top team remain to be filled after the mass resignations.
The shadow cabinet walkouts - in a bid to oust Mr Corbyn - came after the sacking at the weekend of shadow foreign secretary Hilary Benn, who told Mr Corbyn he had lost confidence in his leadership.
But Momentum, the campaign group that grew out of Mr Corbyn's successful leadership bid, has said 4,000 people attended the rally outside Parliament, and Mr Corbyn has also been backed by the Unite, GMB and Unison trade unions.