Ex-leader Lord Elis-Thomas left Plaid Cymru over Labour stance
Lord Elis-Thomas left Plaid Cymru to sit as an independent AM because the party was not "serious" about taking part in the Labour-led Welsh Government, he has said.
He said pro-devolutionists should be working together to ensure Wales does not lose powers.
The former party leader dramatically announced his exit on Friday, prompting anger from some of his colleagues.
He has rejected demands for him to stand down and trigger a by-election.
Labour has a minority of AMs in the assembly and needs other parties to help govern.
Plaid Cymru has not agreed to a formal coalition deal with Labour, but did allow Carwyn Jones to return as first minister following an earlier deadlocked vote over whether Mr Jones or Plaid leader Leanne Wood would take the top job.
Lord Elis-Thomas, who represents Dwyfor Meirionnydd, said: "It was clear to me that the Plaid group in the National Assembly was not willing to seriously participate in government and to give Wales a stable government, particularly after the decisions taken in Wales and elsewhere about our future relationship with the European Union".
"There is a serious constitutional challenge now and we might well lose powers from Wales," he said.
"I think all pro-devolutionists should be working together to ensure that doesn't happen."
Lord Elis-Thomas was critical of the deadlocked first minister vote, caused when the Tories and UKIP backed Ms Wood for the top job.
He said it was done without his knowledge and was "totally inappropriate".
Asked if he would support the next Welsh Government budget, which is due to be tabled on Tuesday, he said: "It depends on what's in it."
"I have to be satisfied that that budget contains funding for the work I'm doing for them now", he said, pointing to his work on the future of designated natural landscape areas in Wales.
Last year Lord Elis-Thomas clashed with his party over remarks he made about its general election campaign.
He suggested efforts in the wake of the row to make him refrain from public statements about controversial issues breached his human rights.
On Saturday the peer alleged that he had been "forced to attend an unconstitutional party meeting" by the national executive committee "in which they sought to deselect me as a candidate in Dwyfor Meirionnydd after I had been selected properly by the constituency party".
"The whole episode was totally disorderly. I said so at the time, they wouldn't listen to me," he said.
Lord Elis-Thomas said he had been "forced to agree to not speaking publicly about any matters they regarded as controversial without specific permission of the party chair".
"I stuck with that until the last election," he added.
"I certainly don't think I had any further obligation to anyone within the party and that I should be able to speak my mind as an assembly member.
"It's contrary to democratic principles as well as my human rights to try to stop an AM from speaking out in support of his constituents and in support of his own principles."
Former Plaid Cymru president Dafydd Iwan was among party figures who criticised the former presiding officer at the weekend.
He said: "I'm saddened by his decision.
"But I think Dafydd has never been a good team player and and he's failed to put the cause and his party before his own self-preservation.
"I think this is a step too far," he said, adding that there should be a by-election.
Plaid AM Bethan Jenkins said: "I am sad and angry at the decision made by Dafydd."
The South Wales West AM said: "Does Dafydd want his legacy to be that of letting Plaid voters and our nation down in such a way, after achieving so much as a Plaid MP, presiding officer, and former leader of the party?
"This action... has shown that he has let his constituents, the nation, and the party down."
Rhun ap Iorwerth, Plaid AM for Anglesey, told BBC Radio Wales: "If this was happening towards the end of a parliamentary term, it might be different.
"But this is within five months of the election in the context where Dafydd Elis-Thomas, a major contributor to Welsh political life over the years, had an opportunity to stand as an independent, which he had every right to do."
"And he said, no, I'm loyal to Plaid Cymru, trust me on this one," said Mr ap Iorwerth.
"And there's an element of breaking that trust."
At least one Plaid Cymru party member, who did not want to be identified, had sympathy with Lord Elis-Thomas.
The source said: "I'm not surprised. The group is not run in an inclusive way."
The Plaid member added that Lord Elis-Thomas "should have been shown more respect".
A Welsh Tory source on Saturday suggested that Plaid Cymru should give up the title of the official opposition now that it has the same number of AMs as the Conservatives in the assembly.
The source said: "Its like bald men fighting over a comb. We're not going to fight over the designation - but either the title of official opposition is shared by Plaid or it is dropped."
A spokesman for the party added: "The Welsh Conservatives will continue to be the real opposition".
Analysis by BBC Wales political editor Nick Servini
As we have seen with Nathan Gill's departure from the UKIP group at the Senedd, it does not take long for internal splits to develop in this assembly term.
There are potentially big implications for the assembly.
Labour narrowly missed out on an overall majority, and with two floating votes up for grabs in the form of Lord Elis-Thomas and Nathan Gill, Plaid and the Tories may find it more difficult to put the squeeze on Labour.
Having said that, Labour has formal mechanisms in place with Plaid to discuss the budget, Brexit, legislation and the constitution, so there's likely to be a willingness to make that work.
Dafydd Elis-Thomas - a timeline
Carmarthen-born Lord Elis-Thomas began his political career in Westminster, serving as a member of parliament for Meirionydd between 1974 and 1983 and for Meirionydd Nant Conwy between 1983 and 1992.
In 1992, he was made a life peer.
Lord Elis-Thomas was elected to the newly established National Assembly for Wales in 1999, representing the Meirionnydd Nant Conwy constituency until it was abolished in 2007.
He has served as the AM for the constituency of Dwyfor Meirionnydd since that year's election.
The peer served as presiding officer from the start of the institution until 2011.
As the assembly's figurehead he clashed with Leanne Wood, who is now party leader, in 2004.
Lord Elis-Thomas ordered her to leave the Senedd chamber on the grounds of 'discourtesy' when she refused to withdraw her reference to the Queen as 'Mrs Windsor'.
He had made a bid to become party leader in 2012 but lost to Ms Wood, and since then Lord Elis-Thomas has repeatedly fallen out with the party leadership.
That year Plaid Cymru withdrew the whip from Lord Elis-Thomas after he failed to attend a vote of no confidence in then health secretary Lesley Griffiths.
In March 2014 he was fired as transport spokesperson and chairman of the assembly's environment committee following his criticism of the leader's attack on UKIP at the party's spring conference.
The former assembly presiding officer said it was "facile" of Ms Wood to say a vote for UKIP is a vote against Wales.
In 2015 the AM criticised the party's general election campaign, leading to him to face disciplinary action.
He won the support of local party members at that time, but clashed again with the party when he backed Labour's north Wales police and crime commissioner candidate earlier this year.