Turkey's Prime Minister Ahmet Davutoglu says he will stand down at an extraordinary congress of his ruling AK Party later this month.
Mr Davutoglu is believed to have fallen from favour because he disapproved of Mr Erdogan's plans to move Turkey to a presidential system of government.
But in a speech, Mr Davutoglu pledged his loyalty to President Erdogan, saying he bore no anger against anyone.
His successor will be chosen when the congress meets on 22 May.
Earlier on Thursday, presidential aide Cemil Ertem said there would be no snap elections following the appointment of a new leader.
He also told Turkish TV that the country and its economy would stabilise further "when a prime minister more closely aligned with President Erdogan takes office".
Brutal end: Analysis by Mark Lowen, Turkey correspondent
When the end came, it was swift and brutal. Ahmet Davutoglu bowed out after crossing the man with the real power: President Recep Tayyip Erdogan.
Mr Davutoglu was expected to be a pliant prime minister but disagreed with some of Mr Erdogan's more controversial policies and crucially wavered in his support to change the constitution to boost the president's powers.
His resignation means Mr Erdogan tightens his control of Turkey and is likely to install a more obedient prime minister. It will worry many Western leaders who find the divisive Mr Erdogan difficult to handle.
And it plunges this crucial country into a political crisis amid security threats and rows over the clampdown on human rights and free speech. The message from President Erdogan to Mr Davutoglu's successor is clear: follow my lead or you'll face the same fate.
Mr Davutoglu met Mr Erdogan for nearly two hours on Wednesday but differences were clearly not resolved.
Mr Davutoglu said he would continue as a party legislator and would not try to divide the AKP.
"I feel no reproach, anger or resentment against anyone," he said.
"No-one heard, or will ever hear, a single word from my mouth, from my tongue or my mind against our president."
Why is this happening now?
After he was elected president in 2014, Mr Erdogan hand-picked Mr Davutoglu to succeed him as head of the AK Party (Justice and Development Party).
But the prime minister's unease with Mr Erdogan's plans to move to a presidential system, among other policies, has been evident in recent months.
In a sign of his weakening influence, Mr Davutoglu was stripped last week of the authority to appoint provincial AK Party officials.
What will this mean for Turkey?
The development comes at a time of increasing instability for Turkey, which is tackling an escalating conflict with the rebels of the Kurdistan Workers' Party (PKK), attacks by the so-called Islamic State, and an influx of migrants and refugees.
Turkey is also in the midst of implementing a key deal with the European Union, brokered by Mr Davutoglu, to limit the number of refugees flowing across its border in return for accelerated EU accession talks and financial aid.
The future of that agreement, which Mr Davutoglu was seen as having agreed with little input from the president, could be plunged into doubt by his departure.
Who will be his successor?
Among those tipped as successors to Mr Davutoglu are Transport Minister Binali Yildirim, who is close to Mr Erdogan, and Energy Minister Berat Albayrak, who is the president's son-in-law.
The leader will be formally elected at the party congress.
What has been the fall-out?
Main opposition leader Kemal Kilicdaroglu said Mr Davutoglu had been forced from office through the "will of one person".
"Davutoglu's resignation should not be perceived as an internal party issue, all democracy supporters must resist this palace coup," he said.
The political uncertainty also rattled the financial markets. The Turkish lira suffered its heaviest daily loss on Wednesday, down almost 4% against the US dollar.
It rallied slightly on Thursday but was still well off its previous trading levels.