Ukraine's president has hailed the recapture of the rebel stronghold of Sloviansk as the start of a turning point in the three-month conflict.
Petro Poroshenko said it was not a total victory, but rather an event of "huge symbolic importance".
Government forces have made territorial gains since launching an offensive this week in Donetsk and Luhansk regions, following the breakdown of a ceasefire.
Pro-Russian rebels still hold the two regional capitals and other key areas.
But Sloviansk had been considered a focal point of the rebellion, and was the military centre of the self-declared People's Republic of Donetsk.
Analysis: David Stern, BBC News, Kiev
Of the government's victories so far, the retaking of Sloviansk and raising of the Ukrainian flag over city hall is by far the most significant. The city was not just a command centre for the insurgency - it was a symbol of the militants' continuing ability to thwart Kiev's attempts to reassert control in the east.
Now, it appears that the insurgents may also be evacuating Kramatorsk, another key city. But the question is whether this is a turning point in the war, or merely a shifting of the battlefield.
The rebels are calling this a tactical retreat, which is typically a euphemism for a defeat. But if they are moving en masse to Donetsk, it could still present a major military challenge to government forces. It could also convince the rebels to engage in ceasefire talks more actively - if only to buy more time.
"This is not a complete victory yet, and it is not the time for celebrations," Mr Poroshenko said in a televised address.
"But the cleansing of Sloviansk from gangs who are armed to the teeth is of huge symbolic importance. This is the start of a turning point in the fight against the militants."
Ukrainian officials said the military was now in Sloviansk, and promised humanitarian aid would be delivered soon.
The rebels left the city after a night of mortar shelling in a move apparently orchestrated by the breakaway region's military commander, Igor Strelkov.
A rebel spokesman said their forces would abandon the entire "northern sector" and pull back to Donetsk city.
The Ukrainian government had earlier claimed to have retaken two small towns.
And other reports said Kramatorsk city, near Sloviansk, had also been recaptured.
Who is Igor Strelkov?
- Real name Igor Girkin, self-declared commander-in-chief of Donetsk People's Republic
- Ukraine says he works for Russian military intelligence agency the GRU
- On EU sanctions list for posing threats to Ukraine's independence
- Noted for love of re-enacting Roman, Napoleonic and 20th Century battles
Violence erupted in eastern Ukraine in April, when separatists declared independence in the Donetsk and Luhansk regions.
Earlier this week, the government claimed to have retaken two-thirds of the territory in those regions.
The current crisis started when President Viktor Yanukovych decided last November not to sign an agreement with the EU.
The decision led to street protests in Kiev, and Mr Yanukovich was eventually overthrown.
The developments angered Russia, which annexed Ukraine's Crimea region.