Pro-Russian forces have taken control of two naval bases in Crimea - including the HQ of Ukraine's navy.
Streams of Ukrainian servicemen were seen leaving the bases. Kiev said its navy chief has been detained.
Crimean leaders signed a treaty with Moscow on Tuesday absorbing the peninsula into Russia.
Sunday's referendum approving Crimea's split from Ukraine came nearly a month after Ukraine's pro-Russian President Viktor Yanukovych fell from power.
Crimean and Russian officials said the vote showed overwhelming public support for joining Russia, with 97% of voters in favour.
But the West and the Ukrainian government in Kiev say the hastily organised referendum - boycotted by many of Crimea's Ukrainian and Tatar minorities - was illegal and will not be recognised.
Ukraine's interim President Olexander Turchynov has demanded the Crimean authorities release navy chief Serhiy Hayduk by 21:00 local time (19:00 GMT).
He said that unless Mr Hayduk and "all the other hostages - both military and civilian ones - are released, the authorities will carry out an adequate response... of a technical and technological nature".
It is not clear exactly what he means, but it could involve the electricity or water that Ukraine supplies to Crimea, the BBC's David Stern in Kiev suggests.
Kiev said Mr Hayduk was detained soon after Ukraine's naval headquarters was stormed by some 200 pro-Russian activists, some armed, in Sevastopol - the port city which is also homes Russia's Black Sea fleet.
They were filmed going through offices, removing Ukrainian insignia and replacing Ukraine's flag with the Russian tricolour.
There were cheers from the crowd when Russia's Black Sea Fleet commander Aleksandr Vitko arrived and entered the building.
Ukraine's defence ministry said its navy base in Novo-Ozyorne in west Crimea was also infiltrated after a tractor was used to ram the front gates.
An Agence France Press reporter saw some 50 Ukrainian servicemen filing out of the base watched by Russian soldiers.
A pro-Russian forces member in Sevastopol, Viktor Melnikov, said "not a drop of blood has been spilled" and there was no violence.
But one of the Ukrainian servicemen who left, Captain Olexander Balanyuk, said: "There were many promises from the Russian side and our side that the base will not be stormed, that all issues will be resolved through political means, but as you see now - there was a takeover."
But some said they would never surrender.
One serviceman talked to the BBC by telephone from inside the headquarters, saying he was one of around 100 Ukrainians who had barricaded themselves in using furniture to block the doors.
Ukrainian Defence Minister Ihor Tenyukh and First Deputy Prime Minister Vitaly Yarema reportedly tried to enter Crimea to defuse tensions but were prevented from doing so.
Earlier on Wednesday, Russia's constitutional court approved the treaty absorbing Crimea into the Russian Federation. The treaty now only needs ratifying by parliament which correspondents say it is certain to do.
In an emotionally charged speech on Tuesday, Mr Putin said Crimea had "always been and remains an inseparable part of Russia".
Meanwhile, shocking footage has emerged of MPs from Ukraine's far-right Svoboda party roughing up Oleksandr Panteleymonov, the acting chief executive of the state broadcaster, over his decision to broadcast the ceremony in the Kremlin.
The crisis in Crimea is expected to dominate a meeting of European Union leaders who meet in Brussels on Thursday.
Both the EU and the US have already imposed sanctions on several officials from Russia and Ukraine accused of involvement in Moscow's actions in Crimea.
UK Prime Minister David Cameron said the EU must send "a very clear warning" to Russia. He also said the G8 group should discuss whether to expel Russia "if further steps are taken".
Moscow said any expansion of sanctions is "unacceptable and will not remain without consequences".
UN Secretary General Ban Ki-moon is heading to the region, and will meet Russia's President Vladimir Putin in Moscow on Thursday and Ukraine's interim leaders in Kiev on Friday to try to find a way forward.
Pro-Russian forces effectively took over Crimea - with its predominantly ethnic Russian population - after Mr Yanukovych fled Ukraine on 22 February following protests in which more than 80 people were killed.
Many in Crimea say they oppose the new West-leaning government, which was formed partly from those who led protests against Mr Yanukovych over his decision in November to ditch an EU association agreement in favour of stronger ties with Moscow.