Nato has invited Montenegro to join the US-led military alliance, in its first expansion for six years.
The decision, described by Nato head Jens Stoltenberg as "historic", comes 16 years after the alliance bombed Montenegro during the Kosovo war, when it was still part of Yugoslavia.
Montenegro's accession would result in "retaliatory actions", said a spokesman for Russian President Vladimir Putin.
Accession talks are expected to take about a year to complete.
The invitation to Montenegro is Nato's first expansion into eastern Europe since Albania and Croatia joined in 2009.
The mountainous Adriatic state of 650,000 people has a small military with about 2,000 active members.
Nato diplomats say it sends a message to Russia that it cannot veto the alliance's expansion - but Russia has said it will retaliate.
"The continued eastward expansion of Nato and Nato's military infrastructure cannot but result in retaliatory actions from the east, i.e. from the Russian side, in terms of ensuring security and supporting the parity of interests," Mr Putin's spokesman Dmitry Peskov said.
Mr Peskov added that it was too early to specify what the retaliatory actions would be.
Analysis - Jonathan Marcus, BBC diplomatic correspondent
Montenegro is now on its way to becoming Nato's 29th member, some 16 years after Nato bombs were dropping on Serbian military targets in the country.
It is an indication of the dramatic changes in the region. Montenegro's government believes that its eventual membership will be good news for the western Balkans and will contribute to increasing stability in the region.
Montenegro has tiny armed forces. Its navy largely comprises two ex-Yugoslav frigates. Thus it is unlikely to be a major contributor to Nato's collective security.
For Nato it is a further demonstration that the alliance's door remains open to new members. The invitation for Montenegro to begin the process of membership has clearly irritated Moscow, which regards Nato expansion as a provocation, while Nato insists that this decision has nothing at all to do with Russia.
Montenegrins themselves remain divided over joining.
Many remain angry that Nato bombed Serbia and Montenegro in 1999 as part of a strategy to halt the killing and expulsion of ethnic Albanians in Serbia's then southern province of Kosovo.
The country has also seen an influx of Russian money, homebuyers and tourists since splitting from Serbia in 2006.
The government of Milo Djukanovic - which is in favour of joining - has resisted calls from some opposition parties for a referendum on the issue.
However, Nato diplomats point to polls that suggest public opinion is narrowly in favour of joining.
Besides Montenegro and Georgia, the other current candidates for Nato membership are Bosnia and Herzegovina and Macedonia.