Turkey says the forces of Libyan strongman Khalifa Haftar will become a "legitimate target" if they do not immediately release six Turks.
The Turkish foreign ministry said the detention of its citizens amounted to "banditry and piracy".
Gen Haftar's forces said on Friday they would attack Turkish targets over Turkish support for the internationally recognised Libyan government.
They also claimed they had destroyed a Turkish drone at a Tripoli airbase.
Libya has been torn by violence and division since long-time ruler Muammar Gaddafi was deposed and killed in 2011.
Gen Haftar started an offensive against the internationally recognised Government of National Accord (GNA), led by Prime Minister Fayez al-Sarraj, in April.
Why are Haftar and Turkey in conflict?
Turkey supports the GNA, supplying drones, weapons and trucks to boost its efforts in the ongoing conflict with Gen Haftar's forces, which control most of the east and south of Libya.
Gen Haftar's Libyan National Army (LNA) said it would strike Turkish vessels in Libyan waters and viewed Turkish businesses as targets.
It also banned commercial flights from Libya to Turkey.
Earlier on Sunday, Turkey said it would "retaliate in the most effective and strong way" to any threats from Gen Haftar's army.
Turkish Defence Minister Huluski Akar warned that LNA forces would pay a "very heavy price" for any attacks on Turkish interests.
Turkey says it wants to rebalance the fight against Gen Haftar, who has backing from the UAE and Egypt.
On Thursday, the GNA reclaimed the strategic town of Gharyan, a main supply base for Gen Haftar's forces in their offensive on Tripoli.
Why is there fighting in Libya?
Col Gaddafi ruled Libya for four decades until an armed rebellion toppled him in 2011.
Since then, no authority has had full control and the country is extremely unstable.
Several political and military factions are at play but two are key - Mr Sarraj and Gen Haftar.
Who is Gen Haftar?
Khalifa Haftar has been active in Libyan politics for more than four decades and was one of Gadaffi's close allies until a dispute in the late 1980s forced him to live in exile in the US.
Gen Haftar returned to Libya when the uprising began in 2011 and became a key commander of a rebel force in the east.
Libyans have mixed feelings towards him due to his past association with Gadaffi and US connections, but do credit him for driving Islamist militants out of much of Benghazi and its surroundings.
Gen Haftar's forces have won some support from France, Egypt and the UAE. In April, US President Donald Trump seemed to be showing support for Gen Haftar, but his enthusiasm has since cooled.