Alderney has been forced to drastically reduce its Island Games team because of the cost of travel to Bermuda.
The island, which had a 45-strong team at the Isle of Wight in 2011, will only have five competitors next month in golf, windsurfing and athletics.
Travel costs have also led to Guernsey while many islands are not sending football teams.
"It's bitterly bad luck on Bermuda in terms of timing," Alderney's team boss Michael Ellen told BBC Guernsey.
"These games have been scheduled for six or eight years, we've been conserving financial resources in order to get people there, but its a really expensive trip and it's turned up at the bottom of a recession."
However, Ellen expects a larger squad for the 2015 Games, which will take place in Jersey.
"The indications are that they'll be just as big as the Isle of Wight," he added.
Sark is not sending a team to Bermuda after having 11 Games competitors two years ago in shooting, archery and athletics.
Shooters Nick Dewe and Stefan Roberts won gold in the team automatic ball trap event at the Isle of Wight, while Roberts also won individual silver and bronze, medals.
However, it is the way events have been reorganised, rather than the cost of travel, which has prompted Sark's decision.
"We're rather disappointed that our archers couldn't compete because they're not providing the facilities for any archery in Bermuda," said Sark's Island Games chief Richard Dewe.
"Our archers are very disappointed because they would have moved heaven and earth to get the finances to get them there.
"For our clay target shooters, there's only a minimal amount of disciplines, not the full range of clay target disciplines."