Opponents of President Barack Obama's new Cuba policy have threatened to block his efforts to restore diplomatic relations after 50 years of hostility.
Florida Senator Marco Rubio promised on CNN to block the nomination of any US ambassador to Cuba.
Other anti-Castro legislators suggested Congress would remove funding for any normalised ties with the country.
US-Cuban ties have been frozen since the early 1960s - a policy of isolation Mr Obama condemned as a failure.
On Wednesday, the US president said it was time for a new approach.
As part of the deal, US contractor Alan Gross, 65, and an unnamed intelligence officer loyal to the US were released from Cuban prison in return for three Cubans held in the US.
Analysis, Thomas Sparrow, BBC Mundo
It is interesting to note that there is a marked contrast between Cuban Americans around the US, who tend to favour normal relations between the two countries, and leading Cuban American congressmen in Washington.
Senators Robert Menendez, a Democrat from New Jersey, and Marco Rubio, a Republican from Florida, have played very vocal roles in the Senate Foreign Relations committee, and they have both been very critical of President Obama's announcement yesterday.
Mr Rubio said he didn't care if "99% of people in polls" disagreed with his stance, a message that could resonate in Florida, the traditional home of the Cuban exile community but also a state that is increasingly becoming more diverse and less dominated by Cuban affairs.
Ultimately, comments like these come to show how so-called hardliners still have significant clout when it comes to preventing a new course in bilateral relations, even when common Cubans, especially younger generations, have moved on beyond these traditional views.
The US will now seek to set up an embassy in Cuba, expand US visitors to Cuba, open up banking and increase caps on how much cash Cubans can post to relatives on to the island.
But only Congress has the power to end the full trade embargo, and with many Republicans deeply opposed to such a change, correspondents say it is unlikely to happen soon.
Among those opposed to restoring diplomatic relations was Democratic Senator Robert Menendez who said he was "deeply disappointed".
"It's a fallacy to believe that Cuba will reform because an American president opens his hands and the Castro brothers will suddenly unclench their fists."
Fellow Senator Lindsey Graham, a Republican, said he would be among those trying to pass legislation to undercut funding for policy changes, including setting up an embassy.
"Normalising relations with Cuba is a bad idea at a bad time," tweeted Mr Graham, who will become chairman of a committee that determines state department funding in January.
Republican Senator Marco Rubio slammed the deal as "inexplicable".
"Appeasing the Castro brothers will only cause other tyrants from Caracas to Tehran to Pyongyang to see that they can take advantage of President Obama's naivete during his final two years in office," Mr Rubio said in a statement.
Mr Rubio told CNN on Wednesday he reserved the right "to do everything within the rules of the Senate to prevent that sort of individual from ever even coming up for a vote," referring to the confirmation process for ambassadors in relation to Cuba.
Their objections mirror the concerns of some dissident Cubans living in the US.
"It is a betrayal. The talks are only going to benefit Cuba," Carlos Munoz Fontanil said at a protest in Miami's Calle Ocho.
Despite this opposition by members of Congress like Mr Rubio and Mr Menendez, some analysts have preferred to stress that Mr Obama has considerable support in Congress.
Cynthia Arnson, director of the Latin America programme at the Wilson Center, told BBC Mundo that "it is important to underscore that there are many Republicans who are not resisting this or are neutral".
And Tomas Bilbao, executive director of the Cuba Study Group, a non-governmental organisation in Washington, highlighted the fact that the plane that brought Alan Gross back to the US had three members of Congress on board, including Republican Jeff Flake, from Arizona.
Officials said that Mr Obama and Mr Castro spoke by telephone on Tuesday for nearly an hour - the first presidential-level talks between the two nations since Cuba's 1959 revolution.
In exchange for Mr Gross, who was in poor health, and the unnamed intelligence officer, Washington released three members of the so-called "Cuban Five" who were serving lengthy sentences for espionage.
Mr Gross's five-year imprisonment had undermined previous attempts to thaw diplomatic relations between the two countries.
1959: Fidel Castro and his guerrilla army defeat the US-backed Cuban regime of Fulgencio Batista
1960-1961: Cuba nationalises US businesses without compensation; US breaks off diplomatic relations and imposes a trade embargo in response
1961: Failed Bay of Pigs invasion by CIA-backed Cuban exiles
1962: Soviet Union deploys ballistic missiles to Cuba, prompting Cuban Missile Crisis
2001: Five Cubans, dubbed the Cuban Five, are jailed in Miami for spying
2008: Raul Castro becomes Cuban president
2009: US citizen Alan Gross detained in Cuba accused of spying
Dec 2013: US President Barack Obama and Raul Castro shake hands at Nelson Mandela's funeral - the first such public gesture since 1959
17 December 2014: Alan Gross is released by Cuba