A retired army general, David Granger, is expected to become Guyana's new president, the election board says.
His A Partnership for National Unity and Alliance for Change (APNU+AFC) coalition won by a slim margin.
Guyanese politics is divided on racial lines; the Indian-dominated People's Progressive Party (PPP/C) has held power for 23 years.
Mr Granger has brought together a coalition of Afro-Guyanese parties to win by 5,000 votes.
The 69-year-old had vowed to end race politics in the lead up to the election.
"The time has come for racial and national unity," the AP news agency reports.
"The time has come to end winner-take-all politics, corruption, nepotism and the squandering of our resources," Mr Granger said.
His APNU+AFC coalition, won 206,817 votes, compared to the ruling PPP/C's 201,457.
The current President, Donald Ramotar, has requested a full recount.
"I cannot concede that I have lost this election," he told the AP news agency.
When asked if he would refuse to give up power, he answered, "I have not closed any option as yet."
International diplomats say that the vote was held fairly.
Mr Granger lost the last election to Mr Ramotar in 2011.
The retired general has campaigned on a platform of increased security and fighting drug and human trafficking.
Guyana is the third poorest country in Latin America after Haiti and Nicaragua, according to World Bank figures.