Malawi has imposed a moratorium on anti-homosexual laws pending a decision on whether to repeal the legislation, Justice Minister Samuel Tembenu has said.
A review of all colonial-area sodomy laws will be launched in consultation with the people of Malawi, he added.
Mr Tembenu also ordered the release of two men charged with having sex "against the order of nature".
One human rights activist describes the move as a step in the right direction.
However, many Malawians are not happy with the decision and religious leaders have asked the government not to relent to pressure from Western donors by allowing same-sex relationships.
Since 2012, the US government has put more than $350m (£230m) into supporting gay rights groups in sub-Saharan Africa, according to the New York Times.
The arrest of the two men earlier this month was condemned by the United States and leading international human rights organisations.
Malawi is one of the poorest countries of the world and relies heavily on international aid.
Mr Tembenu said the country was a signatory to international treaties which allowed homosexual acts, the local Nyasa Times newspaper reports.
But he wants the international community to appreciate the "sensitiveness of the matter".
As in several African countries, homosexuality is widely seen as taboo in Malawi. Both the Catholic Church and Muslim leaders have in the past described it as un-Godly.
Malawian human rights activist Billy Mayaya told the BBC that before the law was changed, the government should explain its new position.
"It is on the right track to abolish sodomy laws but the public needs to be sensitised to understand the secular nature of our country," he said.
"Many people here thought Malawi is a Christian country which is not true, we are a secular state," Mr Mayaya added.
The government announced a similar suspension of anti-homosexuality laws in 2012, however this did not prevent the two men being arrested.