Nigeria's army killed some 600 people after a recent attack by Boko Haram militants on a barracks, Amnesty International has said.
Quoting eye-witnesses, it said that after the raid in the north-eastern city of Maiduguri, detainees who had escaped were rounded up and killed.
The army has not yet commented on the allegations but has denied previous reports of abuses.
The conflict has killed some 1,500 people this year, according to Amnesty.
Half of those killed were civilians, the organisation said.
Hundreds of militants were said to have taken part in the 14 March attack on the Giwa barracks in Maiduguri, which BBC Nigeria correspondent Will Ross described as a particularly brazen assault.
At the time, the Nigerian military said the attack was successfully repelled and many attackers were killed by the air force and ground troops.
One eyewitness told Amnesty that a self-defence group known as civilian JTF captured some of the hundreds of detainees who had escaped.
"I saw the soldiers asking the people to lie on the ground," the witness said.
"There was a small argument between the soldiers and the civilian JTF. The soldiers made some calls and a few minutes later they started shooting the people on the ground. I counted 198 people killed at that checkpoint."
Amnesty says satellite images have revealed three possible mass graves around the city.
Captured Boko Haram suspects are often detained in Giwa barracks, and human rights groups say hundreds have died or been subjected to torture there - allegations always denied by the military.
Boko Haram has since released its own video of the attack on Giwa, showing crowds of people walking out of the barracks.
Amnesty says all parties are violating international law and wants the United Nations to help investigate potential war crimes and crimes against humanity.
"The scale of atrocities carried out by Boko Haram is truly shocking, creating a climate of fear and insecurity," said Amnesty's Netsanet Belay.
"But this cannot be used to justify the brutality of the response that is clearly being meted out by the Nigerian security forces."
The violence has forced some 250,000 people from their homes so far this year, according to the government's relief agency.
It said more than three million people are facing a humanitarian crisis.
Boko Haram was launched in Maiduguri in 2009, with the aim of setting up an Islamic state.
A state of emergency was declared in three north-eastern states last year to help the military crush the insurgency.
However, the militants have stepped up attacks in recent months.