Berlin police are trying to find out who opened the unmarked grave of SS officer Reinhard Heydrich, a top Nazi killed by Czechoslovak agents in 1942.
An employee at the Invalids' Cemetery in central Berlin found on Thursday that the grave had been opened.
No bones were removed, police say.
Heydrich was a key organiser of Nazi Germany's mass murder of European Jews. He chaired the Wannsee Conference in January 1942, where Hitler's genocidal "Final Solution" was planned.
Tampering with a grave can be prosecuted under a German law against "grave defilement".
The Allied occupation forces at the end of World War Two decreed that the graves of prominent Nazis should not be marked, to prevent Nazi sympathisers turning them into shrines.
Whoever violated Heydrich's grave is thought to have had inside knowledge of its location.
A similar incident happened at Berlin's Nikolai Cemetery in 2000, when a left-wing group opened what they claimed was the grave of Horst Wessel, a Nazi stormtrooper murdered in 1930, who was turned into a martyr and honoured with a Nazi anthem.
The group claimed to have thrown Wessel's skull into the River Spree, but police denied that, saying the grave was that of Wessel's father and no bones had been removed.
Heydrich, nicknamed "the Butcher", headed the Reich Main Security Office under SS leader Heinrich Himmler. Adolf Hitler called Heydrich "the Man with the Iron Heart".
He ruled over Bohemia and Moravia until May 1942, when British-trained Czechoslovak agents attacked his limousine, and he died later of his injuries.
In retaliation, the Nazis destroyed Lidice village, murdering all the men and adolescent boys and deporting the women and children to concentration camps.