Campaign groups say China has started demolishing buildings at Larung Gar, one of the largest centres of Buddhist learning in Tibet.
The London-based Free Tibet group says demolitions at the site began on Wednesday and a number of people living there have been evicted.
It follows an order last month by the local authorities to cut the number of Larung Gar residents by half to 5,000.
Chinese officials are reported to have cited overcrowding concerns.
Larung Gar is said to be the biggest Tibetan Buddhist institute in the world.
The academy and monastery, founded in 1980, sprawls over a mountainside in Sertar county in eastern Tibet, and attracts thousands of Buddhist monks and nuns who wish to study there.
The students commonly stay in log cabins and correspondents say the site has grown considerably in recent years.
The campaign group said the work team had been accompanied by Chinese police and members of the armed forces dressed in plain clothes.
There has been no formal comment by Chinese authorities.
A Sertar county government official contacted by Associated Press said the purpose of the work was to renovate rather than remove the buildings.
One student at Larung Gar was quoted by Free Tibet as saying: "If the only way to solve the overpopulation is destroying the houses, then why is the same policy not implemented in the Chinese cities and towns where the population is overcrowded?
"Where is the equality, rule by law, public welfare, religious freedom and equal rights of all nationalities (as they say) if you crush down the houses of innocent religious practitioners who are living simple lives?"
Chinese authorities have said Larung Gar's population must be reduced from 10,000 to no more than 3,500 nuns and 1,500 monks by October.
Free Tibet director Eleanor Byrne-Rosengren said: "The demolition at Larung Gar is clearly nothing to do with overcrowding - it is just another tactic in China's attempt to subvert the influence of Buddhism in Tibet."
Beijing claims a centuries-old sovereignty over the Himalayan region. Tibet has spent some of its history functioning as an independent entity and other periods ruled by Chinese and Mongolian dynasties.
China sent in thousands of troops to enforce its claim on the region in 1950. Some areas became the Tibetan Autonomous Region and others were incorporated into neighbouring Chinese provinces.
Beijing says Tibet has developed considerably under its rule. But rights groups say China continues to violate human rights, accusing Beijing of political and religious repression. Beijing denies any abuses.