A mental health patient was pushed to the floor as hospital staff used "disproportionate and unauthorised techniques", health inspectors said.
The Care Quality Commission (CQC) inspection took place after concerns were raised about patient safety at St John's House in Palgrave, Suffolk.
It was rated inadequate and banned from admitting new patients.
Priory Healthcare said it had "taken immediate action to deliver improvements".
Dr Kevin Cleary, CQC deputy chief inspector of hospitals and lead for mental health, said inspectors found an "unacceptable service" during the unannounced visit in December, "where some staff interventions had demonstrated elements of patient abuse".
St John's House, near Diss, is an independent hospital looking after adults living with learning disabilities and associated mental health issues.
A review of CCTV footage showed seven incidents including "a patient being dragged across the floor... a patient being pushed over and the seclusion room door trapping a patient's arm and making contact with a patient's head when closed".
The footage also showed on five occasions, staff were asleep when they should have been completing patient observations.
They also found staff not following infection prevention and control measures, and ward staff without access to working resuscitation equipment.
The report also highlighted low staffing, with a heavy dependency on agency workers, poor record-keeping and 204 instances of physical restraint in a six-week period.
Some patients told the inspectors they did not always feel safe on the wards because of levels of aggression from other residents.
They also said activities were often cancelled because there were too few staff, but praised individual staff who were kind to them.
St John's House will be re-inspected in six months.
A spokesman for Priory Healthcare, which runs the hospital, said it had "taken immediate action to deliver improvements at St John's House".
"We have completed the required actions for 18 of the 24 key areas identified in the report... patient safety remains our absolute priority," he added.