A doctor would have been interviewed for rape and child cruelty over claims a "truth drug" was used to carry out abuse at a hospital, had he been alive.
Dr Kenneth Milner ran Aston Hall psychiatric hospital in Derbyshire from 1947 to the 1970s, which former patients described as "pure hell".
A report by police found 65 children were allegedly drugged to a zombie-like state, stripped and abused.
Police said there would have been "sufficient evidence" to pursue a case.
The vast majority of the allegations were made against Dr Milner, who died in 1975, and police said no inference should be drawn from the report other than the doctor would be questioned if he were here today.
Other staff members accused of physical abuse have either been eliminated from the inquiry, have died or were unable to be identified.
The Derbyshire force, under the direction of the area's safeguarding children's board, started an investigation in February 2016 after concerns were raised about the hospital which closed in the 1990s.
Allegations first came to light in 2011 then 2014 and 2015. No allegations of sexual abuse were identified in 2011 by police.
Then in July 2014 an individual made an allegation of rape followed by another person alleging abuse at the hospital in September of that year.
In July 2015 another person came forward and said she was injected, gassed and woke up feeling sore between her legs while at the hospital.
Some children, who were sent to the hospital between the 1950s and 1970s, also claimed they were put in straitjackets.
As well as the sexual abuse, the report found claims of patients being hit while trying to resist treatment, one having her head repeatedly forced under water, and another having her hair repeatedly pulled to force her to take medication.
Aston Hall was opened in the 1920s for those deemed to have mental health problems. It could cater for about 100 children of both sexes, but took adults as well.
Allegations centred on the use of sodium amytal as part of "narcoanalysis" - the uncovering of repressed traumatic memories.
But stories emerged, in part via social media, of children being isolated, stripped and abused sexually, physically and mentally.
Police collated 115 witness statements, recorded 77 crimes - including 33 instances of physical abuse and 40 sexual - and heard from 65 alleged victims.
Stephen Edwards, from law firm Bond Turner, which is representing 47 patients sent to the hospital in the 1960s and 1970s, said the "horrific treatment" had "undoubtedly caused permanent, damaging effects".
"Every doctor should have to demonstrate their accountability in treating patients, especially those as vulnerable as children," he said.
After trying to take his own life Jason Stubbings, from Cornwall, was sent to Aston Hall in the 1960s, aged 14, where he said he was injected with drugs and abused.
He said he remembers being injected with what he now knows is sodium amytal, also known as the truth drug, as well as another drug which is still unknown to him.
Mr Stubbings said he was then taken back to a memory where his brother was beating him with a garden cane and it felt like he was being hit at the time.
When he woke up, he found his hands had been tied and he had marks on his body.
Mr Stubbings also remembers looking out the window once and seeing "a guy in the exercise yard in a straightjacket screaming" and thinking he did not know if he was ever going to get out of there.
"It said on the administration form 'a slight timid boy', and they did that to me," he added.
The report said there were consistent allegations of drugs being given to restrained children, with limited or no records being taken.
Current medical opinion was these methods were not acceptable, even by the standards of the day, it added.
But officers said the length of time since the alleged crimes had proven a major issue - to the extent they had been unable to find out whether some members of staff were still alive.
The report said: "Having regard to these difficulties and the scale of the alleged physical and sexual abuse that has emerged in the evidence; we do not consider that it would be either practicable or appropriate for us to attempt to reach firm conclusions on each specific allegation.
"There would have been sufficient evidence to justify interviewing Dr Milner under caution in relation to a number of potential offences."
The report said the offences would have namely been "rape, indecent assault contrary to the Sexual Offences Act 1956, Child Cruelty and Assault contrary to Children and Young Persons Act 1933 and Offences Against the Person act 1861 respectively".
While it noted children were sent to Aston Hall from a variety of homes, the report found no evidence of a conspiracy to supply children for abuse.
Barbara O'Hare, who claims she was abused by Dr Milner in the 1970s, told the BBC the report was positive.
"It gives recognition to what happened to me and others at the hospital and I'm very grateful to the police," she said.
Derbyshire Safeguarding Children board said the treatment of children had radically changed and if a similar situation occurred today, there were clear routes for them to raise concerns.
At least 30 former patients who have submitted a claim for compensation to the Department for Health through a solicitor. Other claims are also believed to have been submitted.
A spokesman for the department has said the matter was under investigation.
An NSPCC spokesman said the report was "truly harrowing reading and shines a light on the decades of sickening abuse".
"It is crucial that lessons continue to be learned from shocking cases like this."