Concerns of a risk to patient safety at Dr Gray's Hospital in Elgin must be tackled "quickly and effectively", the Royal College of Midwives in Scotland has said.
A review of maternity services described clinical governance as "not fully functional", and "dysfunctional and damaged" working relationships.
NHS Grampian said it wanted a "safe and sustainable" system.
The RCM in Scotland said it was "imperative" issues were addressed.
Maternity services at Dr Gray's in Elgin were downgraded in July 2018, with some women being sent to have their babies elsewhere.
The externally-led review was commissioned after fears were raised over safety.
It found "no evidence of a trend in adverse outcomes for patients".
The report recommended NHS Grampian undertakes a "full and detailed review of clinical cases to assure themselves of the historical outcomes for individual patients" within the Dr Gray's maternity service.
NHS Grampian said it accepted the report's recommendations.
'Highest quality care'
The RCM's Scotland director Mary Ross-Davie, said: "It is clear from the review that there have been significant cultural and professional problems within the hospital which have unacceptably put women and babies at risk.
"It is imperative that the work already begun by NHS Grampian to address these problems is progressed quickly and effectively.
"Women and families in Moray deserve the highest quality maternity care."
A confidential staff helpline has been set up for anyone with issues or concerns on 01224 556705.
A special telephone number for the public is available on 01343 567443.