The maternity unit at Milton Keynes Hospital has been given a clean bill of health following the deaths of three babies in as many years.
In April 2010 the trust that runs it was put on special measures but now the last three conditions placed on its licence have been lifted.
The Care Quality Commission (CQC) was satisfied that the hospital had improved maternity care in key areas.
But a separate CQC report raised concerns elsewhere in the hospital.
Inspectors were satisfied that the hospital was properly detecting and monitoring "at-risk" mothers and babies.
Curtains 'too short'
They also found appropriate plans were in place to cope with times of high demand and that enough senior midwives and obstetricians were available to cope.
However, an inspection in January raised concerns that on ward 20 in particular - for female gastrointestinal medicine and surgery - the hospital was not meeting essential standards.
Inspectors found patients' privacy and dignity could not always be respected because curtains around some beds were not long enough.
Patients were not given enough information to help them understand care and treatment options.
Patients and visitors were also judged to be at risk because just under half of staff had not attended infection prevention and control training.
Records 'not secure'
Inspectors found staff did not follow proper procedures, including removing wrist-watches and rolling up sleeves, and raised concerns that records were not always being stored securely or kept confidential and up to date.
Roxy Boyce, regional director of the CQC in the South East, said: "It is clear that Milton Keynes Hospital NHS Foundation Trust has made considerable progress to address the issues that we had raised in the past with the maternity services.
"However, we were disappointed that the trust was not meeting all essential standards of quality and safety.
"These are the standards of care anyone should be able to expect in any hospital.
"We will return to the trust in the near future to ensure that improvements have been made."
Mark Millar, interim chief executive of the trust, said he was pleased CQC had recognised improvements in maternity, and thanked staff for their efforts.
And he said the latest concerns were already being addressed.
"Patients are our top priority, and we will always take any concerns that standards are not at the level our patients would expect very seriously," he said.