Until a storm last week, there were "no signs of any major structural problems" before a wall collapsed at a castle originally built by Henry VIII, owner English Heritage has said.
A section of Hurst Castle's 19th Century east wing, near Milford-on-Sea, Lymington, fell on Friday.
English Heritage said it was already aware of the serious issues affecting the foundations of the east wing.
Work had been due to begin next week to stabilise the foundations, it added.
The charity carried out £750,000 worth of work in 2019 to stabilise the foundations of the west wing and to reinforce its sea defences.
A spokesperson said: "At the west wing, deterioration had occurred at a very slow rate and we were using that as a guide."
The organisation was aware of the "very serious issues" affecting the foundations of the castle's east wing, the spokesperson said, but "until the storm of 23 February, there was no signs of any major structural problems".
They added: "However the storm damage exacerbated the underlying issues, prompting the rapid collapse of the relevant section of the wall, days before our protective works were due to commence."
Engineers are now assessing the damage and planning remedial works, English Heritage said.
The site was closed to the public at the time of the collapse and there have been no reports of any injuries.
Local historian, Brice Stratford, said erosion at the castle has been a known issue for decades.
He told the BBC English Heritage had "dismissed the warnings and concerns" of people working at the castle site and of heritage campaigners.
Hurst Castle was originally built by Tudor king Henry VIII between 1541 and 1544 to guard Needles Passage, which is the narrow western entrance between the Isle of Wight and the mainland.
From 1860, the large east and west wing batteries were added as part of a defence programme.
The castle was also used for searchlights and guns during World War One and World War Two.