Farmers in Lincolnshire have claimed losses to the cauliflower crop caused by severe winter weather may reach £7m.
The county usually supplies about 70% of the UK's crop from March to May, but many growers have lost up to 80% of their harvest.
The Brassica Growers Association said the situation was so bad it could foresee many farms giving up on cauliflower entirely.
It added that production may well move to milder parts of the country.
Phillip Effingham, chairman of the Brassica Growers Association, which also includes Brussels sprouts and cabbage, said some farms had lost all of their early varieties.
"We seem to be getting more extremes of weather lately and the actual profitability of brassica crops over the last few years is not really subsidising the risks we face.
"In these bad winters we risk losing all the crop. It's a big loss and as such its doesn't stack up financially," he said.
Robert Oldershaw, who farms at Holbeach St Marks, agreed saying: "With the weather, the dry summer and then the exceptionally cold winter with hard frosts and snow, it has been tough.
"If it continues we will have to look at what crops we grow and maybe have something with less risk."