Plans to consider closing Dover Castle on week days in the winter have triggered fears of a knock-on effect on businesses in the town.
English Heritage is consulting staff on plans to restrict winter opening to weekends in the wake of a 32% cut in government grant.
But union officials say it could cost up to 40 jobs at the castle.
And one businessman said it could reduce income for other leisure enterprises in Dover.
English Heritage said only 13% of the castle's annual visitors go outside the summer months - and the number in January had been less than 1% of the number expected to visit during the whole of 2011.
It believes cutting the winter opening from five to two days could make better use of available staff and funds.
Tracey Wahdan, visitor operations director for the South East, said the organisation was not ruling out job losses, but was exploring the possibility of spreading some staff pay evenly throughout the year, even though they will be working fewer hours in the winter.
The proposed rearrangement could also enable English Heritage to open attractions that are normally closed in the winter, such as Deal Castle.
Mark Hodgson, of the Public and Commercial Services Union, is urging the body, which is funded largely by government, to explore ways of maximising income from its treasures.
It argues that the cost of making up to 40 people redundant could outweigh potential savings.
And Dover businessman Peter Garston told BBC Kent that local pubs and restaurants were likely to lose income as a result of fewer visitors to the area during the winter.
He said: "It's just another example of businesses... centrally run by bureaucrats not looking around them and saying what is the effect on the local communities that they serve, or should be serving, of the decisions they are making."
Ms Wahdan said no decision would be made until the end of the 90-day consultation.