Former player Ralph Calvin has stepped down from his position as Whitehaven chairman, and from the board of directors, with immediate effect.
Calvin, also a one-time president of the Rugby Football League, rejoined Haven's board in 2009 after a previous spell in the early 1990s.
However, falling attendances have prompted his decision to withdraw.
"It's making it very difficult to sustain the club at the level it is at now," Calvin told BBC Radio Cumbria.
"Everybody was very optimistic about the game after the World Cup and the success of the games at Workington.
"The crowds had turned out for that and we were optimistic for the start of the season that it was a shot in the arm the game needed.
"But it didn't materialise and the support is still dwindling. I've had enough and feel I've done my time."
The former Cumbria representative prop also played for rivals Workington Town,
Whitehaven, who avoided relegation from the Championship in 2014 subject to Keighley's points deduction appeal, appointed Steve Deakin as head coach last month.