Former Aston Villa assistant boss Peter Grant believes relegation would be disastrous for the Premier League club.
New chairman Steven Hollis has said the club do not fear the drop and will come back "on a stronger footing" if they find themselves in the Championship next season.
But Grant, who followed Alex McLeish from Birmingham to their West Midlands rivals in June 2011, disagrees.
"It would be a nightmare for a club of Villa's stature," he told BBC WM 95.6.
"There are 18 or 19 clubs in the Championship who have been in the Premier League and it is not as easy to automatically come straight back up.
"In one respect you have a chance to rebuild, but there is a different pressure to the Championship. Fans will expect you to win every match and it is not a simple as that."
Since Martin O'Neill resigned as boss before the start of the 2010-11 season Villa have had five managers, with current head coach Remi Garde appointed in November.
Budget cuts saw a number of senior players including James Milner, Ashley Young and Stewart Downing leave the club, with subsequent sides built around younger players.
They are six points adrift at the bottom of the table and Grant believes the fans are "delusional" if they expect the current team to be a force in the top-flight.
"They have changed [managers] too often, too quickly with the amount of young players they have," he said.
"When I talk about the fans being delusional, you've got to accept where you are at this moment in time and give somebody the confidence to build instead of being critical.
"Even if it takes you 18 months to two years you have to give that manager an opportunity to try and get a structure, a way of playing and find a winning formula."