Almost 2,000 swimming pools in England could be closed by 2030 without urgent government action, a Swim England report has suggested.
The number of quality facilities could drop by 40%, limiting access for competitive and recreational swimmers.
It adds that pools built in the 1960s and 70s have not been refurbished at a sufficient rate.
Without investment of £1bn, it says there will be a "huge decline" in the availability of pools.
Team GB secured their best ever medal tally in the pool in Tokyo with eight, including four golds, while ParalympicsGB also secured 17 medals in the pool at Tokyo 2020, five of which were gold.
Latest figures from Swim England show that 4.7 million people swim at least twice a month, making it one of the most popular physical activities in the country - their predictions suggest that 3.86 million people could be pushed out of swimming.
Swim England suggests that the issue has been exacerbated by the coronavirus pandemic, with 206 swimming pools already having closed.
The report, A Decade In Decline: The Future of Swimming Pools In England, comes on National Fitness Day as research emerges from UKActive that less than a third of Britons are happy with their general fitness levels and 27% are less fit than they were before the pandemic.
It estimates that before the end of the decade the number of pools will have dropped from 4,336 to around 2,468, if sufficient intervention is not made.
That equates to almost a quarter of local authorities in England having a deficit of at least one average-sized pool.
It is estimated that 23% of local authorities have a shortage of at least one swimming pool but that figure could increase to 73% before the end of the decade.
Swim England chief executive Jane Nickerson said: "Swimming pools are naturally essential to the future of all our aquatic sports if we are to nurture the next Adam Peaty, Tom Daley, Anna Hopkin or Maisie Summers-Newton.
"Pools are hubs of the local community, helping people of all ages to lead healthier, happier lives and saving the NHS hundreds of millions of pounds each year.
"They are also the places where millions learn a skill that could one day save their life - or someone else's.
"Without adequate investment into the new pools this country needs, we are forecasting a huge decline in the available water space by the end of the decade, threatening the future of our sports, leaving millions shut out of the activities they love and widening health inequities."
However, the government has defended its record on spending.
"We have been clear that the nation's health and fitness is a priority as we build back better from the pandemic," a Department for Digital, Culture, Media and Sport spokesperson said.
"We have provided an unprecedented £1bn to ensure the survival of the grassroots, professional sport and leisure sectors. This includes our £100m National Leisure Centre Recovery Fund which has secured the survival and reopening of more than 1,100 swimming pools across the country."
They added Sport England had also provided over £8.5m to swimming and diving projects, and over £16m to Swim England since 2017 - however, none of that funding is for building or refurbishing pools.
- How has Wayne Rooney dealt with anger? Listen now as he tells Tony Bellew what makes him see red
- Can Emma Raducanu's win halt participation decline? The Sports Desk Podcast goes beyond the headlines to find out