Sports Minister Tracey Crouch has promised to deliver a new strategy for sport "as a matter of urgency" following a decline in participation.
According to Sport England, swimming is the most badly affected, even though it is the country's most popular sport.
More than 2.5 million people swim weekly - down 144,200 in six months.
The statistics, covering October 2014 to March 2015, show 15.5 million people took part in sport once a week, every week - a drop of 222,000 in six months.
Crouch, who became Sports Minister after last month's General Election, said: "It's not the return we expect to see for a large investment of public money.
"A significant amount of public funding has been invested in sport in the last decade, but the results simply aren't good enough."
Sport England chief executive Jennie Price admitted: "These are really disappointing results.
The Active People Survey began in 2005 and results are published every six months. Click here to view the full report.
There are increases for running, tennis and basketball.
- Running up 63,000 to 2.1 million per week
- Tennis up 38,200 to 422,400
- Basketball up 21,800 to 152,900
The number of people from black and ethnic minority backgrounds playing sport has increased by 37,600 in the past six months to 2.9 million.
The other significant drop was in the keep fit and gym sector. A fall of 153,000 was the first dip since Sport England started counting.
The number of disabled people playing sport is also down 37,300 overall, in large measure due to the decline in swimming and gym.
Other notable trends?
The number of 16 to 25-year-olds playing sport has remained stable at 3.8 million per week.
Some popular disabled sports, such as football, tennis and running, have seen positive increases.
The gender gap has also reduced slightly, with 6.86 million women playing sport once a week compared to 9.15 million men.
Why are the numbers dwindling?
Amateur Swimming Association chief executive Adam Paker told BBC Radio 5 live: "We have to understand what is turning the consumer off and why it's turning that person off. Access to pools certainly is a factor.
"There has been a drop-off in the number of pools we've got in this country. We also have an old stock of pools. Almost 45% of our pools were built before the 1980s so there is an aging stock of pools.
"There are some fantastic pools before that time but they cost a lot of money to maintain and to keep at the level people expect these days."
What are the options?
"It's about taking a fresh look at things, not doing the same old things and expect new results," added Paker. "We've appointed a new head of strategy from Song Playstation who will bring a commercial end to this and see it very much as a marketing job.
"It is about changing consumer's behaviour. And the key to that is taking the costumer minded approach and really looking at the numbers."
What is the next step?
"It's a question of us understanding the data and swimmers as consumers," said Paker. "What are they looking for?
"Then it's working with our industry partners with pool operators, leisure operators, and local authorities with government to create the pool experiences that might make people return to the water."