Sports Minister Helen Grant has told an inaugural women's sport conference that "we need to do a lot more" to improve female participation.
The MP addressed the Transforming Sport conference at Lord's on closing a "two million gender gap" between men and women playing sport.
Sport England launched its national campaign This Girl Can on Thursday.
"Young women have real issues about how they're judged when playing sport," chief executive Jennie Price said.
"That is a barrier we want to break down.
"We want to create a new way of thinking and talking about sport."
Grant also responded to the suspension of John Cummings, the Northumberland County FA vice-president who told a female referee that "a woman's place is in the kitchen and not on a football field" - describing his comments as "totally unacceptable".
She said it showed there was still work to do despite progress being made.
Earlier this week, a BBC prize money survey revealed that 30% of sports do not pay women equally to their male counterparts.
The Transforming Sport conference looked at issues such as participation, media, role models and sponsorship that can shape the future of women's sport.
Grant, who said she was "very proud" to be attending, said: "Record numbers of women aged between 16 and 25 play sport once a week but there is still a two million gender gap between men and women playing sport and that needs to change.
"We've got to listen to what women and girls want and give it to them."
Key athletes in attendance included Sochi 2014 skeleton gold medallist Lizzy Yarnold, Rugby World Cup winner Sarah Hunter and former world cycling champion Victoria Pendleton.
As part of the conference, the Department of Culture, Media and Sport's Women and Sport Advisory board released its interim report on the challenges facing women in sport.
It collated figures to show 1.97m more men than women play sport in England and 48% of girls feel sweating "isn't feminine".
However, it stated the number of women playing sport has risen by 62,800 to 6m this year and women received 40% of sporting honours in 2014 up from 31% in 2013.
|How far has women's sport come?|
|British Fed Cup team captain Judy Murray: "When Jamie and Andy were six and seven, I applied for a personal coaching award scheme organised by the LTA. There were 20 places available and two women were given places. A man complained asking 'what on earth has she got to offer in performance coaching when she had two young kids?' Funny how it's never mentioned now!"|
|Chief executive of Women in Sport Ruth Holdaway: "We've made really significant progress and there's a lot to celebrate and cheer about. However... at every level, there's still a gap."|
|Baroness Sue Campbell, head of the Youth Sport Trust: "We have to get this right in schools. We've still got a long way to go for those kids who are not competitive but who want to do activity and will develop their confidence through that. That's when we've got real participation."|
|Lizzy Yarnold, Sochi 2014 Olympic champion: "It was the role models that were set for me, seeing Denise Lewis at the Olympics - her being a complete professional and being magnificent and someone totally in control. I didn't even question it. Why wouldn't I be able to do that for my career? Why shouldn't I?"|
|Triple Olympic cycling medallist Victoria Pendleton: "I was told to be less open about my emotions because it made me look weak in front of my competitors."|
|Listen to more opinions from BBC Radio 5 live's women in sport special 'Game Changers'|