Children with poor language skills at age five are significantly more likely to struggle with maths at age 11, a study for Save the Children suggests.
It found 21% of pupils who struggled with language as they began school, failed to meet the expected standards in national tests when they left.
The researchers said poor language skills had an effect on all children, regardless of family background.
Factors like parents' education and poverty were also tied to attainment.
Academics at the Institute of Education analysed the progress of 5,000 children using data from the Millennium Cohort Study and the National Pupil Database in England.
Some 23% of children who struggled with language at age five also did not reach the expected standard in their Sats (national tests) in English at age 11, the study found.
Under government plans, from 2017 children who do not pass these tests will have to re-sit them in their first year in secondary school.
Gareth Jenkins, from Save the Children, says the research demonstrates for the first time the most crucial determinant of success in Sats tests is how well children can communicate when they start school.
The poorest children are more likely to start school without simple skills, such as being able to tell a short story, express feelings and communicate easily with a wide range of adults
The charity says the research should prompt a national debate about how to drive up the quality of nursery provision.
"The most important thing we could do is to see every nursery led by an early years graduate teacher, because the research is really clear that this helps improve the quality of the activities, and the poorest children in particular benefit from that extra care," he said.
It is estimated that this would need a further 11,000 graduates to be employed.
The legal minimum set out in the early years framework in England says a nursery should be led by someone with two years experience and a Level 3 early years qualification, such as an NVQ Diploma.
Most two- and three-year-olds are in nurseries run by private or voluntary organisations, with 85% being rated by Ofsted as good or outstanding.
The Pre-School Learning Alliance has estimated that employing a fully, graduate-led workforce would result in a funding shortfall of 11% for funded three- and four-year-olds, and a 19% shortfall for funded two-year-olds.
Alliance chief executive, Neil Leitch, said early years had remained a low pay sector, and although recent increases in funding were welcome there were still financial pressures.
"What is not clear, however, is how providers are expected to recruit and retain graduate-level staff without the funding needed to pay adequate wages," he said.
A Department for Education spokesman said: "The number of graduates in the workforce continues to rise, and we want to see more trained graduates in these roles.
"That's why we provide funding course fees and bursaries for eligible trainees, and are also supporting employers to help with their staff training costs.
"We continue to look at what more can be done to encourage talented staff to forge a career in the early years and this will be a key strand of our Workforce Strategy which will be published in 2016."
In the UK, a higher proportion of public spending already supports early years care and education than in other developed countries, between 0.5% and 0.9% of national wealth, according to the Institute for Fiscal Studies.
Current government policy in England is focused on making childcare more available to working parents by extending free provision in term-time from 15 to 30 hours.
The first pilot schemes, costing £13m, will begin in eight areas of England this September, with the policy being fully implemented in 2017.