The performance gap between pupils from less well-off homes and their classmates has failed to narrow over a decade and remains a "major challenge," the education watchdog has warned.
Closing the gap has been a flagship Welsh Government policy, with £100m a year in grants to boost the performance of more disadvantaged children.
But Estyn said the disparity in attainment had not reduced.
Ministers said the gap was narrower than in other parts of the UK.
At GCSE, 32% fewer pupils who receive free school meals get five good passes, including English or Welsh and maths, compared to pupils overall.
Attendance rates are also worse for poorer pupils and this gap too has persisted over a decade, the watchdog said.
In the first of a new three-yearly assessment of trends in Welsh education, Estyn chief inspector Meilyr Rowlands said Wales did well compared to other countries on equity amongst pupils.
There had also been "some real improvements" in pupil referral units and independent special schools which often supported the most disadvantaged children.
These gaps typically widen as learners become older
"But it is a real problem to tackle the gap that poverty creates," he said.
"Clearly there are things that schools can do and the most successful schools do do it.
"A clear strategy and clear leadership and building a really good relationship with parents and with the community is a very important part of it."
The report calls for a two-pronged approach, with improved teaching and learning but also a more "community-focused" schooling system.
"These gaps typically widen as learners become older, and at least partly result from how families and the local community view education, as well as from the quality of the education provision itself," the report said.
"Pupils cannot benefit fully from improved education provision if they are not in school or not supported with difficult circumstances in their lives," it said, adding that the agencies that help schools to support pupils and families needed "management, co-ordination and resourcing".
What else did Estyn find?
- Standards are good or better in around 80% of primary schools and the proportion with excellent standards has continued to rise, with 10% receiving the highest level
- Standards are good or better in nearly half of secondary schools inspected but the proportion of these schools causing concern remains a challenge
- There have been improvements in independent special schools and pupil referral units, with examples of excellence for the first time for many years
'To be successful you have to be happy first'
At Ysgol Gymraeg Brynsierfel in Llanelli, there are regular yoga sessions to help children with breathing techniques and to calm down.
They also open up similar sessions for parents and the community.
Head teacher Jayne Davies said staff were the "eyes and ears" to anything which might be wrong, with children sometimes coming from difficult circumstances.
"Well-being is at the centre of everything - you know as an adult, if you feel unhappy or something traumatic has happened in you life, it's extremely difficult to concentrate on your work," she said.
"To be successful you have to be happy first."
Closing the attainment gap is a key element of the Welsh Government's "national mission" to improve education and it has invested heavily in that aim.
The pupil deprivation grant was introduced in 2012, evolving into the pupil development grant (PDG), and expanded for younger children and to cover costs such as uniform and kit.
In the Welsh budget for 2020-21 £101m is allocated for the PDG.
It provides £1,150 extra to schools for every pupil eligible for free school meals, and an extra £700 for eligible three and four-year olds.
Plaid Cymru education Sian Gwenllian spokeswoman said the Welsh Government could not ignore "these shocking statistics that traps another generation of our children in poverty".
She said there needed to be investment in high quality early years education and childcare to give children the best start in life "wherever they were from".
A Welsh Government spokesman said policy changes were raising standards.
"For example, last year 80% of pupils eligible for free school meals achieved at least one science GCSE, compared to just 45% in 2015," he said.
"Previously these pupils were being written off and not even entered for GCSEs, with fewer than half taking a science GCSE.
"Last year's Pisa report also showed that the attainment gap in Wales was far narrower than other UK nations or internationally, meaning a child's background in Wales has far less impact on their attainment than other countries."