Some children are not being educated because local authorities are often unable to track youngsters who are not being taught, inspectors have warned.
Ofsted blames bureaucratic failings and is not clear how many are at risk.
None of 15 councils it surveyed in England was "confident" it knew the whereabouts of every child in its area.
Ofsted said children out of education and whose whereabouts were unknown risked failing academically and were vulnerable to harm.
That could be physical, emotional or psychological harm, Ofsted said.
Local authorities have a legal duty to ensure all children are in a safe environment and get a suitable education, either at school or home.
Ofsted said schools needed to talk to each other and local authorities more effectively to ensure children who were at risk received an education and were protected from mistreatment.
For its report, entitled "Children missing from education", Ofsted surveyed 15 local authorities of different sizes across England in urban and rural areas.
It found that children sometimes go missing because schools are not fully aware of policies and procedures for informing their local authorities when a child is suspended or excluded from school.
However, even when these are understood, some schools do not comply with the guidelines set out, it said.
Ofsted found that all the local authorities it surveyed worked to forge links with others such as health professionals.
However, the extent and effectiveness of the exchange of information varied from one council to another.
In addition, information sharing between local authorities about families moving in and out of areas was too varied.
Ofsted's director for education and care, Patrick Leeson, said: "Local authorities and their partners need to share information effectively and more systematically to identify children and young people who are missing from education, particularly when their whereabouts are unknown.
"Children who become lost to the system not only risk failing academically but can be exposed to vulnerable situations.
"It is of serious concern that some schools are not following agreed procedures and legal requirements for notifying their local authorities when pupils are excluded or taken off the roll."
However, he added that the report did highlight some of the good work local authorities had done to "overcome the challenges in ensuring all the children living in their area are identified and are receiving a suitable education".
The death of seven-year-old Khyra Ishaq, in 2008, was a high-profile case of a child suffering from neglect after falling out of the education system.
She died five months after her mother had withdrawn her from school.
The severely malnourished child weighed just 2st 9lb (16.5kg) - when paramedics found her emaciated at her home in Handsworth, Birmingham.
Earlier this month ContactPoint, a £235m government database containing the records of England's 11 million children, was switched off.
The database was established in the wake of the Victoria Climbie child abuse case to aid child protection and its demise sparked concerns that there is no system of collating key information in one place.
The system was used by doctors, social workers, schools, charities and other individuals involved in the protection of children.