Only about a third of students who took Labour's flagship diplomas in England had managed to achieve them two years on, Ofsted has found.
Inspectors said the vocational parts of courses inspired many learners, but that they had been let down by English and maths teaching.
The report said the diplomas, which combine practical and academic learning, were too complicated.
And many students failed to understand what was required of them.
Diplomas are alternatives to GCSEs and A-levels and offer learners the opportunity to develop vocational skills using industry-standard equipment.
Schools, colleges and businesses team up to offer the qualifications with students travelling between them to attend their courses.
Former Schools' Secretary Ed Balls said he hoped they would one day replace A-levels as the "qualification of choice".
But the take-up has been slow with just 11,000 students starting them in the first year. And of these, only 4,000 had gained the qualification after two years.
Some 36,000 learners started courses in 2009-10 which are yet to finish.
However, the Ofsted evaluation of the qualification found some elements worked well.
This was especially true of the main subject area of the qualification known as "principal learning".
Many learners responded well to the opportunity for active and practical learning and managed to develop their independent learning in this area. And behaviour and engagement was good in most places visited by inspectors.
But the part of the course covering English, maths and ICT or "functional skills" was "highly problematic", according to Ofsted.
This part was often taught in isolation, mostly back at schools and colleges, and many students did not realise that "functional skills" were an important part of their course.
Most thought that the "principal learning" - the specialist learning about their chosen subject - was the only part of their diploma.
However, many supporters of the diploma say it is this part of the qualification that makes the courses more demanding than other options.
Ofsted chief inspector Christine Gilbert said: "In some areas diplomas are starting to provide young people with valuable learning opportunities and work-related skills.
"However, a number of problems stand in the way of more young people benefiting from the high quality learning seen on some of the visits for this survey.
"The complexity of the qualification is proving a real challenge to both learners and institutions, with only just over a third of the first cohort gaining a diploma after two years."
She said more needed to be done to improve the teaching of functional skills and make it more joined up with the main subject content.
The report evaluated the success of the diplomas in their second year of operation by visiting 21 of the groups of schools, colleges and employers offering the qualifications.
The first five diplomas, in information technology, health, construction, media and engineering were introduced in 2008, and schools began offering a further five, in business, environmental studies, hair and beauty, hospitality and manufacturing last year.
A Department for Education spokesman said: "Ministers have already made a number of changes to the diploma to make it easier to offer this qualification.
"Professor Alison Wolf has also been appointed to lead the independent review of vocational qualifications for 14 to 19-year-olds, and we look forward to seeing her report next year."
National Association of Head Teachers general secretary Russel Hobby said Ofsted's conclusions about diplomas resonated with his members' experiences.
"Learners and teachers are enthusiastic but find the system complex. We can only be thankful that they didn't get the uptake initially projected!
"The report raises doubts about the capacity to expand further; a fear compounded by decisions taken by the government in recent months. It has dammed the diploma with faint praise whilst reducing levels of centrally provided support."