The government wants to "get rid of the modularisation of GCSEs" and move back towards a single final exam, in plans to be set out later this week.
Education Secretary Michael Gove said that instead of GCSEs being examined section by section, pupils should be assessed at the end of the course.
Details of how this might be put into practice are expected in a White Paper set to be published on Wednesday.
Head teachers' unions said the plan was a retrograde step.
GCSEs are taken by pupils in England, Wales and Northern Ireland, but students in Scotland take Standard Grades.
"We want to get rid of modularisation of GCSEs," Mr Gove told the BBC's Andrew Marr show on Sunday.
"Instead of GCSEs being split into bite-sized elements we think it's important that at the end of the GCSE course the student should be examined on everything they have learnt at one time," he said.
"We'll have fewer exams but a concentration on a more rigorous approach at age 16," he added.
Mr Gove said that less time and money being spent on examinations would be a good thing.
But the Association of School and College Leaders said modular GCSEs were "highly successful qualifications" and said there was no reason why these tests should be any less rigorous than a traditional linear examination.
"Relying on performance on one day in an examination room is an outdated model which goes back to an education system which aimed solely to test recall of knowledge," said General Secretary Brian Lightman.
Dr Chris Howard, president of the National Association of Head Teachers, said teachers faced another "structural upheaval" of the system, "and that is not going to give them more time to teach, it is going to bog them down in more preparation".
On Monday, the educational charity the Sutton Trust published a report saying that the GCSE system should be adapted to become an examination for 14 year olds.
One of its authors, Professor Alan Smithers of the University of Buckingham, said a shift towards more end-of-course exams seemed to be "a move in the right direction".
"It's also been true that schools have been game-playing the modules, and re-takes mean that the exams aren't such a good comparison in what young people can do," he said, voicing one of the main criticisms of the modular system.
But he added that coursework assessment was still an important part of some practical subjects, such as art and technology.
The government is due to publish its education White Paper on Wednesday.
Mr Gove has already lifted the ban on state schools offering the international GCSE qualification - known as the iGCSE - which is assessed primarily by exams at the end of the course.
He has also said he wants a shift back to traditional exams in the A-level system, which is also currently modular.
GCSEs are offered in England, Wales and Northern Ireland by a range of exam boards.
These are regulated by separate bodies in three nations, but the regulators work closely together so changes to the system in England could also affect Wales and Northern Ireland.
Also expected to be detailed in the White Paper are:
- Plans to give teachers greater powers to search for banned items in the classroom, hand out no-notice detentions, and to clarify teachers' rights to restrain pupils physically
- Anonymity for teachers being investigated for inappropriate behaviour, to protect teachers from malicious allegations
- Plans to scrap rules limiting head teachers' ability to observe teachers' lessons to three-hours each year
- Plans to reform teacher training, including with the introduction of special teaching schools, modelled on existing teaching hospitals
- Plans for more assessments of teacher training applicants, including tests of character and emotional intelligence
- An overhaul of league tables to stop schools from using vocational exams to boost their GCSE-level scores, and to rank the proportion of pupils gaining the new baccalaureate - meaning they have obtained at least a C grade in maths, English, a language, a science subject and a humanities subject
- Slimming down the national curriculum
- Plans to introduce a reading text for six-year-olds
- Plans to overhaul the way funding is allocated to schools nationally.