The current system of paying for road use should change, according to more than half the people surveyed by the Department for Transport.
Of 2,798 adults surveyed, 80% said congestion was a serious problem.
Over half those surveyed said a charging scheme based on travel times and specific routes would not reduce congestion.
Instead, they said the amount paid should reflect how often, when, and where people use roads.
The survey suggested that nearly 25% of drivers use motorways less than twice a year and fewer than 20% use motorways at least twice a week.
Ninety percent said it was important the government tackled congestion.
The survey also suggested that 88% of car drivers use their vehicle at least twice a week, and most drive every day.
The majority of road journeys occur during peak times, although half of those who travel to and from work during peak times said it would be easy for them to travel outside these periods.
Just 30% of those surveyed said that road congestion was rarely a problem for them.
About 38% said they would be prepared to accept road pricing as long as there was no overall increase in the amount paid by motorists as a whole. This is compared to 41% in a similar survey two years ago.
Almost half said that money raised from such a scheme should be spent solely on roads and transport, while more than 10% said that they did not agree with it under any circumstances.
The head of AA roads and transport policy, Paul Watters, said: "It is a failure of the UK's road network that so much traffic on A-roads, and smaller, could be moving on motorways were it not for the fear of congestion and unreliable journeys.
"Motorways are supposed to be relieving other roads but, in many drivers' minds, this is clearly not happening."