Team boss Ross Brawn said Mercedes' first pole position since 1955 was a "fantastic achievement".
Nico Rosberg out-qualified the field by 0.5 seconds as the team took their first pole since returning to Formula 1 as a constructor in 2010.
He will start the Chinese Grand Prix in front of team-mate Michael Schumacher following a grid penalty for McLaren's Lewis Hamilton, who was second.
Rosberg said: "It really shows that as a team we're moving forward."
Rosberg's stunning lap drew praise from several drivers, including Hamilton and Red Bull's Sebastian Vettel and Mark Webber.
The 26-year-old German set the time relatively early in the final qualifying session and stood in the Mercedes garage watching his rivals trying to beat his time on their final runs.
"The lap time was very, very strange," he said. "I had no idea if the track got a lot better or what was going on so I was just hoping, 'Come on, let that be enough please', and it was."
Schumacher had gone on the radio after his own qualifying lap to ask what Rosberg's time was, and after being told, he said: "Not bad."
Afterwards he said: "That was a phenomenal lap time for Nico. He's known as a good qualifier and there's no reason for me to be unhappy - in fact the opposite. It's going to be the first time the Mercedes Silver Arrows have been on the front row for I don't know how many years. I'm very proud of this one."
It is the first time Mercedes have taken a pole position as a constructor since the 1955 Italian Grand Prix, which was also an all-Mercedes front row with Juan Manuel Fangio ahead of Stirling Moss and Karl Kling.
After Fangio won his second consecutive world title in 1955, the company pulled out of F1 at the end of that season, as a result of the disaster at the Le Mans 24 Hours in June, when a Mercedes plunged into the crowd in flames and more than 80 spectators were killed.
Mercedes returned to F1 as an engine supplier with Sauber in 1993, and then McLaren in 1995, and became a constructor in its own right when it bought the Brawn team after they won both drivers' and constructors' world championships in 2009.
Their best results so far have been three podiums for Rosberg in 2010. This year they have scored only one point for a 10th place by Schumacher in Malaysia.
The achievement of locking out the front row in China is not without controversy as Mercedes are running a design feature on their cars that many rivals believe is illegal.
Race stewards rejected a protest against the system by the Lotus team on Thursday.
The system, known as the double DRS, links the overtaking aid on the rear wing to the front wing to increase straight-line speed. It is especially powerful around the Shanghai track because of the long back straight.
Mercedes have been pretty competitive in qualifying this year, with Schumacher taking fourth and third places in the first two races in Australia and Malaysia.
But the car has not been as strong in races because the team have struggled with tyre usage and Schumacher admitted there was a chance he and Rosberg would go backwards in the grand prix on Sunday.
"There's certainly a question mark over whether we can keep our positions from qualifying," the seven-time world champion said, "but we'll try our best."
Brawn said: "At the back of our mind, we know we've got to put the race together on Sunday.
"It was an exceptional lap from Nico. He's not had a great first couple of races. We saw today what he can do. His middle sector was incredible. Michael's lap was very good.
"We backed off when we realised his place was safe [for second] because of Lewis's penalty. We wanted to save a set of tyres."