Greig Laidlaw held his nerve to knock over a penalty in time added on as Scotland secured their first win in Australia since 1982.
Andy Robinson's side turned in a display full of guts and determination in appalling conditions.
Laidlaw kicked two penalties as the Scots led 6-3 at the interval.
Australia dominated after the break but could not find a way past the resolute visitors and Laidlaw sealed a famous win with the last kick of the ball.
The success ends a run of seven consecutive defeats for Scotland and makes it back-to-back wins over the team ranked second in the world, following a similarly dogged 9-8 triumph at Murrayfield in 2009.
The home side had an experimental look, with five new caps starting, but the victory will be a real fillip for Scotland, who play Fiji and Samoa next on tour, as well as providing valuable ranking points.
As the rain came hurtling down at the Hunter Stadium in Newcastle, Ross Rennie was first to break in front of the posts but the Scotland flanker's elaborate offload did not find a team-mate.
Loosehead prop Ryan Grant, making his Scotland debut, bulldozed his way to within a few feet of the line but the ball squirmed from scrum-half Mike Blair's grasp and the momentum was lost at a crucial moment.
With a strong wind behind them, Scotland continued to press and Robinson's men were camped inside the Australia 22.
A penalty followed when the back-pedalling home side were guilty of collapsing a scrum but Laidlaw's angled effort from 25 metres was pushed wide in the gusting conditions.
Stuart Hogg then put more pressure on the Aussie defence with a looping kick that evaded the grasping arms in gold shirts. However, winger Jospeh Tomane made a great recovery as the ball bounced over the try-line, with Joe Ansbro in close attendance.
The Scots were in front on 23 minutes when Laidlaw thumped over a penalty from 40 metres.
And a few minutes later, Laidlaw nailed another kick from similar range.
It took Australia 30 minutes to enjoy any meaningful possession in the opposition half after a knock-on from Hogg.
Paul Higginbotham was stripped of the ball as the home side inched towards the line but the Scots were guilty of not rolling away at the breakdown and Mike Harris kicked a penalty to halve the arrears.
Edinburgh full-back Tom Brown made his first appearance in a Scotland shirt as the first-half drew to a close, with Sean Lamont going off with a rib injury.
The second half was just a minute old when Australia had the chance to level and Harris popped over his kick, despite the posts wobbling like jelly in the high winds.
On 53 minutes, Harris tried his luck from the halfway line but the ball slammed against an upright underneath the crossbar level.
With the Scots under intense pressure, another penalty was kicked to touch, much to the delight of the crowd.
As the ball was worked from the set-piece, Australian replacement Rob Simmons crashed towards the line but a cluster of blue shirts did enough to prevent a score, following a review from the television match official.
A great tackle from replacement scrum-half Chris Cusiter prevented Digby Ioane from jetting away as the Australian winger gathered a cross-field kick and Alasdair Strokosch put in several great hits to repel the home team.
Fly-half Berrick Barnes then skewed a drop-goal attempt wide and Harris made a real mess of a long-range penalty.
In the final minute, Scotland made a foray into the opposition half and put the Australians under all sorts of pressure from a scrum. Another scrummage followed and this time a penalty was given.
Facing a howling gale, Laidlaw managed to slam the ball between the sticks from 30 metres to spark scenes of wild celebration from the Scots.
Australia: Morahan, Tomane, A. Faingaa, Harris, Ioane, Barnes, Genia, Slipper, Moore, Palmer, Timani, Sharpe, Dennis, Pocock, Higginbotham. Replacements: Alexander for Palmer (70), Simmons for Timani (55), Hooper for Dennis (65). Not Used: S. Faingaa, Phipps, McCabe, Ashley-Cooper.
Scotland: Hogg, Ansbro, De Luca, Scott, Lamont, Laidlaw, Blair, Grant, Ford, Murray, Kellock, Gray, Strokosch, Rennie, Barclay. Replacements: Brown for Lamont (39), Cusiter for Blair (64). Not Used: Lawson, Welsh, Ryder, Vernon, Weir.
Ref: Jaco Peyper (South Africa).