Anti-austerity Syriza party leader Alexis Tsipras has vowed to end Greece's "five years of humiliation and pain" after his general election win.
Before cheering supporters, Mr Tsipras again pledged to renegotiate Greece's massive international bailout.
With nearly all of the votes counted in Sunday's poll, Syriza looks set to have 149 seats, just two short of an absolute majority.
Syriza's victory has raised fears about Greece's future in the euro.
The currency fell to $1.11 against the US dollar following the result - the lowest level in more than 11 years.
Greece has endured tough budget cuts in return for its 2010 bailout, worth €240bn (£179bn; $268bn) and negotiated with the so-called troika - the European Union, International Monetary Fund (IMF) and European Central Bank (ECB).
The economy has shrunk drastically since the 2008 global financial crisis, and increasing unemployment has thrown many Greeks into poverty.
Syriza's election result will send shockwaves through Europe, the BBC's Gavin Hewitt in Athens says.
A majority of voters in Greece have essentially rejected a core policy for dealing with the eurozone crisis as devised by Brussels and Germany, our correspondent adds.
The election result is expected to be one of the main issues at Monday's meeting of 19 eurozone finance ministers.
In Germany, Bundesbank President Jens Weidmann said he hoped "the new Greek government will not make promises it cannot keep and the country cannot afford".
Belgian Finance Minister Johan Van Overtveld was quoted by VRT network as saying that Greece "must respect the rules of monetary union", although he added that there was room for some flexibility.
UK Chancellor of the Exchequer (Finance Minister) George Osborne told the BBC Syriza's promises would be "very difficult to deliver, and incompatible with what the eurozone currently demands".
In Italy, EU Affairs Minister Sandro Gozi said the vote offered "new opportunities to pursue change in Europe" to create growth and investment and fight against unemployment."
But the man tipped to become the new Greek finance minister, Yanis Varoufakis, told the BBC the austerity regime had been "kind of fiscal waterboarding policies that have turned Greece into a debt colony".
At the scene: BBC's Chris Morris, Athens
On Klathmonos Square, the flags were flying high, supporters of Syriza were singing and dancing, there were hugs and tears and broad beaming smiles.
This was an extraordinary victory for the radical left in Greece - probably beyond its own expectations.
Alexis Tsipras will now try to lead an anti-austerity revolution, backed by a strong democratic mandate.
He said in his victory speech that he is willing to negotiate with Greece's European partners. The question is: how much are they prepared to compromise with him?
There are some things during the eurozone crisis that we were told would never happen.
The European Central Bank would never flood the market with new money, and Greece would never take a gamble with the radical left.
The past few days have overturned those assumptions, making this week a potential turning point in the recent history of the European Union.
Results from Greece's election commission showed a clear Syriza lead.
With most votes counted, Syriza polled 36%, while the outgoing New Democracy won 28%.
Another five parties - including the far-right Golden Dawn and centrist The River - are expected to be represented in the 300-member parliament.
A right-wing party called the Independent Greeks has already agreed to join Syriza as part of the coalition.
Mr Tsipras will go to the president later on Monday to be given mandate to form a government.
Outgoing Prime Minister Antonis Samaras has admitted defeat.
Addressing his cheering supporters in Athens late on Sunday, Mr Tsipras said "the Greeks wrote history".
"Greece is leaving behind catastrophic austerity, it is leaving behind the fear and the autocracy, it is leaving behind five years of humiliation and pain".
"Your mandate is undoubtedly cancelling the bailouts of austerity and destruction.
"The troika for Greece is the thing of the past," he added.
Mr Tsipras promised to write off half of Greece's debt, but was ready to negotiate "a viable solution" and wants the country to stay in the eurozone.
For his part, Mr Samaras said: "The Greek people have spoken and I respect their decision," pointing out that he had inherited a "hot potato" on coming into office and that he and his party had done much to restore his country's finances.
Who are Syriza and what do they stand for?
- An acronym meaning the "Radical Coalition of the Left", Syriza was formed in 2004 as an umbrella group
- Led by 40-year-old Alexis Tsipras
- The party first came to prominence following the 2008 Greek riots
- It promises an end to Greece's painful austerity measures and wants to renegotiate its debt
- Markets worry about a Greek debt default and a possible exit from the eurozone, though Syriza says it wants to keep the euro
Are you in Greece? What difference will the election result make? You can share your experiences by emailing email@example.com. If you would be happy to speak further to a BBC journalist, please include a contact telephone number.