A so-called HS3 high-speed rail link between Manchester and Leeds could help to create a "northern global powerhouse", George Osborne has said.
He told the BBC better road and rail links would allow cities across northern England from Liverpool to Hull "to take on the world", as London did.
The chancellor said the plan could cost up to £7bn - but could be cheaper if existing rail lines were updated.
Labour said "nobody" believed the Tories could deliver jobs in the north.
Mr Osborne told BBC Radio 4's Today programme - before a speech in Manchester - that the cities in the North of England were individually strong but were "collectively not strong enough".
He said that in the past few decades giant global cities, such as London, had emerged - and that the string of northern cities, with better transport links and careful planning, could take them on and be "greater than the sum of their parts".
Mr Osborne said the building of the east-west link should be considered as part of a review into the second phase of the £50bn HS2 high-speed rail project.
The current plan for the first phase of the project between London and Birmingham has proved controversial. Some residents are set to be disrupted and there is criticism of its price tag.
The government's preferred route for the second phase involves extensions linking Birmingham with Manchester and Leeds - with the final route expected to be chosen by the end of this year.
Mr Osborne said in his speech that it was not "healthy for our economy, not good for our country" if "the powerhouse of London dominates more and more".
Along with improving roads - the M62 already links Liverpool on the west coast with Hull on the east coast, via Manchester and Leeds - Mr Osborne says a new high-speed rail link should be considered, based on the existing rail route but with new tunnels and infrastructure.
The fastest rail services between Leeds and Manchester currently take about 50 minutes - already quicker than many journeys across London.
The plan would be to cut this to 30 minutes, with trains travelling at up to 140mph, compared with the current maximum of 90mph and the 225mph maximum speed for the fastest bit of HS2.
"We need an ambitious plan to make the cities and towns here in this northern belt radically more connected from east to west - to create the equivalent of travelling around a single global city.
"I want us to start thinking about whether to build a new high-speed rail connection east-west from Manchester to Leeds."
A spokesman for Nick Clegg welcomed Mr Osborne's plan, adding that the deputy prime minister had long believed Manchester, Leeds and Sheffield "should become a 'Northern golden triangle' to drive economic growth in the North".
He said: "George Osborne's new found commitment today is welcome and we are now hopeful that he will match his words with action by backing the Leeds City Region growth deal and its proposals for better transport links to other cities across the North."
But CBI deputy director-general Katja Hall gave a cautious welcome: "Better east-west links in the north could provide a huge boost to local businesses, and help further balance the UK economy by creating a northern hub.
"However, any proposals need careful planning. The priority must be boosting connections, not speed. We need to see more detail for such a project, which must be good value for money for the taxpayer."
BBC political correspondent Vicki Young says Conservative support in the North of England has been declining and Mr Osborne's statement will be seen as an attempt to show the party is keen to increase prosperity beyond its traditional strongholds in the south.
There is, however, little detail about how these plans would be funded, and Mr Osborne said he did not yet have timescales - but he wanted "to start a conversation".
Mr Osborne also told the BBC he wanted to see new positions of elected mayoralties being created - including one for Greater Manchester along the lines of London's - to help drive forward economic growth in the north.
For Labour, shadow chancellor Mr Balls said regional growth divides had "widened markedly since 2010", when the coalition government was formed.
"On high-speed rail, we said months ago that we need value for money for the taxpayer and to improve the existing plans to maximise the benefits for the whole country, and strengthen the links between northern cities.
"Ministers need finally to start listening."