Toshiba's chief executive and president Hisao Tanaka has resigned after the company said it had overstated its profits for the past six years.
He will be succeeded by chairman Masashi Muromachi, with vice-chairman Norio Sasaki also stepping down.
On Monday, an independent panel appointed by Toshiba said the firm had overstated its operating profit by a total of 151.8bn yen ($1.22bn, £780m).
The overstatement was roughly triple an initial estimate by Toshiba.
The company's business empire stretches from home electronics to nuclear power stations.
"It has been revealed that there has been inappropriate accounting going on for a long time, and we deeply apologise for causing this serious trouble for shareholders and other stakeholders," the company said in a statement.
"Because of this Hisao Tanaka, our company president, and Norio Sasaki, our company's vice chairman... will resign today."
Mr Tanaka told a media conference that "we have a serious responsibility", adding that the company would need to "build a new structure'' to reform itself.
Mr Tanaka, 64, and Mr Sasaki, 66, both joined Toshiba in the early 1970s.
Mr Sasaki served as Toshiba president between June 2009 and June 2013, covering most of the period during which the firm inflated the profits.
Atsutoshi Nishida, an adviser and former chief executive from 2006 to 2009, also gave up his post.
Mr Tanaka and his predecessors are among eight high-level executives who have now resigned after the independent report found senior management involved in a scheme to inflate profits over several years.
People took to social media to express their concern at the scandal.
One twitter user remarked "It's appalling how long this cover up could have carried on", while another said "clean out the entire company! Toshiba needs to carry on its legacy properly."
He also said it was the most damaging episode in Toshiba's 140-year history. The company was created by a merger in 1938, but its roots date back to 1875.
Toshiba's accounting scandal began when securities regulators uncovered problems as they probed the company's balance sheet earlier this year.
One line that the investigators looked into was that executives set unrealistic targets for new operations after worries that the 2011 Fukushima disaster may hit Toshiba's nuclear division.
While the report did not specifically refer to Fukushima, it did say that pressure within Toshiba was strong in the accounting years of 2011 and 2012.
The findings mean Toshiba will have to restate its profits for the period between April 2008 and March 2014. It is unclear whether it will affect the company's results for the year ending March 2015.
The finance minister, Taro Aso, said the case could undermine confidence in corporate governance in Japan.
He added the accounting irregularities at Toshiba were "very regrettable".
Japan's government has been trying to regain global investors' confidence with better corporate governance after Olympus was found to have covered up $1.7bn in losses in late 2011, in what was until now Japan's worst corporate governance scandal.
Tomoaki Nakamura, research vice president at market research firm IDC Japan, said it was not a surprise that Mr Tanaka, along with the other executives had stepped down.
"In Japan, this news has been in the media for two months already," he told the BBC from Tokyo.
"What they need to be afraid of is criminal action by the [US] Securities and Exchange Commission."
The report's findings are expected to lead to the restatement of earnings, a board overhaul and potentially hefty fines for Toshiba.
The inquiry found that the misreporting of profits began after the financial crash seven years ago, when senior managers began imposing unrealistic performance targets.
"Within Toshiba, there was a corporate culture in which one could not go against the wishes of superiors," the report said.
"Therefore, when top management presented 'challenges', division presidents, line managers and employees below them continually carried out inappropriate accounting practices to meet targets in line with the wishes of their superiors."
One business expert, Loizos Heracleous, Professor of Strategy and Organisation at Warwick Business School, told the BBC there was a wider problem in Japanese business culture.
"Corporate culture in Japan is hierarchical and based on a long history of emphasis on loyalty, doing one's best, and doing all that is possible to avoid bringing shame to one's group," he said.
"These values, combined with unrelenting performance pressures from the market, can sometimes tempt executives to take shortcuts, and can also make it difficult for employees to ask embarrassing or probing questions of executives."
While the resignations were expected, Mr Nakamura said it was still surprising that such a giant tech firm like Toshiba was still operating in the "old way of cheating today".
Regulators are believed to be starting their own review of Toshiba's book-keeping, based on Monday's report.
Shares in Toshiba rose 6% in Japan as the report into the accounting problems ended speculation as to the extent of the problem.
However, Toshiba's shares are still down by about a quarter since the company first disclosed cases of accounting irregularities in April.
Losing to competitors
Karissa Chua, analyst of consumer electronics at market research firm Euromonitor said Toshiba's struggles highlighted the problems that have plagued many Japanese tech firms in recent years.
"With the exception of cameras, Japanese firms are no longer able to compete effectively with the likes of companies like Samsung and Lenovo," she said.
"Even without the scandals, these tech firms are already losing its competitiveness against its Korean and Chinese competitors."
And whether Toshiba can bounce back from the probe is up for debate, according to analysts.
Mr Nakamura of IDC said even though it faces a class action lawsuit in the US, Toshiba's computers, heavy industry and semi-conductor businesses are strong and profitable.
"A new leader and governance structure will renovate the company," he said.
But Ms Chua of Euromonitor said Toshiba was too far behind its competitors to reverse its declining fortunes in consumer electronics, the sector it is best-known for.
"It should instead focus on speeding up its restructuring plans and divert its resources in consumer electronics into its core business - energy and components," she said.
She said Toshiba would find it a challenge in negotiating new contracts with its partners and customers, as they would be hesitant about committing to the firm because of the uncertainty around its future, including potential fines.