A gunman shot and killed a soldier at a war memorial in Ottawa, before police engaged in a gun battle inside the parliament building.
Police confirmed a gunman was shot dead, but Canada's capital remained on alert through much of the day.
PM Stephen Harper said the country "would not be intimidated" by violence.
The shooting came hours after Canada raised its terror threat level. On Monday another soldier was killed in a hit-and-run attack by a Muslim convert.
The country earlier this month announced plans to join the US-led campaign of air strikes against Islamic State (IS) militants in Iraq.
There is no confirmation any of this week's attacks are linked to IS or the new military campaign.
But Mr Harper, speaking late on Wednesday, said the attack would strengthen Canada's resolve in the fight against extremism alongside its allies.
"They will have no safe haven," he said.
Mr Harper was addressing MPs at the time of the shooting but was safely evacuated.
Citing unnamed Canadian officials, US and Canadian news agencies identified the dead gunman as Michael Zehaf-Bibeau.
On Wednesday morning, two soldiers guarding the memorial came under fire from a man carrying a rifle.
One soldier, Cpl Nathan Cirillo, died of his injuries. Three other people were treated in hospital and released by evening.
Minutes later, dozens of shots were fired inside the parliament building, Canadian MP Marc Garneau told the BBC.
Canadian parliamentary waiter Alain Merizier described seeing a dark car stop outside parliament's centre block and a driver with "a long gun" get out and run inside the entrance of the building, pursued by a parliamentary officer.
He said: "I was astonished more than frightened. You don't have time to be afraid."
MP John McKay described the moment the gunman attacked parliament: "There was a pop, pop, pop sound so the guards ushered us to the back of the building.
"How the gunman was able to walk down the hall of honour inside parliament with a rifle will become an area of investigation."
The gunman was shot dead in a heavy exchange of fire with police. It has not been confirmed that he was the one who shot the soldier at the memorial.
In his evening address, Mr Harper gave no details about the gunman's background, identity or motive but said: "In the days to come we will learn more about the terrorist and any accomplices he may have had."
Multiple members of parliament credited Sergeant-at-arms Kevin Vickers, 58, with shooting the assailant dead.
"MPs and [Parliament] Hill staff owe their safety, even lives, to Sergeant at Arms Kevin Vickers who shot attacker just outside the MPs' caucus rooms," New Democrat MP Craig Scott tweeted.
Police told those in the vicinity of central Ottawa to stay away from windows and roofs as they searched for additional suspects.
The nearby University of Ottawa was placed on lockdown, as well as all local police buildings and the US embassy.
Marc Coucy of the Ottawa Police told the Canadian Broadcasting Corporation that officers were looking for "multiple suspects" in shootings at three locations. But police later would not confirm if there was more than one gunman.
About 20:25 local time (00:25 GMT), Ottawa police lifted the lockdown in Ottawa's city centre but said the investigation continued on Parliament Hill, which was closed to the public.
Ottawa Mayor Jim Watson labelled the events a "sad and tragic day for the city and country".
Earlier on Wednesday, White House spokesman Josh Earnest told reporters US officials had been in "close touch" with their Canadian counterparts.
In a telephone call with Mr Harper, US President Barack Obama condemned the attacks and reaffirmed the two nations' close friendship.
The elevated terror threat level came in response to an increase in online "general chatter" from radical groups including Islamic State and al-Qaeda, officials said.
Are you in the area? You can share your experience by emailing email@example.com.