The gunman in the parliament building was shot dead, and Harper was safely removed.
Canadian police said they could not "at this point" confirm whether the man who shot dead the soldier, who was guarding the National War Memorial in central Ottawa, was the same person who shortly afterwards attacked the nearby the parliament building.
Witnesses said at least 30 shots were fired after a gunman entered the parliament building and was pursued by police.
The assault came very near the room where Harper was meeting with members of his Conservative party, a government minister said.
"PM (Harper) was addressing caucus, then a huge boom, followed by rat-a-tat shots. We all scattered. It was clearly right outside our caucus door," Treasury Board Minister Tony Clement told Reuters.
The incident, shocking in Canada's normally tranquil capital, was not over.
Parliament and buildings in downtown remained on lockdown.
Harper stressed that government and parliament should continue its work, a spokesman said. "While the prime minister stated that facts are still being gathered, he condemned this despicable attack," the spokesman said.
Police said that an operation was under way to make parliament safe and they were still in the middle of an active investigation.
"It caught us by surprise... If we had known that this was coming, we would have been able to disrupt it," Gilles Michaud, assistant commissioner of the Royal Canadian Mounted Police, (RCMP) told a news conference.
GUNS DRAWN IN PARLIAMENT
Dramatic video footage posted by the Globe and Mail newspaper showed police with guns drawn inside the main parliament building. At least a dozen loud bangs can be heard on the clip, echoing through the hallway.
Veterans Affairs Minister Julian Fantino, a former policeman, told the Toronto Sun that parliament's head of security, Sergeant-at-Arms Kevin Vickers, shot dead a suspected gunman.
"All the details are not in, but the sergeant-at-arms, a former Mountie, is the one that engaged the gunman, or one of them at least, and stopped this," Fantino said.
Canadian cabinet minister Jason Kenney said a guard in parliament buildings was also wounded in the incident.
There was no word yet on the identity of any suspect or suspects. It was also unclear whether there was any connection to an attack on Monday when an Islamic convert ran down two Canadian soldiers with his car, killing one, near Montreal, before being shot dead by police in the first fatal attack on Canadian soil tied to Islamic militants.
Canada announced this month it was joining the battle against Islamic State fighters who have taken over parts of Iraq and Syria.
Canada said on Tuesday it had raised the national terrorism threat level to medium from low because of a rise in "general chatter" from radical groups such as Islamic State and al Qaeda but said there had not been a specific threat.
The RCMP's Michaud said the threat level on Parliament Hill had been on medium for some time.
No group, Islamic or otherwise, claimed responsibility for either the attack in Ottawa or the one near Montreal. Monday’s attacker, 25-year-old Martin Rouleau, who converted to Islam last year, was among 90 people being tracked by the RCMP on suspicion of taking part in militant activities abroad or planning to do so.
POLICE FLOOD STREETS
As the drama in Ottawa unfolded, police in dark bulletproof vests and carrying automatic rifles flooded the streets near parliament.
Some took cover behind vehicles and shouted to people to clear the area, saying: "We do not have the suspect in custody. You are in danger here."
Police quickly cleared several blocks of downtown Ottawa.
Several police cars were parked on the lawns outside the parliament buildings. Small groups of police could be seen sheltering behind at least two cars.
Members of parliament were told to lock or barricade themselves in their offices, and stay away from the windows.
At the time of the shooting, the opposition New Democrats were holding their weekly caucus meeting. A tweeted picture sent from the room showed a pile of chairs jammed up against the main door to prevent anyone from entering.
The soldier who died in the shooting was taken into an ambulance in which medical personnel could be seen giving him cardio-pulmonary resuscitation. Authorities did not identify him and it was not clear if he was armed or not when he was shot.
CBC News reported that he was a reservist who had been serving in Hamilton, Ontario.
In Washington, a White House official said U.S. President Barack Obama had been briefed on the situation. He said the United States had offered assistance to Canada.
Mass shootings are relatively rare in Canada, which has stricter gun laws than the United States, and the regulations at one point included a national registry of rifles and shotguns. Legislation was passed in 2012 to scrap the registry.
Ottawa has a low murder rate. There were nine homicides in 2013 and seven in 2012, in a city of 885,000 people. Compared with Capitol Hill in Washington, security on Parliament Hill is also fairly low key. Anybody could walk right up to the front door of parliament's Centre Block with arms and explosives without being challenged before entering the front door, where a few guards check accreditation.
Centre Block is the main building on Parliament Hill, a sprawling complex of buildings and open space in downtown Ottawa. It contains the House of Commons and Senate chambers as well as the offices of some members of parliament, senators, and senior administration for both legislative houses.
A construction worker who was on the scene in Ottawa when the shooting began told Reuters he heard a gunshot, and then saw a man with a scarf over his face running towards parliament.
"He was wearing blue pants and a black jacket and he had a double barreled shotgun and he ran up the side of this building here and hijacked a car at gunpoint," construction worker Scott Walsh told Reuters.
The driver got out safely, then the man drove the car to the Centre Block, where construction work is underway, Walsh said.
The Canadian military closed its bases across the country to the public following the events in Ottawa, CBC TV said.
Canadian and U.S. stock markets declined after the shootings in Ottawa. The Toronto Stock Exchange's TSX index .GSPTSE dropped 1.6 percent, while the S&P 500 .SPX gave back 0.7 percent.
The attacks in Ottawa and in Quebec took place as the Canadian government prepared to boost the powers of its spy agency, the Canadian Security Intelligence Service. Public Safety Minister Stephen Blaney said last Thursday the new legislation would let the agency track and investigate potential terrorists when they travel abroad and ultimately prosecute them.