According to the Associated Press, the opposition held the Conservatives in contempt of Parliament for failing to disclose the full financial details of Harper's tougher crime legislation, corporate tax cuts and plans to purchase stealth fighter jets.
Legislators in the House of Commons voted by 156-145 to back the motion, which was presented by the main opposition Liberal Party, according to Reuters.
Canadian analysts reportedly expect an election to be called for May 2.
Polls suggest Harper would win re-election though without a majority. He would then govern with a minority in Parliament, dependent on opposition votes.
According to the BBC, recent polling suggests the Conservative Party holds a lead going into the campaign period, with the Liberal Party in second, the New Democratic (NDP) Party third and the Bloc Quebecois, which campaigns only in Quebec, fourth.
According to the AP, there is a slight chance the left-of-center parties could join forces in a coalition.