Families, the old and young walked slowly, with many holding portraits of the opposition politician and former deputy prime minister who was shot dead while walking home from a nearby restaurant on Friday night.
The authorities have suggested the opposition itself may have been behind his shooting in an attempt to create a martyr and unite the fractured movement.
His supporters have blamed the authorities.
"If we can stop the campaign of hate that's being directed at the opposition, then we have a chance to change Russia. If not then we face the prospect of mass civil conflict," Gennady Gudkov, an opposition leader, told Reuters before the march.
"The authorities are corrupt and don't allow any threats to them to emerge. Boris was uncomfortable for them."
His murder has divided opinion in a country where for years after the Soviet Union collapsed many yearned for the stability later brought by former KGB agent Vladimir Putin.
A small but active opposition now says Putin's rule has become an autocracy that flaunts international norms after Russia seized Ukraine's Crimea peninsula last year, fanned nationalism over the separatist war in eastern Ukraine and clamped down on dissent.
"(Nemtsov) was harmful to the authorities, but the authorities themselves are criminal. The authorities have trampled on all international rights, seized Crimea, started war with Ukraine," said Yuri Voinov, an elderly physicist.
Police said 21,000 people attended the march. The organizers put the numbers at tens of thousands, but attendance appeared smaller than the 50,000 people the opposition had hoped for.
Reuters reporters at the march estimated the numbers in the tens of thousands.
People walked in the rain within view of the Kremlin's red walls and past the spot, now covered in flowers, where Nemtsov was shot dead.
Some carried large banners carrying Nemtsov's face reading "Heroes Never Die", the same slogan used in Ukraine to celebrate more than 100 people killed in protests that overthrew Moscow-leaning President Viktor Yanukovich a year ago.
One elderly woman, her hair tucked into a woolen cap, held up a hand-written sign to cover her face: "It's a geopolitical catastrophe when a KGB officer declares himself president for life. Putin resign!"
Putin remains dominant
Putin has vowed to pursue those who killed Nemtsov, calling the murder a "provocation".
National investigators who answer to the Russian leader offered a 3-million-rouble reward, around $50,000, for information on Nemtsov's death. They say they are pursuing several lines of inquiry, including the possibility that Nemtsov, a Jew, was killed by radical Islamists or that the opposition killed him to blacken Putin's name.
Nemtsov's funeral is due to be held on Tuesday in Moscow.
Putin's opponents say such suggestions, repeated over pro-Kremlin media, show the cynicism of Russia's leaders as they whip up nationalism, hatred and anti-Western hysteria to rally support for his policies on Ukraine and deflect blame for an economic crisis.
"We are told on TV that a conspiracy by the West and those among us who have sold out to them are behind our poverty. People should throw away the TV set and go to protest," said Olga, 42, who declined to give her last name.
Some Muscovites have accepted the official line and appear to agree that the opposition, struggling to make an impact after a clampdown on dissent in Putin's third spell as president, might have killed one of their own.
"The authorities definitely do not benefit from this. Everybody had long forgotten about this man, Nemtsov ... It is definitely a 'provocation'," said one Moscow resident, who gave his name only as Denis.
Some young people walking in central Moscow asked: "Who is Nemtsov anyway?"
Nemtsov, who was 55, was one of the leading lights of a divided opposition struggling to revive its fortunes, three years after mass rallies against Putin failed to prevent him returning to the presidency after four years as prime minister.
With an athletic build and characteristic mop of curly hair, Nemtsov had been a face of the opposition for years, along with anti-corruption blogger Alexei Navalny and former prime minister Mikhail Kasyanov, though no one figure has succeeded in uniting the ranks of opposition-minded Muscovites.
The opposition has little support outside big cities and Putin has now been Russia's dominant leader since 2000, when ailing President Boris Yeltsin chose him as his successor, a role Nemtsov had once been destined to play.
Even many of Putin's opponents have little doubt that he will win another six years in power at the next election, due in 2018, despite a financial crisis aggravated by Western economic sanctions over the Ukraine crisis and a fall in oil prices.
Many opposition leaders have been jailed on what they say are trumped-up charges, or have fled the country.
Nemtsov, a fighter against corruption who said he feared Putin may want him dead, had hoped to start the opposition's revival with a march he had been planning for Sunday against Putin's economic policies and Russia's role in east Ukraine.
The Kremlin denies sending arms or troops to Ukraine.
In a change of plan, the opposition said Moscow city authorities had allowed a march of up to 50,000 people alongside the River Moskva to commemorate Nemtsov's death.
Ukrainian President Petro Poroshenko said Nemtsov had told him about two weeks ago that he planned to publish evidence of Russian involvement in Ukraine's separatist conflict.