The inaugural move marks a major gain for advocates of same-sex couples after the Supreme Court issued two rulings expanding gay marriage rights last year.
Under the new policy, same-sex couples will enjoy the privileges even in states that do not recognize their marriages, so long as they legally married in another state.
Among the key benefits the Justice Department will now ensure are extended to same-sex couples are the compensation fund for the September 11, 2001 attacks, as well as death and educational benefits for the surviving spouses of police officers and firefighters injured or killed in the line of duty.
"In every courthouse, in every proceeding, and in every place where a member of the Department of Justice stands on behalf of the United States, they will strive to ensure that same-sex marriages receive the same privileges, protections and rights as opposite-sex marriages under federal law," Holder said in advance excerpts of his speech.
Same-sex couples would also be held to the same legal standard as their heterosexual counterparts on matters such as how debts are handled in federal bankruptcy proceedings and visitation policies at federal prisons, as well as compassionate release or sentence reductions in certain cases of crisis involving their spouse.
Advocates of gay rights immediately hailed the decision.
"This landmark announcement will change the lives of countless committed gay and lesbian couples for the better. While the immediate effect of these policy decisions is that all married gay couples will be treated equally under the law, the long-term effects are more profound," Human Rights Campaign president Chad Griffin said in a statement.
"Today, our nation moves closer toward its ideals of equality and fairness for all."
Holder, who is due to deliver his speech at the Human Rights Campaign's gala at 7:00 pm (0001 GMT Sunday), plans to detail the new policy in a memorandum to employees on Monday.
The text will "formally instruct all Department employees to give lawful same-sex marriages full and equal recognition, to the greatest extent possible under the law," according to the excerpts.
Holder, the first African American attorney general, also plans to stress that his predecessors played a key role in the civil rights movement half a century ago.
"As all-important as the fight against racial discrimination was then, and remains today, know this: my commitment to confronting discrimination based on sexual orientation or gender identity runs just as deep," Holder will say, according to the excerpts.