Bertelsmann, the German parent company of Random House, will hold a controlling 53 percent stake in the new company, to be called Penguin Random House, CNN Money reported.
Pearson, the British owner of Penguin, will hold 47 percent.
"Pearson and Bertelsmann today announce an agreement to create the world's leading consumer publishing organisation by combining Penguin and Random House," the companies said a statement, cited by the Agence France-Presse.
"The combination brings together two of the world's leading English language publishers, with highly complementary skills and strengths."
"Random House is the leading English language publisher in the U.S. and the UK, while Penguin is the world's most famous publishing brand and has a strong presence in fast-growing developing markets."
The Associated Press reported that the merger brings together some of the best-known classics of English literature as well as top-selling authors.
Random House is the publisher of this year’s literary hit Fifty Shades of Grey, while Penguin has published books by authors including George Orwell, Jack Kerouac and John Le Carre.
The merger, which needs the approval of competition regulators before it can go ahead, highlights the challenge facing publishers from electronic books and online booksellers such as Amazon, the Wall Street Journal reported.
Pearson Chief Executive Marjorie Scardino said the combined group would be able to share costs and “be more adventurous in trying new models in this exciting, fast-moving world of digital books and digital readers.”
The announcement comes after weekend reports that News Corp was considering making a bid for Penguin.