In a letter he wrote at the height of the scandal to his military mentor, Brig Gen Shelton, a Vietnam Veteran, Petraeus also says he "screwed up royally", The Daily Mail reported.
The newspaper obtained a copy of the letter, which reads in part: "I screwed up royally. I paid the price (appropriately) and I sought to do the right thing, at the end of the day.
"Team Petraeus will survive though have obviously created enormous difficulty for us. Holly is however once again demonstrating how incredibly fortunate I was to marry her."
Petraeus, 60, has not been sighted since he resigned as CIA director on November 9, and according to friends he spends his days inside reading or working out.
The New York Times reported recently he is missing the pace of his old job but is considering writing a book or teaching, as well as regaining the trust of his wife of 37 years, Holly.
Earlier this month he said he regretted his affair with the married Broadwell, who is 20 years his junior, but insisted he did not pass her classified intellilgence, the London Telegraph reported.