‘Affairs are cruel, and if they are sin, they carry the punishment with them’, he wrote to one of the many women he cheated on throughout a long life