The author’s Japanese ghost stories brought him fame and fortune – but his own life was even stranger than fiction