What a weird lot crime writers are. I don’t come to this conclusion lightly, since I’m a crime writer myself, but on the evidence of this magisterial but wickedly entertaining book the conclusion is inescapable.

As you turn the pages, the evidence mounts up. One crime writer has been considered a serious candidate for sainthood and another has been convicted of murder. Wilkie Collins simultaneously maintained two mistresses and their children but never bothered to marry either. Mary Roberts Rinehart, an early 20th-century queen of American suspense fiction, narrowly escaped being murdered by her chef because she wouldn’t promote him to butler. Agatha Christie famously engineered her own disappearance, and Dorothy L. Sayers spent most of her adult life pretending that she didn’t have an illegitimate son. At one point, Patricia Highsmith had 300 snails, some of which she smuggled through customs in her bra. And so on.

Perhaps wisely, Martin Edwards tends to be more discreet about the private lives of living authors, and this book is primarily a history of crime fiction from the 18th century to the present. He is himself a crime writer (and a winner of the genre’s highest award, the Diamond Dagger), and he also has an impressive track record as a critic and editor. He sets out to tell the story of how crime fiction has developed over the years.

The ghost of Julian Symons hovers over the pages. Symons, a distinguished crime novelist whose sales never matched his glowing reviews, published Bloody Murder, his influential history of the genre, in 1972, followed by two revised editions over the next 20 years. It is much more acerbic in tone than The Life of Crime, and less biographical in content. Where Symons preserved a critical distance from his subject, Edwards frankly revels in it.

More than 20 years ago, as Edwards acknowledges here, I encouraged him to write his own version of Bloody Murder. I thought then, and think now, that he is the ideal person for the job. The result is a long book, but it’s reliably readable and frequently amusing. It also inspires awe: Edwards combines wide reading with a good memory, meticulous control over his unruly material, critical acumen and sheer bloody persistence.

Some experts have rather fancifully found the genre’s roots in certain Old Testament stories, but Edwards takes for his main starting point Caleb Williams, William Godwin’s 1794 novel about a young man whose quest for truth puts him at the mercy of a ruthless and powerful man. Godwin was a radical, as befitted the husband of the feminist author Mary Wollstonecraft, and their daughter eloped with Shelley and subsequently wrote Frankenstein. Like many crime writers who came after him, he had doubts about the artistic value of what he called his ‘mighty trifle’:

What had I done? Written a book to amuse boys and girls in their vacant hours, a story to be hastily gobbled up by them, swallowed in a pusillanimous and unanimated mood.

Edwards gives the narrative a broadly chronological structure. Within that, each chapter explores a particular trend or development – from Victorian sensation novels to postmodernist crime fiction, to California-based crime, to women as private investigators. Some authors merit all, or most, of a chapter to themselves – for example Arthur Conan Doyle, Georges Simenon, Highsmith and, of course, Christie.

As the introduction makes clear, Edwards is determined to avoid what he calls ‘recency bias’. As a result, many currently popular authors are absent, or dealt with only cursorily, perhaps on the grounds that their work hasn’t contributed anything significantly different from that of other writers. It’s a safe bet that this will upset a few readers, though not me, since Edwards is kind enough to mention some of my books.

What’s it all for? Isn’t crime fiction a debased cultural form, appealing to our worst instincts? Well, yes and no. It’s unashamedly populist. Its main job is to entertain – and judging by sales figures, it succeeds. On the other hand, discriminating readers have always recognised that the best crime stories are rather more than entertainment. Edwards quotes Wittgenstein in support of this argument: ‘More wisdom is contained in the best crime fiction than in philosophy.’ Cyril Connolly was another unexpected ally, predicting that ‘in 100 years our thrillers will have become textbooks… the most authentic chronicles of how we lived’.

There are fashions in all things. In the so-called ‘Golden Age’ of crime fiction between the wars, puzzle novels – rather like articulated crosswords with corpses – had a cachet. In recent years, noir crime – mean streets, tough guys and faux realism – has occupied the cultural high ground, inspiring imitations by literary authors such as Martin Amis and John Banville.

The final chapter of the book deals with the work of P.D. James. Most of her books are traditional whodunits in form – and her police protagonist is a published poet, for heaven’s sake! But in tone her novels are very different from those of her predecessors. They engage with a contemporary world which contains real suffering and the harsh dilemmas of life and death. They are underpinned by questions of good and evil.

Edwards ends the chapter with two quotations which, it’s reasonable to infer, reflect his own feelings about crime fiction. James Runcie suggested that ‘we write, and we read, not just to be entertained but in order to work out who we are and how we might live a better and more meaningful existence on this frail Earth’. The late Reginald Hill thought that the quest for detective, reader and author alike was to reveal the ‘truth about human character and experience. That’s the ultimate goal, isn’t it?’

Well, isn’t it?