‘Where are all my favourite parts?’ Arnold Schoenberg asked, on being presented with a severe academic analysis of the Eroica symphony. ‘Oh, there they are. In the tiny notes.’ The tendency of many people, presented with the overwhelming abundance of an art form, is to exclude as much as possible. Reduce the wonderful life of incidental invention to the tiny notes; erect walls excluding the fascinating curiosity, the eccentric, the madly idiosyncratic. Produce a list of the 100 Best Books, sticking to declared Greatness.

People have been producing lists of the Best Books for a hell of a long time. When copyright law was reformed in 1774, it enabled publishers to produce collections of novels for the first time. James Harrison’s multi-volume The Novelist’s Magazine was the earliest, presenting Richardson, Fielding, Smollett, Charlotte Lennox, Eliza Haywood and Sterne as the classics of this new genre. (He also included Edward Kimber, the Revd Dr Dodd and John Shebbeare, and only Robinson Crusoe of Defoe’s novels.)

The publishing of collections continued, with what often seems to us eccentric choices. Ballantyne’s Novelist’s Library in the 1820s removed most women novelists and Defoe altogether. Sometimes selecting an author for Greatness required editorial intervention. Anna Barbauld’s The British Novelists, from 1810, was the work of a highly intelligent writer and commentator. She included Maria Edgeworth’s wonderful Belinda, but the romance between a black servant and a white English girl was removed. That was not compatible with Greatness.

These collections did a lot of good, as energetic and opportunistic publishers often have done. When Richard Bentley bought the copyright in Jane Austen’s novels for his collection of classic novels in 1832, she was a middle-ranking author, out of print. Bentley secured her path to greatness. But when the curatorial tendency to reduce takes precedence, and a list of books looks less like an explorer joyously unearthing, and more like a stern reduction to a list declaring Greatness, the true life of the novel quickly evaporates.

I have a catalogue of a 1951 exhibition, mounted on a suggestion from the recently formed Unesco, of 100 Modern Books and Writers. Put on for the Festival of Britain, it marked a century since the Great Exhibition, which took no account of novels: in the age of Dickens, Thackeray, the Brontës and dozens of others, they were just there. In 1951, the authorities felt able to inform the public where the limits of Greatness lay. Only nine women are included, and six of those had close family connections to the aristocracy. Nobody seems to have thought that odd.

In the 19th and early 20th centuries, the principle of critics like George Saintsbury was: ‘Read everything’

These lists of the 100 Greatest Novels have continued ever since. The Guardian has just published one, compiled by asking a highly curated selection of journalists (including The Spectator’s books editor Sam Leith), reviewers, administrators and human rights lawyers fighting for the rights to sell the Chagos Islands to nominate their ten highest-rated novels. The paper asked some novelists to contribute, too. Like all such lists, it reveals an unconscious bias – here, towards books taking an approved stand on major social issues, and away from, say, Aunts Aren’t Gentlemen. Some of the greatest novels in English don’t take a position Guardian-approved critics would agree with; you aren’t going to find Smollett’s Humphry Clinker here, or Surtees’ sublime Mr Facey Romford’s Hounds, or, ludicrously, anything by Trollope or Anthony Powell. You would have done better in 1951 if you were the daughter of a viscount; in 2026, other forms of identity will smooth an author’s path.

This, of course, is just a parlour game, and quite unobjectionable. But to a very large extent, the activity is replacing a serious consideration of what the history of the novel actually consists of. In the 19th and early 20th centuries, the principle of literary journalists like George Saintsbury was: ‘Read everything.’ Some studies of the novel from that time, such as Saintsbury’s histories or the slightly later book by J.M.S. Tompkins about the popular novel in the 18th century, have never been surpassed for their voracity.

That voracity survives in the occasional scholar, like the great Regency specialist Peter Garside. In most areas, though, the mindset of 100 Great Books has spread into academic work, and we get a survey of pinnacles, isolated from one another. Oxford’s supposedly authoritative Handbook of the Victorian Novel has a chapter entitled ‘Novels of the 1860s’, one of the liveliest periods for fiction. It considers just eight titles.

You won’t understand anything about the novel by reading a list of 100 great novels divorced from the circumstances of their writing. Middlemarch, which topped the Guardian list, is great, of course, but nobody seems to wonder why it’s so long, and why nobody in it can commit adultery. (The answer is that the circulating library mogul Mudie committed to buying novels in three, or in Middlemarch’s case four volumes, but on condition that there was no open immorality.)

Being interested only in Greatness has, it turns out, isolated Middlemarch in an extreme way. No one will get a proper sense of what Eliot could do without reading the childhood scenes of The Mill on the Floss, the Grandcourt half of Daniel Deronda or the magnificent Transome strand in Felix Holt, the Radical. Yet this list – and this approach to literary history – suggests readers could stop at Middlemarch without reading the rest of Eliot, or other women novelists: Harriet Martineau, Frances Trollope, Elizabeth Gaskell, Margaret Oliphant. But Oliphant’s Miss Marjoribanks is one of dozens of Middlemarch’s predecessors about the limits of women’s lives in provincial society. Setting Middlemarch aside for our sole attention is the same mindset that leads to millions swamping Florence and ignoring the sublimities of Parma.

The life of the novel is, in reality, contained not just in Clarissa, Middlemarch, Nostromo, and A House for Mr Biswas. It is there in interesting fiction with awful titles (Edgar Mittelholzer’s A Tinkling in the Twilight, for instance). It includes books that openly imitate a bold innovation, and bring something wonderfully new to the table. David Copperfield encouraged Thackeray to examine a writing life in Pendennis, which, as Andrew Lang said, tempted a generation ‘to run away from school to literature’. Some of its imitations are glorious – but Robert Brough’s Marston Lynch, or Charlotte Riddell’s account of a woman novelist’s failure, A Struggle for Fame, are never going to make a Greatest Books list.

The novel, and the commercial market for new novels, is how, for centuries, extraordinary new voices have made themselves felt – Peacock, Meredith, Firbank. Unheard subjects have been tried out with the aim of presenting a novelty to the public. Julia Frankau introduced Victorians to middle-class Jewish society in Dr Phillips; once Mudie’s moral demands went after 1894, sauciness entered, not just in sexy bestsellers like Mary Cholmondeley’s Red Pottage, but also in Henry James’s What Maisie Knew.

We understand this crucial breadth of vision not by a newspaper burbling, ‘The Magic Mountain! Pride and Prejudice! Dracula!’ We understand it by looking at the history of the form. Someone ought to read a couple of thousand novels, thinking about why they were written and published the way they were. And then that someone should write a history, telling the whole story. That’s what I thought five years ago. So I did.