Thirty-five years ago, the late Christopher Hitchens published a book about the Elgin Marbles. Unsurprisingly, it was a polemical work; he was passionately campaigning for the return of the sculptures to Athens. But that was not the reason why I wrote a scathing review of it for The Spectator. Parts of it were plagiarised, as I showed, from the classic book by William St Clair; and in some places Hitchens dealt with the awkward fact that the evidence did not fit his claims by abbreviating the quotations, filtering out the unwanted bits.

Hitchens replied with a thunderously disdainful attack on me in the letters page. I said to the then editor, Charles Moore, that I feared that if I met Hitchens at a Spectator party he would punch me on the nose. ‘Don’t worry,’ Charles replied, ‘he’s not like that. No, he’ll just wait until you publish a book, and then write a review trashing it.’ (I said he would have a job on his hands, as my next book would be an edition of 17th-century correspondence, mostly in Latin. ‘Oh, that won’t stop him,’ said Charles, merrily.)

There is a long history, much of it honourable, of British people campaigning for the return of the sculptures and lambasting Elgin, the British government and the British Museum. Sometimes special factors apply: in Hitchens’s case, it was probably relevant that he had a Greek Cypriot wife. Philhellenism has been a strong element in our culture – in the old days, arising from a classical education, and more recently from Greek holidays. ‘We love Greece’, British people say, as if referring to a personal friend, when what they mean is that they have enjoyed their visits there and met many friendly Greeks.

A new lobbying group, The Parthenon Project, is just the latest in a sequence of return-the-Marbles campaigns and organisations. Its founder is a rich Anglophile Greek businessman, Ioannis Lefas, who says, of Greece and Britain: ‘I want to bring them together with a win-win proposition.’ Understanding that only a change in the law can make possible the removal of the Marbles from the British Museum, his aim is to work on the legislators. This involves flying MPs and peers to Athens ‘to be wined, dined, and persuaded to the cause’, as a recent news report put it, ‘with a crash course on Lord Elgin’s alleged theft of the Marbles’, and on the merits of displaying the sculptures there. The project sponsored a debate hosted by The Spectator at the Tory party conference.

Mr Lefas’s sincerity cannot be doubted. The approach is almost touchingly Trumpian: let me through, I’m a businessman, I know how to do deals. And much of his emphasis is – appropriately, for his target group of politicians – on solving a political problem. As he puts it: ‘It would improve the sentiments between Britain and Greece. You cannot let this small thing mar this relationship.’

Real politicians will know, of course, that the effects of grand gestures and ‘resets’ are very short-lived, lasting only until the next cause of disagreement. But just to talk in terms of a ‘problem’, to be solved by a deal, is already to shift the argument in one direction only: for the problem exists only insofar as Greeks are opposed to the present situation, and a deal could be made only by changing it. The Greek government tried this tactic in 2015, when it asked Unesco to ‘mediate’. The then culture minister, Ed Vaizey, curtly informed Unesco that ‘we have seen nothing to suggest that Greece’s purpose in seeking mediation on this issue is anything other than to achieve the permanent transfer of the Parthenon sculptures now in the British Museum to Greece’.

But that was then. In a podcast in December last year, Lord Vaizey said he would probably support ‘the return of the Marbles’, and this month he told the Telegraph: ‘I was in Athens recently with The Parthenon Project to understand more about their vision for a “win win” solution, and I fully support their approach.’ Another former minister, George Osborne, who happens to be chair of the British Museum, said in June that he thought a deal could be done. The Parthenon Project wants to move the sculptures permanently to the Acropolis Museum in Athens, in return for rotating exhibitions in London of other things from Greek museums; Osborne’s idea, apparently, is for only partial rotations of the Elgin Marbles to Athens, with the British Museum remaining their permanent home – which would not solve the problem in Greek minds. But still, a certain direction of travel is visible.

When something is presented as a political problem, opinion polls are often invoked to gee-up undecided politicians. A YouGov poll last year found that 59 per cent of adults favoured returning the sculptures, with only 18 per cent preferring their retention here. The question was ‘Where do you believe the Parthenon Marbles belong?’ and it followed just two explanatory sentences, of which the second said that the Greek government had requested their return ‘but the British Museum has refused’. A preliminary question, ‘Have you ever given any serious thought to this issue?’, might have been helpful.

Our legislators, at least, should consider complex issues seriously, with access to all the relevant facts. Whether they will get those from their briefings in Athens on Elgin’s ‘alleged theft’ of the sculptures, I do not know. But they really should not listen to one of The Parthenon Project’s noisiest supporters, Stephen Fry, who has compared Elgin’s acquisition of the Marbles to an American buying the Eiffel Tower from the Nazi occupiers in Paris in 1941. This level of historical crassness is wince-making. There had been no functioning Greek state just a few years before Elgin set to work in 1801, and no sudden recent conquest; for centuries, the Greek territories had been integrated into a multinational empire. Sometimes, however, the accusation shifts the other way, with the claim that Elgin was a ‘thief’ because he failed to satisfy the Ottomans’ own standards of legality. Historians have pored over this issue; the documents are incomplete, but William St Clair’s conclusion is that his actions were in the end condoned by the Ottoman authorities.

Sensible defenders of Elgin’s actions rely not on those aspects of the story, but on the fact the sculptures had previously been vandalised and destroyed at a rapid rate, and there was every reason to think such literal dilapidation would continue. As St Clair notes, a traveller drew 12 figures in the Parthenon’s west pediment in 1749; by 1800 there were just four. Five slabs of the frieze drawn by James Stuart in the 1750s had disappeared; two pediment figures, intact in 1765, were now headless. Such facts seem to have influenced Elgin’s decision to remove, rather than just record, the sculptures. He was more saving than stealing.

Does this decide the matter? No, because there are so many other issues to consider. One is the artistic principle that to reassemble a work of art is an intrinsic good. That is true, but although the marbles could rejoin other Parthenon sculptures in the Acropolis Museum, they would not return to the original artwork, the Parthenon – they would move from one museum to another. The argument that there is something specially valuable about so-called ‘universal museums’, of which the British Museum is one, also needs careful consideration. And Parthenon Project supporters who say that high-quality replicas will do just as well are wielding a double-edged sword: if so, why not have replicas in Athens? (Many are already there.)

Hovering behind all this are some basic philosophical questions about whether, and how, things done in the distant past can generate rights and duties today. Recent campaigns about the legacies of colonialism have generated more heat than light. But they have undoubtedly raised the pressure on museums. Although Elgin was obviously not a colonial ruler of Greece, the prim insistence that this case is sui generis, with no possible implication for other cases, seems untenable. There are fundamental issues to be addressed here, too important to be nudged aside by wining, dining, and the temporary improvement of Anglo-Greek relations.