If ever you find yourself in Berlin, there are three places you absolutely must visit. The first two are museums: the Neues Museum, to see the well-worth-the-detour head of Nefertiti; and the Pergamon Museum, so you can offer up a prayer of gratitude for the arrogance of all those 19th-century imperialist looters who understood that the treasures of classical antiquity are far too precious to be wasted on the barbarous cultures which, by geographical accident, have inherited them since.

Yes, perhaps I’m overstating it. ‘Barbarous’ certainly isn’t a term you’d apply, say, to Khaled Assad, the heroic and scholarly Syrian archaeologist who preferred to die rather than betray to his Isis killers the secrets of Palmyra; nor to the refugee from Aleppo described in the Guardian last week being moved almost to tears by the 17th–century wood-panelled interior at the Pergamon which reminded him so much of his childhood.

But what you can’t deny is that if half those ill-gotten treasures now on display at places like the Louvre, the Pergamon and the British Museum were still in situ, they wouldn’t exist at all.

Take the Pergamon Altar, transplanted in the late 19th century from Turkey to Berlin. You gaze up at it now and wonder: did the locals not mind when the Germans came along and helped themselves to such vast chunks of their architectural heritage?

And the answer is that they didn’t give a toss. When Carl Humann, a German engineer, first spotted it in the 1860s, the altar of the temple — built in the 2nd century BC for the Greek King Eumenes II — was being used by the local Turks as a quarry, with the stones pillaged for new buildings and the marble burned for lime.

The same was true, of course, for the Parthenon marbles removed from the Acropolis under the direction of Lord Elgin. As well as ‘continually defacing the heads’ on the sculptures, the Turks — who’d been using the place as a citadel — had sometimes pounded them down for use as mortar.

It’s all very well arguing that the Ottoman bey should never have granted the firman permitting their removal. But the facts are that he did, that the arrangement was perfectly legal and that the outcome for the marbles was about as good as could be hoped for because there they are, centuries on, safe in the British Museum rather than a pile of rubble and dust in Athens.

Why don’t they get this, those impeccably liberal types who argue for the restitution of ‘stolen’ artefacts to their countries of origin? Well, probably they do but like all lefties they’re wired always to prioritise emotion over empiricism, to do the wrong thing because it feels right. In the case of the marbles, it is, as Stephen Fry once put it in a debate, about ‘atonement’ — a way, perhaps, of apologising not just to the Greeks but to the whole world for the political incorrectness of the imperial era.

There are lots of sensible arguments against this wooliness, all of them outlined in Tiffany Jenkins’s brilliant and fascinating book Keeping Their Marbles. Is there really that strong a moral case — as Jesus College, Cambridge, bowing to the pressure of its wanky undergraduates, has just agreed to do with its cockerel — for sending back the Benin Bronzes to Nigeria? In terms of the fact that they were seized as war booty during the 1897 British military expedition, maybe. But probably not in the sense that Nigeria has a unique cultural claim on them. Modern Nigerians have no more in common with the bloodstained, human-sacrificial kingdom of Benin than modern Greeks do — sorry, Taki — with the Athens of Pericles.

This doesn’t stop the grievance-mongers trying it on, of course. Worst offender, at the moment, is Erdoğan’s Turkey — forever cry-bullying about its plundered heritage. In the case of non-Islamic objects, Jenkins suggests, this has more to do with power politics than cultural need: every time some gullible institution caves in and restores your lost treasure, you look like a strong man who has the puny West in your thrall.

With museums, as in so many other areas of our spineless, guilt-ridden, perma–apologetic western culture, we’re embarrassingly eager to surrender the pass. The Museum of World Culture in Gothenburg, Sweden, had an unrivalled collection of Peruvian textiles, over 2,000 years old, called the Paracas Collection. At least it did until its curators decided to feature it in a 2009 exhibition subtitled ‘A Stolen World’, with a hand-wringing catalogue about the ‘problematical’ nature of ‘tomb-raiding’. The Peruvian government, which until then had made no claim on the collection, demanded its repatriation. Naturally, the Swedes obliged.

Apparently this nonsense started in the late 1980s and has been getting worse ever since. According to a plausible theory advanced by sociologist John Torpey, since the fall of the Berlin Wall, utopians have given up scheming about how best to change the future and have instead sought to create a better world by trying to remake our relationship with the past.

One day, perhaps, we’ll grow out of this fad but by then the damage may have been done. In a 2012 poll conducted by Museums Journal, 73 per cent of respondents said the ‘Parthenon Marbles’ should be returned. Repatriation is very much in vogue.

But it shouldn’t be, for lots of reasons. The most obvious one comprises one word: Palmyra. And if you need a few more: Bamiyan, Baghdad and — I fear — Leptis Magna.

The third thing you should see in Berlin by the way — purely for the horror of it — is the vast collection of modernist buildings in which Germany’s and by extension Europe’s new governing class house themselves. All the ambition and arrogance of their 19th-century forebears; none of their insight or wisdom.