The Keeper’s House sits in the basement of Burlington House, a restaurant in disguise. It is quite different from the grand cafés of St James’s and Mayfair, which are raging exhibitionists with banquettes splayed like limbs. It is secretive and it knows, consciously or not, the tricks of children’s literature: the looking-glass, the wardrobe and the door. It is an 18th-century basement transformed, by magical whimsy, into a restaurant. To visit the loo is a quest for which you need a Gandalf, a hobbit and a lamp.

Burlington House looks like an English mansion that stared at Palladio, had a panic attack and exploded. It is clever-clogs land, home to a pile-up of learned societies, which I hope are ever at war, paintbrush against rock: the Geological Society of London; the Linnean Society of London; the Royal Astronomical Society; the Society of Antiquaries of London; the Royal Academy of Arts; the Royal Society of Chemistry. It reminds me of the Church of the Holy Sepulchre in Jerusalem — which houses so many battling Christian sects that the key was given to a Muslim, for safety — but with jumpers. Intellectuals -loiter in their disarming and shockingly unfashionable clothes.

I understand why the clever clogs cling to Burlington House, even if the government once dreamed of evicting them and renting their rooms to — who knows? — Nestlé. There are few gaudy mansions on Piccadilly now: just Cambridge House, which is shuttered, and the Albany, a rest home for male slags (or ‘swordsmen’, as my colleague Taki might call them), and this. In London, everyone is a postcode snob including Dracula who, as I always remind people, lived — ish — at no. 347, which does not exist. If it did, it would now be a fashion concession, probably Pringle.

Outside, there is babble: Piccadilly, a street that screams with cashmere specialists and bus lanes and actors fretting near Bafta. This ceases when you pass the entrance to Burlington House, or the seal, if you imagine you are in Indiana Jones and the Last Crusade, as I do, due to the presence of antiquities.

The door to the restaurant is marked by an ancient streetlight and a neon sign. Here a man checks that you have a booking, because the Keeper’s House is also a private club for friends of the Royal Academy, who must be protected from nameless dangers emanating from the street. He is very nice but I do not think he could keep Isis out if Isis wanted to steal a Titian, but then Isis would never do anything so sensible. They would take a Constable, oblivious to the fact they were contemplating a vision of 19th-century rural Britain that, though comforting, was entirely bogus.

Inside there is a red bar, a slightly overstyled garden and a pair of tiny green dining rooms, frozen to the same temperature as a slate mine to protect the art, which is by Royal Academicians, and for sale. I like a restaurant that prioritises the happiness of decor over customers, particularly in Mayfair, the global capital of fake deference to rich people in the service of taking their money. At least one customer is in a coat, and it is spring outside, beyond. (A fellow critic claims she saw a woman in a Brezhnev hat.) This cold becomes something like precision because once you are settled with the richness, the darkness and the strangeness, it is perfect, and I am amazed it is not full. The food is small, bright and superb; the service is charming. We eat chicken liver paté and summer vegetable risotto in an almost empty room; pink rump of lamb with miniature potatoes; a glorious chocolate mousse.

If this restaurant seems secretive, it is both bad marketing and good fortune. I would call it a functional oddity — and what is better than that?