‘Lounge suit’ is normally a reliable signifier of supine gentility. But there it was on the invitation to Richard Rogers’s 80th birthday retrospective. Can this be the same architect once praised by a president of RIBA for his admirable ‘sod you’ approach to the public?

The same man the Parisians sniffily called an ‘English hippie’ when working on the Centre Pompidou? Surely not the architect who had to buy a cheap suit and borrow a tie to visit his new client, Lloyd’s of London?

And there he was at the head of the receiving line. Balsamic brown face, steel-grey buzzcut like a Florentine cab-driver, baggy grey cargo pants, signature violent-green collarless shirt and DayGlo pink sneakers with some sophisticated ventilation thing going on. ‘What happened to the lounge suit?’ I asked. His Lordship grinned and shrugged.

The dress code anomaly is a small matter, but illustrates larger anomalies in Richard Rogers’s illustrious career. This is not the first time he has been at the Royal Academy. In 1986, with Pompidou and Lloyd’s already and justifiably global landmarks, he appeared in a celebrity triptych with the late James Stirling and his one-time partner Norman Foster. This event established a distorting personality-based narrative from which modern architectural criticism is only now escaping.

Richard Rogers always had stand-out credentials. Like most of his London generation, early influences were the Pop-inflected architectural fantasies and fixations of the Sixties: walking cities, clip-on buildings, fun palaces, an intense infatuation with machinery. But there was more. Rogers was born in Florence, a descendant of an English family that moved to Venice in the late-19th century. His was a culture where great food and elegant, intellectual conversation were both nutrients and pleasures.

Inside Out says there can be no dispute about Richard Rogers’s achievement. He has designed astonishing, engrossing, enlivening, original buildings by which the late-20th century will always be remembered. Additionally, he is a man of great personal warmth and style and an exemplary citizen, committed to good works and civic responsibility. What began as his staff canteen is now the River Café, an Italian restaurant that redefined expectations of Italian food. Expensively.

Impossible not to like and admire Richard Rogers, but when you look closely at the work, the curiosities are as startling and as profound as the art. First, a vehement rejection of tradition and of Englishness. An early house in Wimbledon (now on sale for £3.2 million) is a glazed shed whose sole concession to decoration is a yellow-painted I-beam. In SW19, the scriptural home of domesticity, this was belligerent. Second, mechanical detail elevated to the status of sculpture. Good stuff, but a paradox because Rogers the man is not at all machine-minded. He does not even drive.

Third, an insistence on evisceration. In Paris, this led to the Pompidou’s epic whackiness with trunking and soil pipes where columns might once have been, leaving vast, uncluttered, impressive (but essentially useless) interior spaces where the art gallery has to install partitions. The external guts are also a maintenance cauchemar. Va te faire foutre, indeed.

Meanwhile, in his own handsome Chelsea house(s), Rogers has eviscerated big time, creating glorious knocked-through double-height spaces, but ones that are surely the revelation of a psychological state more than a functional advantage. Fourth, the piazza. In his insistence on public delight in traffic-free open air, Rogers betrays his Italian inheritance with its Renaissance-inspired quest for the ‘ideal city’, but has recently departed from the democratic principle at the sinister and overbuilt One Hyde Park. Defending this horror to Bryan Appleyard, Rogers’s biographer who now disavows his cause, I said, ‘But it is very well detailed.’ Appleyard replied, ‘Yes, but that makes it worse.’

Perhaps the reclusive non-doms of One Hyde Park got what they wanted in this lifeless luxury prison camp, but were other clients so fortunate with their monuments? While enriching the rest of us, the Pompidou has faults as an art gallery and, by most accounts, Lloyd’s experienced collective toxic shock when they saw what had been erected in their name.

So you wonder about the perverse nature of genius. And as you wander Inside Out you become convinced that architecture never ever was about function. It is always about wilful expression. I’d like to have seen more on this in the exhibition. How does he do it? Rogers is dyslexic and does not draw well. They say most of the Pompidou drawings were by Renzo Piano’s office: like Hawksmoor, others realise his vision. But that vision was something wonderful. Note the past tense.

If you reject history and tradition, they return and bite you on the bum. The apostate Rogers has become an English tradition: that same Lloyd’s building, once so shocking, was recently Grade II listed. The wardrobe remains defiant, but sod you architecture has put on a lounge suit.