Last week there was a public toilet for sale on the coast of Cornwall. The Kent-based auctioneer called it ‘an exciting and rare opportunity’, although its video tour of the property did not even undo the padlock on the security door. It was on the market for £20,000, which was a bargain — the last exciting and rare toilet block to be auctioned in Cornwall went for five times its asking price, even though it didn’t have as nice a view. It did, however, have windows.

It’s undeniable that the property market in Cornwall is overheating. The backlash to the toilet auction was such that it was withdrawn from sale, but house prices in parts of the county are increasing at double the national average. Property searches have gone up 140 per cent in two years, and last month a bungalow in Port Isaac sold within five minutes of the listing going online.

Some of this bubble is the result of movement out of cities. But the part that causes resentment is second homes. Cornwall has 1 per cent of England’s population and 17 per cent of its second homes; in some towns, second homes are the majority. Already the signs are going up in the south-west: ‘No more second homes, our village is dying.’

The withholding of planning consent for new builds unless they are guaranteed not to become holiday homes is well-intentioned, but merely intensifies the problem within the existing housing stock, and villages such as Mousehole are becoming as empty over winter as they are overrun during summer. My son, like every schoolchild in Cornwall, loves the story of The Mousehole Cat — in which Tom Bawcock goes fishing in a storm so that the village children will not starve — and he pretends not to see Daddy crying when we get to the bit about the villagers standing with lights on the quay to guide the boat back in to port. Even he notices when we walk round the harbour in December and see not a single light in any of the windows.

It’s easy to see this as an issue of Rich against Poor — but the rich are not the problem. The properly rich in Cornwall — that is, the people who made money from tin 100 years ago, or from captured prizes 200 years ago — have estates, and they have given several families I know their only chance of renting within an hour’s journey of their work. I also know of people who bought buy-to-let property and take justifiable pride in the fact that they rent it to local people rather than as holiday accommodation. But even a Tory romantic like me will have to admit that there is not enough noblesse oblige to go round — other friends rely on landlords who collect the rent in cash.

More to the point, if the issue is simply Rich vs Poor, it is quite easy for the rich to show that they bring money to Cornwall. Tourism generates millions for the local economy. But that is nothing to do with having a second home. Contractors and tradesmen would have work to do whether the houses were owned by local families or visitors, and it would be business coming in all year round, rather than just in the summer.

I am convinced that the best way to attack second-homers is not to mock them for being too rich, but to attack them as cheapskates. For all the locals’ complaints about being outbid by people from London, the second-homers often pay less. It is only this year that the government agreed to close the loophole that second homes could avoid council tax by registering as businesses and avoid business rates by qualifying for small business rates relief and thus contribute nothing to local services.

Even the price of the property can be less if it’s a second home. I was talking to someone selling her house in Cornwall who admitted that her mortgage was interest-only, and so less than half the price it would have been if she’d been paying off the capital. In effect, she had rented her property for a few years, then had the rent refunded in full. A local person who doesn’t have a house in London to move into or sell at the end of the mortgage term can’t compete with that. Since house price inflation outstripped interest rates, as it usually does, she was in effect paid for having a holiday home.

I have never seen the point of having a holiday home. Why would you spend hours travelling to do exactly the same chores as at home but nearer to the sea? The only reason for a self-catering holiday at all is if you can’t afford to go to a hotel. There is no shame in that. There is, or should be, shame if you could afford to stay in a hotel and decide instead to be cheap and buy a second home.

We need to stop pretending that having a holiday home in Cornwall is an aspirational lifestyle choice. We need to stop saying: ‘Oh you’ve got a little place near Polzeath? Lucky you!’, and start saying: ‘Can you not afford a hotel? Shame.’