On the face of it, the moral case against tax avoidance seems pretty straightforward. If you’re a UK taxpayer and benefit from public goods and services, then you should pay your fair share of tax. If you’re paying less than that, then you’re a free rider. You’re breaking the social contract.

But what do we mean by ‘fair share’? The standard defence of tax avoidance is that it’s perfectly legal — if it wasn’t, it would be tax evasion — and the social contract only obliges people to obey the law, not to pay more tax than they have to. To maintain that people are morally obliged to pay an additional amount of tax, over and above what they’re legally required to pay, is a tricky position to defend.

For one thing, it means we’re all guilty of tax avoidance. I’m not just thinking of people who buy whisky in duty free or take out an Isa. If the ‘fair’ rate of tax is higher than the actual rate, and anyone not paying the ‘fair’ rate is ‘dodgy’, then everyone who fails to make a voluntary donation to HMRC on top of their annual tax bill is at fault.

More fundamentally, where does this obligation come from? A common argument made by those who think rich people should pay more tax than the legal minimum is that they benefit disproportionally from things like the rule of law, particularly when it comes to the protection of their property. But if you calculate that benefit, it’s likely that Britain’s multi-millionaires are paying more than their fair share. After all, the top 1 per cent of income earners pay roughly 25 per cent of the total income tax take but don’t consume a quarter of public services. You could argue that the super-rich benefit in so many intangible ways — the roads, schools and hospitals used by their employees, for instance — that the total value is incalculable. But if that’s true, it doesn’t follow that they’re paying too little. They could be paying too much. We just don’t know. That’s what ‘incalculable’ means.

More sophisticated critics of tax avoidance stop short of saying people should pay more than the law requires, but distinguish between the spirit and letter of the law. Many forms of avoidance, such as filling up with petrol the day before a rise in fuel duty, are fine because they are in keeping with the spirit of the law. But avoidance that involves paying expensive accountants to exploit loopholes is wrong. Even though you’re complying with the letter of the law, you’re flouting the intentions of those who passed the law. The sin here consists in not complying with the will of our democratically elected representatives when it comes to tax, rather than not paying your ‘fair’ share, although the two arguments are often run together.

But there are problems with this position too. How are people supposed to know what the intentions of legislators are when it comes to taxes other than by examining the laws they pass? This was a point made by Lord Hoffman, a senior British judge, in a lecture on tax avoidance in 2005. ‘The only way in which Parliament can express an intention to impose a tax is by a statute that means that such a tax is to be imposed,’ he said. In other words, when it comes to tax there is no distinction between the spirit and letter of the law. They’re one and the same.

No doubt opponents of avoidance would dispute this and claim that it’s usually perfectly obvious when a tax loophole is contrary to the spirit of the law. However, even supposing that to be true, it’s a bit of a leap to argue that taxpayers have a moral obligation to look beyond the tax code and do their best to honour the intentions of the lawmakers who created it. Ultimately, it comes down to where you think the onus of responsibility lies. Is it the duty of the taxpayer to hand over more than the letter of the law requires if the law in question fails to capture the intentions of the lawmakers? Or is it the duty of the legislature to draft the law properly in the first place?

I’m framing that question in a way that favours the libertarian point of view, but the truth is I haven’t completely made up my mind. Like most people, I instinctively want to condemn aggressive tax avoidance as immoral. But I can’t come up with a good argument for doing so.