The idea that left vs right has been replaced by open vs closed is one of the most self-serving conceits of contemporary politics. I have never met anyone who wants to live in a closed society, but I have met plenty of people who think that the forms of openness of the past couple of decades have not served their interests.

Factories and offices have moved abroad. EU free movement has brought a new workforce to compete with the one already here, and an extra four million people overall have arrived in the past 15 years, while wages have barely grown. Combine that with open public services and an estimated illegal migration population of at least 500,000 and you can see why so many people have not needed Nigel Farage’s help to worry about borders.

One such person is a British Caribbean ex-serviceman, Vince McBean, who appeared last week on a Channel 4 news special on the Windrush scandal to report that despite being wounded in action he had to pay £90 for his British passport. Departing from the programme’s narrative he also blurted out his resentment against the Europeans ‘who come over here and benefit from our hard work’.

It is not just the white majority who want a return to more modest levels of immigration and a sense that the authorities have some control over who is here. Working-class black and Asian Brits often share the same sense of being pushed aside by the high inflows that have become too central to Britain’s economic growth model.

It is also minorities and recent immigrants who are most likely to be disadvantaged by illegal immigration. Illegality, higher here than in comparable countries, is not only unfair on legal immigrants who wait their turn and the businesses who play by the rules, but it fosters as well a twilight world of criminality and exploitation. A 2013 poll found that 82 per cent of people want stronger action to remove illegals.

It is worth rehearsing these basic points as the many enemies of the more stringent border controls introduced in May’s time as home secretary (2010-2016) jump on the Windrush bandwagon with a view to undoing her achievements. It is also worth pointing out that the Home Office has become a very different place in recent years. One official I know recalled joining in the early 1990s ‘when it was completely white and very right-wing’.

Small-c conservative attitudes remain in parts of the department and there were some people who grumbled about Amber Rudd’s more permissive approach. But having spent some time in recent months mixing with immigration officials at several different locations, and levels of seniority, I have been impressed by their generally humane attitudes. As one of them put it to me: ‘A lot of the immigration offenders we pick up are victims too.’

The Home Office reflects the ethnic mix and views of the country. The London-based illegal immigration enforcement team I witnessed in action earlier this year was half white and half black and Asian, led by a charismatic British Nigerian woman.

They were raiding shops and restaurants in Thornton Heath, south London, based on tip-offs from the public, looking for illegal workers. They usually find plenty —although just two when I was with them.

The Windrush scandal has left many people with the impression that deporting people from Britain is easy. That is not how it looks from the front line where a combination of resource constraints (lack of detention centre space), legal reform (human rights law) and political obstruction (countries of origin refusing to accept returns) makes it feel like the system is ‘broken’, as one regional director described it.

That’s an exaggeration. The number of deportations rose from 25,000 in 2004 to 45,000 in 2013, though it has trailed off a bit in recent years and there has been a steep decline in the number of forced removals — last year it was just 12,321, and half were prisoners. (The top three countries for deportations are all white European: Romania, Albania and Poland.)

One big reason forced deportations are harder is that asylum claims can be spun out until an individual has been here for more than two years and can then try to claim family life protection under human rights law.

These obstacles explain why all governments in recent times, but especially Theresa May’s Home Office, tried to nudge illegals to the door via the now-to-be-renamed ‘hostile environment’—meaning that, mimicking a European ID card system, people have to prove their legal status when opening a bank account, renting a flat, using the NHS, taking a job and so on.

But it has also come to mean the general shift to a more controlled border. Some of this is surely welcome: we issue 2.6 million visas a year and thanks to tighter procedures, especially for students, the number of visa over-stayers has declined sharply. We have seen, too, the successful rolling out of Biometric Residence Permits (BRPs) for non-EU people here more than six months.

It is too early to tell whether the more intense hostile environment policy, legislated in 2014 and 2016, is making any difference. Critics claim that it merely drives people further underground. In any case, the Windrush scandal has ensured that the hostile environment now faces a hostile environment of its own.

The pro-mass-immigration NGOs, and some people in the Home Office itself, did warn that requiring more people to prove their status created two big problems. First, the people who were most likely to be asked to prove their status would be people who did not look or sound like the ethnic majority, though around half are likely to be white.

The bigger problem is the one we are all now familiar with: no thought was given to historical anomalies, above all the Commonwealth citizens who were given indefinite leave to remain in 1971 but could not prove it. The vast majority of such people can prove their status but there is a small remnant of maybe a few thousand who cannot, most of them belonging to the 30,000-plus group of older Commonwealth citizens who do not own a passport.

The fact that this remnant, for various reasons, is mainly composed of British Caribbeans, the totemic post-war immigrant group towards whom white Britain feels a mixture of affection and guilt, has made the outrage about their harassment even greater.

There will be further apologising and compensating to come but it is worth considering what Sajid Javid should now do to reform the hostile environment without returning to the more laissez-faire border that Theresa May inherited.

The first and most obvious thing is to create a clear distinction in the status-checking process and the Home Office procedures between recent and long-standing residents (plus their children). Anyone in the latter category should get the benefit of the doubt, and of an independent ombudsman attached to the Home Office if there is a genuine question about their right to be here.

Windrush, plus Brexit, provides an opportunity to grasp a bigger nettle. There is a large group of people, maybe 200,000, who are here illegally (though they may have arrived legally on a visa) but have survived 10 years or more. Many are people with forged identity documents who did not have to take up BRPs in 2008.

They are often well-integrated, with National Insurance and NHS numbers. They are the sort of people who become the focus of community ‘let them stay’ campaigns. And we should let them stay. A general amnesty would send the wrong signal and would encourage a surge of would-be illegal entrants but we should restore the rule that legalises anyone who has been here for 10 years — and who has not committed a serious crime.

At the same time, to signal that the government is not letting up on illegal immigration, Sajid Javid should ensure that the Home Office moves more swiftly to deport people after asylum appeals have failed, rather than waiting for a judicial review to be lodged. This is where it would be sensible to have targets.

And illegal immigration should be re-framed as a fairness issue, connected not only to anti-trafficking initiatives but to better enforcement of minimum-wage legislation, private-landlord licensing and so on. Without rogue employers, illegal immigration would scarcely exist. Most employers, especially in London, are now aware of the importance of checks and of the £20,000 fine for hiring an illegal. Yet of the £50 million in the more than 3,000 fines levied in 2016, only £16 million had been collected in early 2018.

We are in the middle of a slow shift from a low documentation society to an ID card one and from a laissez-faire to a more controlled border. The dim-witted, automatic-pilot implementation of the illegal immigration policy that led to the recent scandal was a mistake not an inevitable consequence of that shift, as so many border-sceptics gleefully claim. Sajid Javid should not abandon the policy but learn from the mistakes.