When the political cabinet met on Tuesday, by-elections were on the agenda. The Prime Minister is facing four of them. David Warburton, suspended from the party last year over a sex and cocaine ‘sting’, is the latest to step down. On 20 July the Tories will try to defend his constituency of Somerton and Frome, Boris Johnson’s old seat of Uxbridge and South Ruislip and Nigel Adams’s Selby and Ainsty. Despite announcing she was also quitting on social media, Nadine Dorries is taking her time to trigger a vote in Mid Bedfordshire – and the whips’ office is assuming that she may hang on all the way to the next election.

Tory election strategist Isaac Levido told the ministers assembled that the by-elections required local strategies – away from the drama of Westminster – as the seats are so varied. ‘We’ll lose them all,’ predicts a gloomy government aide. But the irony is, the by-election that should most worry ministers is the one they don’t hold a stake in.

After the former SNP MP Margaret Ferrier was suspended from the Commons for breaking lockdown rules, a recall petition has been opened in her constituency of Rutherglen and Hamilton West. If by the end of next month 8,000 voters have signed it, a by-election will take place where Scottish Labour will try to take the seat back and wipe out the SNP majority of 5,230.

It will offer an important indicator of how strong the Scottish Labour recovery is. A new Panelbase poll predicts Labour is on course to defeat the SNP, win 26 MPs and become Scotland’s largest party. The glow of success would extend well past the party’s cheery Glaswegian leader Anas Sarwar to Keir Starmer in Westminster. One of the reasons Labour has struggled in recent years is because of the collapse in Scottish support. The party was wiped out in 2015, where the SNP won 56 of the 59 seats. For the first time since 1959, Labour failed to hold a majority of Scottish seats. Since then the party has found itself down and out. At one point the staff Christmas party was hosted in a corridor.

Part of the reason Starmer’s team were so triumphant after the local elections (despite not getting the national poll swing required to guarantee a majority) is a belief that gains in Scotland mean they may still win outright. Winning 20 seats north of the border is viewed as the difference between a majority and a hung parliament.

Labour’s improved chances are partly down to the scandals facing the SNP. Nicola Sturgeon’s arrest this month, as part of the long-running Operation Branchform into possible fundraising fraud, has only added to her party’s woes. ‘The sheen is coming off,’ says a Labour figure.

Starmer and his close-knit team have developed a reputation for having little love for the regional mayors. Manchester’s Andy Burnham – a former leadership hopeful – recently made a plea on a broadcast round for the Labour briefing machine to ‘leave me alone!’. He is one of several mayors to be critical of Starmer’s tightly gripped selection process after Jamie Driscoll, the North of Tyne mayor and ‘last standing Corbynite’, was excluded from a candidate list.

Sarwar has no such problems. While Starmer can appear aloof to colleagues, he has embraced the Scottish Labour leader. Sarwar is viewed within the shadow cabinet as one of the most influential figures – second after Rachel Reeves – when it comes to having the Labour leader’s ear. Starmer has visited Scotland seven times this year and the pair meet regularly. What they both share is a serious desire to win – something not every Labour politician prioritises. There is a similarity in their pitches: a focus on improving public services while talking up the difficult financial situation they are likely to inherit from the incumbent.

On the Tory side, there is mixed opinion as to who now poses the greater threat, Scottish Labour or the SNP. The initial decline of Sturgeon and her party was met with delight across government. There are senior unionists in government who would always prefer a party against independence – as Labour has insisted it is – to one that was founded to campaign for Scottish independence.

But Labour soaring in Scotland is a bigger political problem for the Tories’ hopes of clinging on to power. ‘That narrow path to victory is looking narrower and narrower,’ says a government aide. ‘We need to stop calling Humza Useless and rename him Humza Useful,’ says another senior aide.

At the political cabinet, Scottish Conservatives leader Douglas Ross offered an update on the situation in Scotland. He told ministers that while one could be forgiven for writing the SNP off at this point, their core support remains. He also highlighted where the Tories had an opportunity to get the upper hand over Labour: North Sea oil.

Since reports first emerged that Starmer wants to ban new licences being granted to allow exploration for oil and gas fields in the North Sea, he has faced attacks. It’s an area where Sarwar is seeking to reassure voters. He has been talking up the idea that the oil and gas industry will be here for some time to come. Labour could still allow exploration if it was deemed a threat to security to rely on imports – that is, if the Ukraine war carries on. However, some in Scottish Labour see an upside to the industry backlash against Starmer’s plan, because it helps to solidify support among young eco-voters. The Tories want to position themselves as the only party that supports the oil and gas industry. Given that the SNP are in coalition with the Greens, their own credentials are in doubt. It’s why there is optimism in cabinet that the party could actually win a seat or two in the north-east.

Other senior Tories think that Labour might be punished over their differing stances in Scotland, Wales and England on the Gender Recognition Act. One senior minister adds that the Welsh government could be helpful in other ways: ‘We need to start shouting about all the failures of the Welsh government, so voters understand what a Labour government would mean.’

In Scotland, the Tories and Labour still have some common ground. This week, Sarwar joined forces with the Scottish Tories to push a no-confidence motion in Lorna Slater, a Green MSP who is a coalition ally of the SNP. It makes sense strategically; most of Labour’s target seats will be ones currently occupied by the SNP. But in the end, the Tories could pay the price for Scottish Labour success.