Once again, question marks surround Jeremy Corbyn’s leadership. This is not new. While I was at 10 Downing Street, with the small but significant possibility of a sudden Corbyn departure, we spent some time exploring the electoral impact of who might come next. To work out who might put up the best fight and how best to counter them, I discussed potential candidates in focus groups, played videos to voters, and polled frontbenchers’ perceived attributes. The most consistently effective potential leader? Shadow chancellor, John McDonnell.

This may seem surprising – and as a Conservative it was a painful discovery. But he ‘focus grouped’ remarkably well. Voters described him as ‘strong’, that he ‘knows what he is doing’ and that he understands the economy. When I played interview footage, including to those who do not know who he is before the session, respondents nodded along, seeing him as confident and competent.

This competence matters. Jeremy Corbyn’s deep unpopularity is not because of his past and his links to unsavoury characters. His background did not register with voters in the 2017 general election, nor does it now.

Instead, voters dislike Corbyn because they think he is fundamentally a bit useless. Time after time, they have seen him struggle to control his party, fail to take clear positions, and handle Brexit haplessly.

People view McDonnell differently. He might have the same, if not more, disconcerting past. But whereas Corbyn is described as the ‘wet rag’, McDonnell is the ‘bank manager’ with a strong grip.

Labour also has to work out how to manage its base. Unlike the Conservative core vote, which is mostly homogenous, Labour’s splits into two groups. At No. 10, we called these ‘Labour’s Left’ – younger, Remain voters, heavily left-leaning and socially liberal – and ‘Labour’s Working Class’ – older, inner city Leave voters who tend to be in unskilled work or on social benefits.

A Labour leader needs to be able to excite and mobilise the former group, while also keeping the latter on side. Many of ‘Labour’s Left’ voted Remain, and they certainly want a leader who argues strongly for staying in the EU. But they also want someone who takes a radical and unapologetically left-wing approach to policy at home. In numerical terms, it was politicians on the Corbynite wing of Labour, not the moderates, who scored best across these two groups.

It is worth remembering too that the charge that a party is ‘radical’ is a net positive with core and swing voters alike. This is an electorate desperate for change, who look around and see other people benefitting while they feel they work hard and do not get rewarded.

One thing is for sure. Jeremy Corbyn is a busted flush. His ratings are at historic lows. Minds are made up. His closest advisers are abandoning him. It is hard to see how he recovers enough in any general election campaign.

But people underestimate his right-hand man at their peril. If the appeal of Corbyn is over, do not assume the allure of Corbynism is. John McDonnell is a key proponent of that form of politics. And certainly when I was doing the Conservative party’s polling, it was McDonnell I was most scared of.

James Johnson was senior research and strategy adviser to the prime minister 2016-19