Are British teenagers suffering from an epidemic of mental illness? Yes, according to a ‘government-funded study’ which found that 24 per cent of 14-year-old girls are suffering from depression. This has been seized upon by critics of Conservative education policies; they see it as ‘proof’ that the increased focus on teaching children knowledge, as well as more frequent testing and the GCSE reforms, have literally driven children mad. ‘One in four girls is clinically depressed by the time they turn 14,’ reported the Guardian.

I’m sceptical about this and I took a look at the research carried out by the Centre for Longitudinal Studies, which is based at UCL Institute for Education. It involved asking 14-year-olds in the Millennium Cohort Study to fill out a questionnaire called Short Moods and Feelings. This is an American document that uses the Diagnostic and Statistical Manual of Mental Disorders’ criteria for depression. It lists 13 symptoms in the form of first-person statements like ‘I thought I could never be as good as other kids’ and asks respondents to indicate ‘Not true’, ‘Sometimes’ or ‘True’, depending on how often they’ve experienced these feeling in the past two weeks. A total of 11,394 children completed the survey, which is a respectable sample size.

The first thing I noticed is that the report doesn’t actually claim that a significant percentage of these children are suffering from depression, much less ‘clinical depression’, since no clinicians were involved. Rather, it says that 24 per cent of girls are ‘suffering from high symptoms of depression’, without providing a definition of ‘high’. If they were using the questionnaire as it is supposed to be used, they should have assigned a score to each answer, with different values awarded to ‘Not true’ (0), ‘Sometimes’ (1) and ‘True’ (2), and only claimed that children who scored 12 or above might be suffering from depression. Did they follow this methodology? We don’t know, but ‘high symptoms of depression’ sounds like a sleight of hand designed to conceal the fact that the number scoring 12 or above was much lower than 24 per cent.

A more fundamental problem is that the questionnaire is not designed as a diagnostic tool. Indeed, if you look it up you’ll see the following explicit warning: ‘This instrument should be used as an indicator of depressive symptoms and not as a diagnostic tool and therefore does not indicate whether a child or adolescent has a particular disorder.’

It may be that the researchers used the phrase ‘high symptoms of depression’ to avoid falling into this error, and not because they were trying to exaggerate the scale of the problem, but no such circumspection was shown by newspapers reporting it. The Guardian was far from alone in this respect. Here’s a sample of the headlines: ‘A quarter of all 14-year-old girls are depressed’ (The Independent), ‘One in four teenage girls are depressed’ (Daily Telegraph), ‘One in four 14-year-old girls is depressed’ (The TES). In fact, the research tells us nothing of the kind.

Another difficulty in using this study to try and ‘prove’ that Michael Gove’s education reforms are responsible for higher levels of mental illness is that it is the first time 14-year-olds have been surveyed in this way, so we don’t know if the level has climbed or fallen since 2010. Natasha Devon, a self-described mental health champion, wrote in the TES saying that it was ‘helpful to have some solid statistical analysis to support the notion that poor mental health in young people is increasing at a dramatic rate’ and claims the research shows that the prevalence of depression among 14-year-olds has ‘doubled’ in ten years. She evidently doesn’t know what ‘Millennium’ signifies in ‘Millennium Cohort Study’. It means that it’s a longitudinal study of a group of children born in 2000 so, by definition, the same researchers could not have surveyed a different group of 14-year-olds ten years ago and compared their recent findings with those. Ten years ago, the oldest children in the cohort were seven.

So treat this survey and the inflated claims being made on the back of it with a large dose of salt. It doesn’t tell us much about how prevalent depression is among teenagers, and it doesn’t tell us anything about whether it’s rising or falling.