For non-Catholics, the most luridly fascinating aspect of Catholicism is confession. Telling your inmost sins — and we know what they are — to a male cleric, eh? In a darkened booth. How medieval is that?

Well, the fantasies that people who never go to confession nurse about it are about to be shored up by a new book on the subject by the Catholic author John Cornwell. It’s called The Dark Box: A Secret History of Confession. On the cover is a scary-looking picture of a confessional — not somewhere you’d take the children, frankly, but right at home in a Hitchcock movie.

John Cornwell is a friend, and moreover an intelligent and thoughtful man, but if ever there were a book that played to its gallery, it’s this one. The thing is riddled with sex, including child abuse, which is plainly working miracles from the point of view of publicity. I’ve been asked to review it by three newspapers; another is carrying extracts. It’s a cue for every ex-Catholic in the commentariat to discuss their angst about sharing their sex lives with elderly priests. The trauma!

The last time I went to confession was on Christmas Eve, in my home parish in Ireland. Confessions were from 10 until noon. There had already been a penitential service that week, so I assumed anyone who had anything on their conscience would have gone then. To my surprise, there were priests in three confession boxes and two rows of people, constantly renewed, waiting their turn in front of each of them. The penitents weren’t all elderly, either. There were teenagers and children.

My confession went just fine, thank you. There was, as ever, a grille between me and the priest; the box was dark but not intimidatingly so, which is how I like it. The priest didn’t dwell on stuff relating to sex; he homed in on the opportunity of opening ourselves up for divine forgiveness. From the point of view of a non-Catholic, it would have been a bit dull. But when I left, with my penance (an Our Father and Hail Mary) I felt that curious lightness I always feel after I’ve been told: ‘I absolve you from all your sins.’ Later, I asked the parish priest how many people came normally. ‘A dozen, two dozen a week,’ he said. ‘But people want to talk about all sorts of things. There’ll always be the need for that.’

That’s just one parish. And plainly in the church in Britain and Ireland, and in comparable countries, confession has changed inexorably in the last couple of decades. The lines of penitents that turned up in every Catholic church every Saturday aren’t there any more; the young are even more conspicuously not there. Quite a few of the children who went to first confession with my son a couple of years ago haven’t been since. In my church in London, penitent numbers are small and steady.

But things are changing. I’m not entirely sure about the ‘Pope Francis effect’ — since his election there’s apparently been an increase in Mass attendance — but anecdotally there’s talk of an upturn in confessions too. In Westminster cathedral, there’s been a discernible increase in the numbers. The cathedral attracts transient penitents who don’t want to tell their sins to a priest they know. (Shallow, I know, but I’m just the same.) It always has people waiting in line. A priest I know who served in Bristol for three months over the summer said that in his church — again, one attended by irregulars — he had daily confession for a half hour every day, and he was busy all that time. ‘Quite a number were young men,’ he said. ‘And they had taken the trouble to prepare, to make very good confessions.’ The same goes for the Cambridge University chaplaincy. It’s bringing in an extra Mass on Fridays, and at the request of the students, it’s going to be preceded by confession. ‘These are balanced, normal young people,’ said the chaplain, Mark Langham, ‘and they make wonderful confessions.’

The opposite of a good confession is the laundry-list approach, whereby people who feel obliged to attend come up with a catalogue of sins that they repeat week after week: a failure to remember prayers, say. I know several Catholics who say they made things up. A few of my acquaintances stopped going to confession when their sex lives so plainly didn’t square with Catholic ideals that there seemed no point, really.

‘Patterns of usage have changed,’ said one priest friend. ‘Confession is used less but it’s not used less well.’ Advent or Holy Week (pre-Easter) penitential services are well attended, as are those before Christmas; in north London, one priest said wonderingly that he had a packed church, all ages. At these services, there are prayers and readings, followed by individual confessions in public view, but not in public hearing. One woman I know said that her daughter, whose home life had been a bit rackety, had gone to a penitential service just before Christmas and when she went up to the priest for confession she hadn’t been able to stop herself crying and crying; the poor priest said to her that she was welcome to come and see him any time.

That’s another thing you hear, that people still come to priests to talk things over; confession isn’t counselling but if that’s what you want, you can have it, so long as the two aren’t conflated. And while things like contraception don’t register on the radar, other things do; one priest remarked that several young male penitents were troubled about their use of pornography sites.

Plainly, many unmarried people in sexual relationships and gay people have no intention of changing their way of life but still want to confess things that do trouble them. ‘That’s fine,’ said one priest. ‘That’s the Pope Francis approach. If there’s one aspect of life that’s problematic, let’s have a look at something else if that helps. If it’s uncharity that’s bothering you, let’s look at that. Confession is about your whole life. You’re not defined by one sin.’ Another priest remarked: ‘People in those situations don’t confess what they don’t regard as sins. And it’s not for me to ask whether they’ve forgotten anything.’

Cardinal Cormac Murphy-O’Connor, the archbishop emeritus of Westminster, takes a robust approach: ‘It’s like the pebble in your shoe; normally it’s one thing that’s bothering you and you should come to confession about that. Or maybe two or three things. If I were bishop now, I’d advise people just to come to confession twice a year, perhaps before Christmas and Easter. But of course for some people more frequent confession is very helpful.’ It’s a big change from the old way of doing things, whereby Catholics were exhorted to go to confession every week. Cardinal Murphy-O’Connor says the Pope should bring the bishops of the church together to deal with the whole issue of confession. ‘I think he should hold a consistory about it,’ he says. ‘When I see him I shall suggest that.’

In a way, it would be no big deal. The form confession takes is less important than the thing itself, the forgiveness of sin. As I say, I’ve never left the dark box without feeling lighter. Call it grace, if you like.