A young man looks at a red chalk drawing of a muscly torso, made years before. He registers the residual heat of homoerotic longing in this ‘ancient pornography’, but has no idea he’s looking at his own father’s flesh, captured in youth. Johnny Sparsholt is the gay son of a closeted father, David. Or, he’s a man growing up into relative freedom whose father was mired in a corruption scandal with a side order of rent boys. How we view the past, and what we find in it, are questions at the heart of Alan Hollinghurst’s sixth novel, one of his most richly absorbing.

What is ‘The Sparsholt Affair’? Depending on your view, it’s ‘a dim nexus of provincial misconduct’ or ‘a national scandal’ – or, for someone intimately concerned, ‘a horrible mess in the life of a man he’d once adored’. Hollinghurst never brings us close to David Sparsholt: we first see him in wartime Oxford – an athlete among aesthetes, presumed heterosexual, but that proves negotiable. He becomes a patriarch and thriving Midlands architect, before being derailed by a scandal involving money, politics and illegal sexual activity. Its details are never explained, but its effects and prurient appeal inform the rest of the narrative, and never cease to beset Johnny, even late in middle age.

Such a knobbly name, Sparsholt. It’s ‘a word like a machine part’, muses someone early in the novel, ‘or a small hard sample, perhaps, of some mineral ore’. People stub their toes on its unyielding singularity all through the sixty years of Hollinghurst’s narrative; it even finds a late, faint echo in the porny Tumblr account of a chap called Snapstud. A lifetime of fraught concealment is followed by a spurt of unashamed self-exposure.

The Sparsholt Affair is nicely timed for the fiftieth anniversary of the Sexual Offences Act, a partial decriminalisation of gay relations. It’s hardly an unambiguous celebration, but it has allowed its author to burrow back into the past. The Swimming-Pool Library, Hollinghurst’s spectacular debut, also hinged on a buried sex scandal – though the foregrounded groinal antics largely stole the limelight. The hero was up to his plums in present pleasure; only gradually did he understand that the past was full of unfinished business for his generation of gay men, waiting to bite. This new novel is more overtly aware of the seep of the past: like the old Oxford cohort’s exchanges of memoirs, it is ‘a little experiment in history’.

In five ample sections, progressing from 1940 to 2012, Hollinghurst adapts the structure of the family saga, with one generation lapping after the other on a tide of marriage, progeny, inheritance. Here, however, family is a complicated term. There are long-lasting partnerships but also one-time affairs. Children grow up in tense estrangement from their parents; a lesbian couple solicit a gay man’s sperm donation (‘this nearly heterosexual act’). The extended stretch of time – taking up Johnny’s story from sheltered adolescent to young man entering haute bohemia to society portraitist – nurtures the generosity of Hollinghurst’s project. Following many of the characters from youth into old age allows reappraisal. Even spiky characters soften with the years. Ivan, a supporting character who sidles into a distinguished writer’s life as amanuensis, lover and eventually carer, initially seems manipulative. Queasily obsessed with the past, he truffles about in the past like the necro-biographer in Hollinghurst’s most recent novel, The Stranger’s Child, snaffling information about the old Oxford circle: ‘like most of the members of the now vestigial gang, she had a pocket of her own in his concertina files’. Johnny fancies him rotten, and it’s hard not to cringe when Ivan exploits this to accrue information on David, trading sex for scandal, their intimacy just ‘a game of closeness’. For Ivan, their weekend away ‘would be another chapter, an amusing one, in the folder he had long been keeping, marked “The Sparsholt Affair”.’

Yet Ivan’s middle years are kindly drawn. He’s staunchly protective of his elderly partner and his scrapbooks become a rich source of obituaries. All these histories and unreliable memories produce an undertow leading into the past, and an old man’s failing memory seems especially poignant in this story of accumulated knowledge and experience. Elsewhere a solemn young schoolboy arrives for a playdate and instigates an incongruous reading of Mary Rose, J M Barrie’s drama about a woman who periodically vanishes but reappears to a subsequent generation. The schoolboy himself later appears shirtless on the dance floor, metamorphosed from geek to gadfly.

Hollinghurst has always figured beauty as a commodity, entailing a negotiation that compromises both the beautiful and those who desire them. During the early sections of this novel, intimacies are used and discarded, and it seems as if there’s no such thing as sex without strings. Get your kit off and the bargaining begins. As a boy, Johnny wonders at ‘the dense tangled stasis of adult life, whose language he still hardly understood’, yet it is his sustaining innocence that ensures the narrative’s tenderness.

Hollinghurst’s prose has a velvety assurance: he writes with beady apprehension about subterranean shifts of feeling. As always in Hollinghurst’s work, desire pulses unapologetically through The Sparsholt Affair, but so does ageing: few other writers note the thickening of the waist so feelingly, or make you ponder armpit puckering and its discontents. The layered sense of the past is always alive in the present moment.