Somewhat rashly I took the opportunity, when meeting former heads of MI5 and MI6 on separate occasions, to ask each of them whether such clandestine organisations aren’t bound to be inept and corrupt. My question was not primarily prompted by the intelligence services’ long list of failures, of which the histories of the five Cambridge spies, recounted in this book with Richard Davenport-Hines’s usual vim and brio, are the most salient example. Instead I was suggesting that security agencies are inevitably unreliable because their stock in trade is deception and they cannot be subjected to the normal processes of public scrutiny and accountability. After all, as Lord Acton said, ‘Everything secret degenerates … nothing is safe that does not show how it can bear discussion and publicity.’ The ex-chief of MI6 predictably rejected my proposition. But to my surprise his MI5 counterpart agreed with it in principle and to some extent in practice, while adding that new controls virtually ruled out a repeat of the kind of treachery perpetrated by Anthony Blunt, Guy Burgess and Kim Philby.

Davenport-Hines is a staunch defender of MI5, the domestic (and colonial) intelligence agency, and MI6, otherwise known as the Secret Intelligence Service (SIS), which is responsible for foreign operations. He gives a character study of these organisations, which were formed shortly before the First World War as a result of spy scares whipped up by fiction writers such as William Le Queux and newspaper magnates such as Lord Northcliffe. Davenport-Hines charts their struggle against German and later Soviet espionage, and relates them to other institutions, such as the Foreign Office, the ancient universities and the national press. In a vivid panorama, similar to his recent account of the Profumo scandal, he depicts Britain’s intelligence officers not as ‘inflexible blimps, superannuated Indian policemen, expensively educated silly asses or obtuse reactionaries’, but as operatives who ‘were generally subtle, patient, responsive and astute’.

It is Davenport-Hines’s main contention that the Cambridge spies inflicted less damage on national security than did the hysteria that followed the escape of Burgess and the Foreign Office mole Donald Maclean to Russia in 1951. The public suspected an Establishment cover-up, failing to understand that prosecuting spies was less useful  than trying to turn them into double agents, and it swallowed the Marxist argument that their activities had been directed against an arrogant bourgeoisie on behalf of the international proletariat. The resulting miasma of mistrust has, in Davenport-Hines’s opinion, given rise to a populist hostility towards elites, an undermining of authority and a rejection of expertise, foolishly enunciated by Michael Gove, all disastrously eventuating in Brexit.

Davenport-Hines bases his case on wide research and illustrates it with a wealth of piquant anecdote. When Herbert Morrison was asked in 1940 whether he wanted to learn anything particular about his new  job as home secretary, he replied that he would like to watch a woman being hanged for murder. Earlier, the Tory MP Richard Briscoe had declared that if only Hitler and Mussolini could relax over a good game of bowls once a week, European peace would be untroubled. Meanwhile, the British minister in Prague explained that he had no friends among the Czechs because ‘they eat in their kitchens’. There is also a hilarious saga, alas too long to quote in full, involving the disposal of filthy pyjamas and disgusting socks that Burgess and Maclean left on board the cross-Channel ferry when they defected.

Davenport-Hines pens incisive cameos of the main dramatis personae.  Some spies, such as Ernest Oldham, who stole Foreign Office documents before the war, were drunks. Others, such as Jack Hooper,  who had the unique distinction of working for the SIS, the NKVD and the Abwehr, were squalid mercenaries along the lines of Mr Verloc in Conrad’s The Secret Agent. Still others, among them the atom spy Alan Nunn May, believed that to smash fascism it was vital to support the USSR. John Cairncross, the Cambridge fifth man, gave ULTRA decrypts to Moscow out of pride and a craving for status. George Blake, whose perfidy sent at least forty Western agents to their deaths, was a kind of Walter Mitty. Philby, though a Marxist ideologue, revelled in deceit for the power it gave him. Blunt aspired to become Britain’s commissar of culture. Davenport-Hines is especially  informative about the sex lives of his subjects. He suggests that few homosexuals were blackmailed into betraying their country, not even John Vassall, though much was made at his 1962 trial of his entrapment by the KGB. Burgess went so far as to exaggerate his promiscuity, though at Eton he had certainly indulged in a lot of what Anthony Powell called ‘manual labour’.

Of course, the yellow press needed no excuse to indulge in drooling spy mania. Without any evidence, Fleet Street branded Maclean a homosexual and Hugh Cudlipp, editor of the Sunday Pictorial, denounced the ‘spreading fungus’ of perversion. No McCarthyite witch-hunt occurred, but the Foreign Office did resolve to purge its ranks of gays under pressure from the FBI, which was obsessed with the subject – ironic in view of the fact that its boss, J Edgar Hoover, was a closet queen. Never willingly outdone in championing atavistic prejudice, the Daily Express excoriated homosexuals. Equally virulent was its creepy espionage ‘expert’ Chapman Pincher, memorably described by the historian E P Thompson as a ‘kind of urinal in which, side by side, high officials of MI5 and MI6 … stand patiently leaking in the public interest’. As for the Daily Mail, it attacked a diplomat opposed to the Korean War, Sir John Pratt, on the grounds that his younger brother was Boris Karloff, the actor who played evil monsters in Hollywood films. And all this before the appearance of Rupert Murdoch, who thought The Sun’s headline on the outingof Blunt – ‘Tinker, Tailor, Poofter, Spy’ – was the acme of wit.

However, Davenport-Hines mars hisnarrative by odd and tendentious assertions. He remarks that Churchill had an ‘absolute incapacity for irony’, which could hardly be further from the truth – in Churchill’s autobiography My Early Life, for example, the mature statesman casts a consistently ironic eye over the brash doings of his younger self. At the very time when the harm done by privatisation and deregulation has become crystal clear, Davenport-Hines describes the nationalisation of industries such as the railways as ‘misguided’. Moreover he concludes that the denigration of an honest and efficient administrative cadre of diplomats and secret service officers by Labour politicians such as George Brown and Richard Crossman was ‘intended to neutralize, subdue and weaken government departments for the benefit of socialist programmes. The aim was to force civil servants into submission to the will of politicians claiming to represent the will of the people.’ This sounds a bit like a conspiracy theory itself – of the kind that vitiated the security services after the exposure of the Cambridge spies and encouraged the paranoid activities of men such as Peter Wright, author of Spycatcher.

Needless to say, the reluctance of spy chiefs to speak truth to power was amply demonstrated in the run-up to the Iraq War in 2003. Eliza Manningham-Buller, director-general of MI5, was an honourable exception and she exemplifies Davenport- Hines’s point that the long-standing bar on female membership of the security services was a grave source of weakness. But his further contention that they were not impaired by class exclusivity does not hold water. To be sure, not all spies came from public schools, but it would have been strange indeed if, at a time when Old Etonians constituted a quarter of the Foreign Office’s annual intake, conventional social prejudices did not pervade the whole of Whitehall. Even Dick White, who headed both MI5 and MI6, acknowledged in 1961 that there had been ‘too much reliance on “the old boy network” in vetting intelligence officers’. And Hugh Trevor-Roper, his former colleague, insisted that many were recruited from Brooks’s or Boodle’s.

In fact, Trevor-Roper (whose letters Davenport-Hines is editing) provides an essential corrective to this enjoyable but partisan book. He complained about the mediocrity of intelligence officers hired as a result of patronage and nepotism, a charge endorsed in the Radcliffe Report (1962), which called MI5 and MI6 ‘the natural home of the incompetent’. He criticised the damaging rivalry between the security organisations, the activities of which were necessary for the protection of the public, and said that secrecy was all too often used as a cloak for their scandals. He warned about the inherent risks of all secret societies and the occupational hazards to their members of fantasy and fraud. But even Trevor-Roper could not suggest an answer to the perennial problem of how to manage secret agencies in an open society.