GCHQ Review: Part 1, The World's Most Classified Blog and Other Stories

So, Richard Aldrich's book on GCHQ. This looks like it's going to be another in our occasional series of multi-part book reviews that nobody reads, as the book is nothing if not comprehensive. (It's a mere Laundry-esque 666 pages in paperback.) Apart from being packed with good things, like paper and words, as Spike Milligan said about his autobiography, I think it's undeniable that this is the best factual account of British signals intelligence you're likely to get. It practically bursts with detail and is clearly the fruit of an enormous effort of primary research, and a fair bit of the secondary kind too. If you want to know about the continuation of the First World War crypto effort into the inter-war era, the construction of the Hong Kong over-the-horizon radar site on top of a sheer cliff thousands of feet high and the number of Land Rovers the RAF Regiment lost over the edge, or exactly how many index cards Special Branch found in Geoffrey Prime's private database of young girls, it's here. This is in itself quite an achievement, given how much of this stuff remains classified.

Of course, what everyone wants to know about is the intelligence special relationship with the US and the other Commonwealth nations. You will not be disappointed. Aldrich argues that we're unlikely ever to find a smoking document, even after the release of what was described as the UKUSA agreement earlier this year - the terms of the alliance were repeatedly renegotiated, and its content is spread over many different documents. In fact, it might be more interesting to think in terms of the technical documents. He makes the excellent point that the alliance consists, in practice, of a set of shared operating procedures and technical standards, rather like the Internet, with the distinction that here everything is secret. Rather than gaining access to the IETF by making your work public, you gain access to the tribe of SIGINT by submitting to ever greater secrecy, in a sort of masonic career of increasingly complex rites. Crucially, wherever the documentation goes, the internationally agreed security requirements go with it. This, of course, has an impact on parallel technological decisions, but I'll come to those later.

This tribal nature - and in many ways it is tribal, with different agencies' membership in the relationship stemming from their alliance with the founding couple of Bletchley Park and US Naval intelligence - has important and counterintuitive effects on the politics of SIGINT. For example, the tribal leaders have frequently been keen to help their kin succeed in developing new technologies, extracting more funds from national budgets, and securing their secrets from their common enemies. On the other hand, they have also been very keen to prevent them from developing relationships that bypass the central alliance, and to restrict the degree to which they can secure their own traffic against the "level one agencies", GCHQ and NSA. All tribes, however, are in part mythical, and the status of the leader derives in part from the consent of the led.

In the early 1970s, for example, Henry Kissinger ordered the NSA and the National Reconnaissance Office to cut off intelligence sharing with Edward Heath's government (Heath's GCHQ director was, among other things, in the process of negotiating a special link between the Joint Intelligence Committee and the French equivalent). The British were horrified, but it's telling that the NSA itself was very suspicious of the move and took steps to undermine it - it seems that information kept reaching Britain via sharing with Canada and Australia. When the Yom Kippur war broke out, Heath retaliated by refusing to let SR71 reconnaissance flights land in the UK or at Akrotiri, and imposing conditions on U-2 operations from the UK, specifically that the imagery from them could not be shared with Israel.

In the 1980s, the Reagan government imposed a similar "cut-off" on New Zealand to protest their refusal to let US warships call without saying if they were carrying nuclear weapons. The New Zealanders were unexpectedly unimpressed, which was at least in part explained by the fact that the other alliance partners continued to pass information to them, and also by the fact that the New Zealand GCSB was a major analysis centre for traffic from Asia. Notably, GCHQ was collecting French traffic on their behalf as part of the Rainbow Warrior inquiry.

Over the years, the power-relationships within the alliance shifted with the varying scarcity of different resources. To begin with, in the heroic days of Bletchley Park, the UK had a strategic advantage based in its extremely scarce knowledge of cryptanalysis and computing. As the importance of computing and bulk data processing in general grew, this shifted towards the US; they had more money, and their own technology was improving fast. The result was that the Commonwealth partners essentially traded collection for analysis - we had territory, relationships, and collection platforms that the Americans didn't. That included some hideously dangerous overflights, submarine missions, and covert actions around the edge of the Soviet sphere of influence. Again, if you want to know what it was like sailing an old submarine into Polyarnyy harbour in 1959 without asking, it's here.

This oversimplifies; in fact, however much money the Americans threw at the problem, they didn't break the Soviet high-level ciphers between Black Friday in 1948, when the USSR carried out a forklift upgrade of their whole crypto network to end the VENONA codebreak, and the late 1970s. Information had to come, instead, from new forms of collection, targeting networks that weren't encrypted because they were thought to be secure, and by studying the electronic signatures of new weapons. As a result, the inter-allied playing field had a structural skew towards the British, who specialised in forward collection and in ELINT, building up an enormous library of Soviet radars and emplacing microwave listening stations in unlikely places. However, it's unlikely that this was realised at the time - it was all too obvious that Fort Meade was filling up with more and more computers, and it's not clear how honest they were about their successes or failures. There was a sort of technical cultural cringe on the British side.

The other new field was of course space. Starting in the late 1960s, the US began to collect much more of its signals intelligence from satellites, invulnerable to the political turmoil down below. However, this brought about another twist in the political relationship. The Americans had ELINT and COMINT satellites, the allies didn't. But when the RHYOLITE satellites, originally intended to spy on missile telemetry, started to pull in more and more data from the new microwave telecomms backbones, the NSA was forced to rely on its allies to deal with the mountains of data. That meant, among other things, a momentous step - intelligence sharing now included readout, letting the allied agencies point their dishes at the satellites and receive the stuff directly. (Incidentally, this is the purpose of Menwith Hill - it slurps intercept material from satellites and passes it to Cheltenham.)

At the beginning of the 1980s, then, the alliance was undergoing the sort of integration process that the founders of the European Union hoped to see. Rather than painful negotiations in high politics, technical interworking would result in a natural binding together. The system was evolving from the original hierarchical structure into a flatter network, with much greater interdependence. The Americans seem to have been aware that control was slipping away, and made efforts to assert traditional rights, for example by trying to impose the lie detector as part of the common security rules, which even Margaret Thatcher considered illiberal and unscientific. Some tribal practices didn't translate. The New Zealand cut-off was part of this, as was its failure - among other things, what was to happen about the New Zealanders seconded to Canada and the UK, and the Canadians and Brits in New Zealand? What would the US customers for Korean traffic processed at GCSB say when it ceased to arrive?

Interestingly, the US seems to have found continental Europe more interesting as a result. They made efforts to cooperate more closely with West Germany, while the Germans for their part were organising a new European alliance, and the UK was developing close links with the Mitterrand government's intelligence chief (while also helping the New Zealanders get information on his agents).

Yet another shift, possibly even more important than the end of the Cold War for the tribe, was now approaching - the end of the microwave network era and the dawn of widely available strong cryptography. Arguably, what is now scarce is code-breaking of any kind, again, and intelligence analysis; computer power has never been cheaper, while mass collection is much less practical outside one's own borders. In fact, pharaonic proposals like the Intercept Modernisation Programme may be better understood as a sort of atavistic harking back to the microwave era or even to the high Cold War's tunnels under Berlin and Vienna.

However, it's certain that they ain't going away. One thing that SIGINT has which other forms of intelligence don't is that it works, it produces physical output, yer actual primary-source documents - every day, as well as the formal, all-source intelligence reports on particular topics, the prime minister is also sent a wedge of selected quotes from the raw traffic. It's the world's most classified blog! Thatcher's civil servants referred to it as Comic Cuts (in the 1950s and 1960s, similar files were known as Blue Jackets or BJs - another way to make the president feel special, I suppose...), but she lapped it up, like they all do.

Daniel Davies once remarked that secret information is a drug - it alters your perception of reality and makes you feel superior to other people - and that it isn't usually considered wise to make important decisions on drugs. Here's the problem; whether or not the raw matter is actually useful, whether it's typical or misleading, whether GCHQ is breaking a lot of the target's traffic or none of the circuits that matter at all, it's incontrovertibly present. They will produce something rather than nothing.

In our next thrilling instalments: GCHQ and technology, overseas outposts, internal surveillance, and the future...