Churchill was wrong for most of his career, you know...

This Ha'aretz piece is interesting for the insight it gives into Israeli policy and especially into process, but also for a couple of other things. Notably, it's remarkably frank about the Obama administration deliberately trying to stop Netanyahu going to war, and the role of dodgy casino guy Sheldon Adelson in both US and Israeli right-wing politics, and it provides the new information that the Americans have given up on the formal diplomatic channel and concentrated on influencing the Israeli military directly, on a brasshat to brasshat basis. The implied conclusion is that the IDF leadership are interested in external reality while Bibi is too busy being Winston Churchill, and further that they are interested in getting information from the Americans about what their own prime minister is thinking.

Also, Netanyahu considers himself an expert on US politics. The danger here is that the America he is an expert on may not be the same America everyone else is dealing with. If, as I suspect, he is getting a lot of his information from his Republican contacts, he's living in an alternate universe. In so far as people like Sheldon Adelson are impressed by US politicians who know Bibi Netanyahu personally, his contacts are literally being paid to tell him what he wants to hear. It's ironically similar to Bush before the Iraq war, just with the stove-pipe reversed.

However, I was astonished by this quote:

While the Fifth Fleet of the U.S. Navy is operating in the Straits of Hormuz, just as the Pacific Fleet was anchored at its home base near Honolulu on the fateful morning of December 7, 1941, the two instances are not really comparable.

Well, no, they're not, are they? Some tabloid journalists keep a few paragraphs of general-purposes "sexy" in a file they can drop into a story as required and just change a couple of parameters to fit. This sounds like the same thing, but with Churchill!

Meanwhile, Colin Kahl, and this. It does look like there's a coordinated push-back against the bullshit, which is good news for those of us who remember 2002. The US Navy bombs Iran...with love. Of a purely Platonic form between comrades of the sea. Oops. while also bringing the carrier back.

US policy does look like it's trying to achieve three goals - 1) no war with Iran, 2) reassure the GCC countries (so they don't start one), 3) restrain the Israelis (without pressing so hard they freak and start one). These are partly contradictory, but then what isn't? Certainly, the combination of being ostentatiously nice to Iranian sailors while also sailing a giant carrier up and down the Gulf does fit the needs of 1) and 2).

a short telegram, or a very long tweet

Everyone's linked to Mark Perry (of Conflicts Forum/Alistair Crooke fame)'s piece on Israeli spooks running around Baluchistan posing as the CIA already, but I will too as it's very interesting indeed. I'm not sure what their bag in this is, other than the notion of "always escalate" and hope to profit from the general confusion.

But what's really interesting is what the story is doing out there now. Here's Laura Rozen's write-up, which introduces the suggestion that they may have represented themselves as being from NATO and notes that a leader of the organisation said as much on Iranian TV before being executed. Meanwhile, the Iranians write to the Americans accusing the CIA of being behind the assassination of another nuclear scientist.

On Twitter, she suggests that the scientist wasn't killed by the Americans (i.e. presumptively by the Israelis, or by people working for them wittingly or otherwise), and that this was staged specifically to queer the possibility of reviving the Iran-Turkey uranium swap deal. (You do wonder what George F. Kennan would have made of diplomatic tweeting.) Further, we know that a back-channel has been set up.

Disclosing information about the Israeli operation in Baluchistan might be a smart way of establishing trust between the US and Iran. Obviously, information about terrorists running about blowing stuff up and killing people is of value to Iran. Information that it's the Israelis is obviously congenial to Iran. Crucially, burning an Israeli spy network is costly to the Americans and not something they would do lightly (the Perry piece is a monument to important people trying all they could to do nothing). In that sense, it is a meaningful signal - much more convincing than mere words. Presumably, Perry's role at Conflicts Forum and with Arafat makes him a convincing postman into the bargain. And third-party spies are just the sort of thing that enemies can bond over. I recall reading about the IRA and the UVF staging a joint investigation to find informers in the early 1970s.

Another dose of speculation - if Baluch rebels were meeting with people who they thought were from NATO, was this plausible because NATO was in fact paying them off to leave the Karachi-Quetta-Kandahar supply route alone?

The intersection of electronic warfare and mall management

Here's something interesting. You may remember this story from back in November about the CIA spy network in Lebanon that met at a Pizza Hut they codenamed PIZZA, and which was rolled up by a joint Hezbollah-Lebanese military intelligence investigation. The key detail is as follows:

U.S. officials also denied the source's allegation that the former CIA station chief dismissed an email warning that some of his Lebanese agents could be identified because they used cellphones to call only their CIA handlers and no one else.
...
Lebanon's security service was able to isolate the CIA informants by analyzing cellphone company records that showed the numbers called, duration of each call and location of the phone at the time of the call, the source said.

Using billing and cell tower records for hundreds of thousands of phone numbers, software can isolate cellphones used near an embassy, or used only once, or only on quick calls. The process quickly narrows down a small group of phones that a security service can monitor.

If the top paragraph is true, it would have been catastrophically ill-advised. Even somebody special, like a CIA agent under diplomatic cover, has a relatively large number of weak ties to normal people. This is the reverse of the small-world principle, and is a consequence of the fact that the great majority of people are real human beings rather than important persons. As a result, things like STELLAR WIND, the illegal Bush-era effort to analyse the whole pile of call-detail records at AT&T and Verizon in the hope that this would find terrorists, face a sort of Bayesian doom. We've gone over this over and over again.

However, phone numbers that only talk to special people are obviously suspicious. Most numbers with a neighbourhood length of 1 will be things like machine-to-machine SIMs in vending machines and cash points, but once you'd filtered those out, the remaining pool of possibles would be quite small. It is intuitive to think of avoiding surveillance, or keeping a low profile, but what is required is actually camouflage rather than concealment.

There are more direct methods - which is where electronic warfare and shopping mall management intersect.

Path Intelligence, a Portsmouth-based startup, will install a network of IMSI-catchers, devices which act as a mobile base station in order to identify mobile phones nearby, in your shopping centre so as to collect really detailed footfall information.

Similarly, you could plant such a device near that Pizza Hut to capture which phones passed by and when, and which ones usually coincided. Alternatively, you could use it in a targeted mode to confirm the presence or absence of a known device. Which makes me wonder about the famous Hezbollah telecoms network, and whether it was intended at least in part to be an electronic-intelligence network - as after all, nothing would be a better cover for a huge network of fake mobile base stations than a network of real ones.

Meanwhile, this year's CCC (like last year's) was just stuffed with GSM exploits. It really is beginning to look a lot like "time we retired that network".

Now that's what I call lobbying

In the recent case of Liam Fox and Adam Werritty, there was an issue that the news media spent an enormous amount of time and effort dancing around with innuendo, newspaper code, and carefully lawyered prose. It is a fact that the word "lawyered" is to the word "lawyer" as the word "doctored" is to the word "doctor". Without understanding this hidden and sordid side of the issue, you would have been seriously misinformed. The matter was very sensitive, and there was an excellent chance of getting sued and probably also demonised as being deranged by shameful prejudices.

I refer, of course, to whether or not the Defence Secretary's private office was having unprotected sex with other defence secretaries' private offices.

It took a while to surface this at all - the Guardian let a wee squeak out on Thursday, and eventually it was the Sindy that took the plunge and surfaced it in the same way you surface a submarine, with an enormous roar of compressed air thundering into the ballast tanks under pressure while the nuclear reactor cranks up to full power. It's a must read.

The fact that Werritty's freebies included trips to the Herzliya Security Conference paid for by pro-Israeli lobbying groups should have been a screaming giveaway, but then, that's what a good cover story is for. I presume that was what the Sindy eventually followed up.

I mentioned this element of the story to Daniel Davies earlier in the week. I can offer no special insight except for the enduring value of pattern recognition. This has, after all, happened before in recent memory, with really bad consequences.

Consider Mr. Michael Ledeen and the affair of the weapons of mass destruction. Mr. Ledeen, a professional neoconservative, claimed to have intelligence about Iraqi efforts to acquire uranium and various other things, which came from his contacts in Iran, some of whom were recommended to him by his contacts in Israel, one of whom, Larry Franklin, was convicted of spying for Israel in the US State Department. Ledeen believed these contacts to be renegade members of the Iranian secret service. (He had never visited Iran, and I think to this day never has, and he doesn't to the best of my knowledge speak Persian, so how he would have known is beyond me.) The CIA, for its part, believed that this was partly true. They just disagreed with the "renegade" bit. But Donald Rumsfeld had deliberately decided to ignore the CIA, so Ledeen's intelligence was accepted. However, that wasn't the end of the story. At some point, the Department of Defense became suspicious and called in its own Counter-Intelligence Field Activity to investigate.

At this point, a thick curtain of secrecy was drawn down on the story, even if we did eventually get the Phase IIA report. Whatever CIFA found out, Ledeen was able to introduce the famous forged documents on uranium from Niger, which seem to have come from the Italian secret service, as being Iranian information with Israeli approval, and this was used in the even more famous dossier.

I wouldn't be at all surprised if old blogging chum from way back in the day, 2004, Laura Rozen hasn't also had this thought, as she was instrumental in digging into the whole Ledeen affair and she's too smart to miss it. Also, hilariously, she and Spencer Ackerman had the honour of being targeted by Ledeen's mates in Silvio Berlusconi's intelligence service with a scurrilous smear-campaign. I should probably hat-tip the lady's Twitter feed.

Note the elements of the story. Ledeen is a semi-official adviser with special, privileged access to policymakers. He is outside the formal requirements of government service, but has access inside it. He is seen to have special access to an important ally, and therefore to be trustworthy. A third party observed this, and took advantage of it to introduce information (or rather, disinformation) into the policymaking system. Does anybody see a pattern here? Similarly, Werritty was offered privileged access from outside the government firewall because he was ideologically congenial. It seems that this was considered acceptable because the influence exerted came from a country considered friendly. But then, there were the rogue Iranian intelligence agents, or were they just ordinary Iranian intelligence agents?

In May 2009, Mr Werritty arranged a meeting in Portcullis House between Mr Fox and an Iranian lobbyist with close links to President Ahmadinejad's regime. In February this year, Mr Werritty arranged a dinner with Mr Fox, Britain's ambassador to Israel, Matthew Gould, and senior political figures – understood to include Israeli intelligence agents – during an Israeli security conference in Herzliya, during which sanctions against Iran were discussed. Despite Mr Werritty having no official MoD capacity, an Israeli source said there was "no question" that Mr Werritty was regarded as anyone other than Mr Fox's chief of staff who was able to fix meetings at the highest levels, and was seen as an "expert on Iran".

Well, at least Werritty actually went to Iran. Unfortunately this is the worst of the story, as it seems he was going round encouraging Iranian dissidents, or people he thought were Iranian dissidents, and promising them British support. This is really incredibly, shamefully irresponsible - he could have got people killed, and it cannot be ruled out that he did, although it's also quite possible that the whole affair was just a massive exercise in bullshitting and wanktankery.

Probably he really believes that he was in contact with the opposition. I'm fairly sure Ledeen doesn't think he's an Iranian agent either. This is where this classic Onion article comes into play. As I said at the time, why *do* all these Iranian agents keep sucking Michael Ledeen's cock?

It is all reminiscent of Bruce Schneier's thoughts on what happens if you create a backdoor into some computer system, so people like us can get in and out without anyone noticing. The problem is that once you do that, it immediately becomes the biggest security threat to the system as anyone else can use it too. Once this new interface to the MoD was created, with Werritty accepting connections from the wider Internet and forwarding them to Fox, of course it attracted dubious actors. Hence the parade of various people trying to sell aircraft spares and dodgy encryption software to the military or to get someone's knighthood expedited.

For my next trick, what parallels do you see between Werritty's role with Liam Fox and those of Andy Coulson and Neil Wallis with No.10 Downing Street and the Metropolitan Police (and of course the Conservative Central Office) respectively? Remember that both of them were at various times funded by third parties. Further, is it not interesting that the same key Conservatives who defended Coulson to the bitter end - George Osborne and Michael Gove - also tried to save Liam Fox? (Jonathan Freedland seems to have sensed something here - check out the reference to "Cheneyite Tories".) And is it not even more interesting that George Osborne actually recommended Andy Coulson for the job? And is it not completely fucking outrageous that William Hague, Atlantic Bridge board member and Foreign Secretary (I think this is the right order of precedence), dares to claim that proper Cabinet government is back in the midst of this berserk threat-chaos?

tasks

I'm not quite as sceptical as some about this. However, it's not clear to me how this differs from the sort of thing UNOSAT does all the time - here's their analysis of imagery over Abyei, the key border area between North and South Sudan. Actually it looks like the "Enough Project" is going to be using UNOSAT imagery itself, going by UNOSAT's own website.

If you follow the link you'll see that they have more than reasonable capability (50cm resolution) and that they routinely observe the presence of refugees/displaced persons and returnees, construction, and the like. There's obvious relevance to an effort to monitor potential conflict along the border, especially as oil prospecting is an issue. You can't easily hide oil exploration from a satellite that can resolve objects 50cm across.

However, the downside is that the UNOSAT report is comparing images over a two-year period. I would suspect that they will need much more frequent passes to be operationally responsive, which is where the costs get interesting.

Also, I've just been over to the website and it's a bit of an unstructured clickaround. What I've always liked about MySociety sites is that they all have a function - FixMyStreet reports things in your street that need fixing, WDTK issues Freedom of Information Act requests, TWFY looks up information on MPs, TheStraightChoice logged what candidates promised and said about each other during their campaigns. DemocracyClub, for example, worked because as soon as you logged in it gave you something to do and some feedback about doing it, and then it hassled you to do something more. It had structure.

Notoriously, if you don't give volunteers something to do as soon as they show up, they'll wander off. It is nowhere easier to wander off than on the Internet. And so there's a button to twitbookspace it and a donation link. There isn't, however, a to-do list or, say, a list of pairs of images that need comparing.

review of a movie that doesn't exist yet

I think most of my readers also read Patrick Lang's blog, but I think this guest post is the best thing yet written on the Taliban/SIS/McChrystal/Petraeus fake sheikh affair. Really, there's a great movie to be made here - the multiplicity of motives, the ironic contrast between the absurd story and the deadly serious interests and emotions that drive it forward, the eternal ambiguity of the relationship between the manipulator and the manipulated.

The ISI comes out of it as being dastardly clever, but in a deeply futile way. They succeed in preventing a dangerous outbreak of peace and sanity, but what have they gained? The wars grind on, the butcher's bill ticks up, the fantasy of a Pakistani empire of trucks and pipes across the Hindu Kush is as far away as ever, the Indians continue with their industrialisation across the other border.

The Americans come out of it as being well-meaning but naive. After all, they only get into this story because they want peace. So does the real Taliban leader. They both share a sort of big, stupid nobility.

The British do almost as badly as the ISI; not only do they end up being the dupes of the piece, they do so without the saving grace of having good intentions. They're as naive as the Americans but more underhanded. SIS gets involved purely as a way of sucking up to the Americans and putting one over its real enemies, GCHQ, Her Majesty's Forces, MI5, and the main-line Foreign Office diplomats. The Government is desperately keen on the project for similarly base reasons - to suck up to the Americans, to grab at an opportunity to solve its problem in Afghanistan, and of course to embarrass the Labour Party. Of course, it would have been a brilliant political fix had it come off - but the master manipulator is not Bismarck but William Hague.

The fake sheikh, meanwhile, is a classic example of the Pinocchio/Hauptmann von Kopenick theme - the puppet of bigger forces who becomes a power in his own right. Without his successful performance, of course, none of the many expectations curling around the tale have a hope of happening. His agency is real, and his character expands to fill the role. The fact that the whole project is an exercise in theatre is interesting in itself - a film within the film. The actors in the film are, of course, puppets of the script and the direction, and it is a work of fiction. The enduring purpose of the theatre and the cinema, however, is that works of fiction have real influence on their audiences. Like the fake sheikh.

After all, the grocer of Quetta (not a bad title) is the only character in the drama who successfully pursues his interests. He gets some interesting time off away from his bazaar stall, and even gets rich. You could play this as the ordinary man who succeeds in making fools of the powerful who insist on involving him in their schemes, or perhaps as a microcosm of all the people who are getting rich off the continued war, Mother Courage rather than Kopenick. Alternatively he could be killed off, casting the whole thing as an utterly bleak tragedy. However, arguably the classic in this vein is The Third Man and that sticks with the tragicomic.

killing data.gov.uk, and thinking aloud about mapping the lobbysphere

So the government thinks this is clever. They also think it constitutes a "searchable online database". It is not searchable, nor is it a database. It is a collection of links to department web sites, some of which actually lead to useful documents, some of which lead to utterly pointless intermediary pages, some of which lead to documents in a sensible format, some of which lead to documents in pointlessly wrong formats, and some of which lead to PDF files. It provides no clue how often this data will be released or when or where. The URIs sometimes suggest that they might be predictable, sometimes they are just random alphanumeric sequences. Basically, what he said.

Meanwhile, very few of these documents have made it onto data.gov.uk, the government's data web site (pro-tip: the hint is in the name) which provides all that stuff out of the box. This is not just disappointing - this is actively regressive. Is it official policy to break data.gov.uk?

Anyway, I've been fiddling with NetworkX, the network-graph library for Python from Los Alamos National Laboratory. Sadly it doesn't have a method networkx.earth_shattering_kaboom(). I've eventually decided that the visualisation paradigm I wanted was looking me in the eye all along - kc claffy's Skitter graph, used by CAIDA to map the Internet's peering architecture.

The algorithm is fairly simple - nodes are located in terms of polar coordinates, on a circular chart. In the original, the concept is that you are observing from directly above the north or south pole. This gives you two dimensions - angle, or in other words, how far around the circle you are, and radius, your location on the line from the centre to the edge. claffy et al used the longitude of each Autonomous System's WHOIS technical contact address for their angles, and the inverse of each node's linkdegree for the radius. Linkdegree is a metric of how deeply connected any given object in the network is; taking the inverse (i.e 1/linkdegree) meant that the more of it you have, the more central you are.

My plan is to define the centre as the prime minister, and to plot the ministries at the distance from him given by the weighting I'd already given them - basically, the prime minister is 1 and the rest are progressively less starting with Treasury and working down - and an arbitrary angle. I'm going to sort them by weight, so that importance falls in a clockwise direction, for purely aesthetic reasons. Then, I'll plot the lobbies. As they are the unknown factors, they all start with the same, small node weighting. Then add the edges - the links - which will have weights given by the weight of the ministry involved divided by the number of outside participants at that meeting, so a one-on-one is the ideal case.

When we come to draw the graph, the lobbies will be plotted with the mean angle of the ministries they have meetings with, and the inverse of their linkdegree, with the node size scaled by its traffic. Traffic in this case basically means how many meetings it had. Therefore, it should be possible to see both how effective the lobbying was, from the node's position, and how much effort was expended, from its size. The edges will be coloured by date, so as to make change over time visible. If it works, I'll also provide some time series things - unfortunately, if the release frequency is quarterly, as it may be, this won't be very useful.

Anyway, as always, to-do no.1 is to finish the web scraping - the Internet's dishes. And think of a snappy name.

2006 again, and a brief history of recent wrong

Adam Elkus has a piece out entitled The Hezbollah Myth and Asymmetric Warfare, in which he criticises what he sees as a tendency to over-rate the power of guerrillas in the light of the 2006 war. Having read it, I think the real question here is about expectations and goals. Hezbollah didn't defeat the Israelis and hold a victory parade in Tel Aviv, but then nobody least of all them expected or aimed for that. The outcome of 2006 can only be understood in the light of a realistic assessment of the conflict parties' capabilities, interests, and priorities. A score draw is a much better result for Stoke City against Manchester United than it is for Manchester United against Barcelona.

For Hezbollah, the first and overriding goal was surely survival - as it is for everyone, it's even the title of the IISS Journal - followed closely by survival as a force in Lebanese politics, survival of their capability to maintain their self-declared insecurity zone in northern Israel, and finally, inflicting casualties and costs on the Israelis in order to create a deterrent effect. In that light, the result of 2006 was surely just as good from their point of view as they made out - they came away still in the field, still firing rockets, and with their status in Lebanese politics enhanced.

For Israel, well, perhaps one day they'll work out what their strategic aims were.

Elkus argues that the tactical situation at the point when the UN ceasefire went into effect was favourable for Israel, and that had the war gone on they might have done better. This is possible. However, it's also very common for wars to end like this. The Israelis' campaign in 1967 was designed, once they got the upper hand, to get to the Canal and onto the Golan before the UN blew the whistle - one of Ariel Sharon's frequent blind-eye manoeuvres in 1973 was also intended to complete the encirclement of the Egyptian 3rd Army before the UN ceasefire went into effect. The Indian plan for the 1971 war was explicitly intended to take Dhaka before a ceasefire was imposed. More recently, the Russian operation in Georgia was subject to a similar deadline. International intervention is part of the environment, and only fools wouldn't take it into account as a planning assumption.

An interesting sidelight on this, also from Elkus, came up in a parallel blog debate about "network-centric warfare" - he pointed to this gung-ho but good piece about the action in northern Iraq in which John Simpson was blown up. What struck me about it, however, was more that it was an example of this kind of thing - which should certainly make you think about 2006, especially in the light of this.

Tangentially, Sean Lawson's essay on the history of "network centric warfare" is well worth reading, especially for the way so many US officials in 2001-2006 seem to have been competing to see who could validate all the most extreme stereotypes of themselves the fastest, and more broadly on the way a basically sensible idea can become a sort of gateway drug to really insane strategic fantasies.

Cebrowski talked of a “booming export market for…security” and warned those who would resist, “If you are fighting globalization, if you reject the rules, if you reject connectivity, you are probably going to be of interest to the United States Department of Defense” (Cebrowski, 2003c).

Measured against the sort of capabilities the NCW thinkers knew they had, and the kind of goals they dreamed on the basis of them, what possible results wouldn't look like failure? Compared with the enormous arrogance of this vision - they really did want everyone who thinks the CIA wants them dead, dead - what resistance wouldn't look like success?

GCHQ Review, Part 5 - The Future and some Current Relevance

A major philosophical difference between the UK and USA halves of the SIGINT tribe, and between the tribe and the military, was who the intended customer for intelligence was. The Americans were traditionally very keen on bringing everything back to Fort Meade for processing and analysis, and then feeding intelligence reports to the top level of government. As very often, the British followed suite, but only up to a point. GCHQ as an institution was traditionally very concerned with its status as a direct contributor of intelligence to the core executive, co-equal with MI6, the diplomats, and the armed forces' Defence Intelligence Staff. In fact, as we saw in part one, in some ways it had greater independence and status - as well as its own private diplomacy with the Americans, it also has the unique privilege of sending the prime minister intelligence outside the formal processes of the Joint Intelligence Committee machinery.

In practice, though, it was often more interested than the Americans in pushing information forward to the military in the field or to diplomatic posts. This was influenced by the British specialisation in ELINT, which tended to be more interesting to the military and more dependent on collection from their ships or aircraft, and also by the Bletchley heritage. ULTRA's triumphs weren't just about Alan Turing or about computers; a huge problem that had to be solved to make it useful was the distribution of highly secret information to the army in the field in near real time. (A key motivation was that GCHQ was well aware that the Germans were in the habit of breaking Allied cyphers, and then transmitting the results over their ENIGMA and FISH radio networks - allied traffic turned up in the take all the time.)

It's probable that a major reason why GCHQ wasn't more like that, rather than less, was that the American approach was useful politically. Supplying the Cabinet directly obviously helps to win the budget wars. Similarly, too much emphasis on tactical work might give the impression that the agency was a support service to the armed forces, rather than something like a fourth service in its own right. Horrors.

But this didn't stop some important projects from being designed to fill the gap. GCHQ had been called in to investigate whether the Territorial SAS's stay-behind reconnaissance teams, intended to target the Red Army's rear areas for air (and specifically nuclear) attack, were likely to avoid getting caught for long enough to be useful. They demonstrated that, even using burst transmissions, the Soviet electronic-warfare units would very likely triangulate on them within 24 hours. This obviously wasn't good enough, and one of the results was the Nimrod R1, the RAF's airborne electronic intelligence system. System is the right word; as well as the planes, the project included a special RAF intelligence centre at Wyton, communications links forward to the army, and the capability to have intelligence analysts, Army liaison officers, or linguists actually fly on the plane with the radio operators. (As well as the R-1s, the Nimrod MR2s have done a lot of this in Afghanistan, and paid the price.)

That was then; the RAF is now leasing three RC135 aircraft from the Americans, actually older than the R1 airframes and designed for the model then considered inappropriate.

This may be a serious problem; one of the big questions facing GCHQ is the age of fibre-optics and open-source cryptography. With less and less telecoms traffic going by satellite or microwave, and less of that going in the clear, what to do? Further, the questions aren't the same ones as they were in the cold war.

An example of why this is relevant is this piece by Spencer Ackerman on the US Air Force's MC-12 aircraft and its role detecting improvised explosive devices in Afghanistan. In fact, as he points out elsewhere, the MC-12 (roughly, a Beech King Air stuffed with sensors, extra fuel, and spooks) does a lot of other things too, although they're mostly classified. It's an example of a current trend - rather than UAVs, there's increasing interest in cheap light aircraft carrying the latest sensor packages. This has the advantage that they can take up intelligence agents and work more closely with the troops, as well as being cheap.

There's much more detail here, which makes the interesting point that the role of Task Force ODIN, set up to kill insurgent bombmakers in Iraq, is now a broader one in support of the counter-insurgency strategy. This changes their relevance from being purely tactical and military to being political and strategic. They haven't been inactive on this - from Aldrich's site, here's a fascinating data sheet on their backpack SIGINT kit, the ideal gift for the geek who has everything and a death wish and very similar to some Rohde & Schwarz mobile network testing gear.

Speaking of mobile networks, Aldrich also confirms that a capability to listen to cellular networks exists, mounted on the British Army's three Islander aircraft - it's not clear from his discussion whether this means the access side or microwave-backhaul, or whether this relies on the old A5/0 and A5/1 cyphers still being in use.

GCHQ Review, Part 4 - History and the overseas outposts

A major claim of the recent group of "intelligence historians" is that the study of the secret world is the "missing element" in contemporary history - that, just as the history of the second world war needed revising after the British government finally let on about ULTRA, history (especially of the Cold War) is missing the perspective provided by intelligence. Richard Aldrich's GCHQ is certainly part of this project, just as his The Hidden Hand was one of the better works in it for covert action, propaganda, and human intelligence.

But do we know that much more about the main line of history from it? There are, of course, a couple of serious documentary and methodological problems with this. Even where we do have good sources on the history of secret intelligence, it's typical for the actual intelligence product to remain secret. We have a reasonable idea of what all those antenna farms were after - we don't have much, post-ULTRA, of what the prime minister actually got delivered to his desk in the blue-jacketed files. Writing my own Master's thesis, I remember that the literature was rather better on the contribution of Soviet intelligence to the 1973 crisis than the US kind, but even that was because various individuals had been forthcoming. The Soviets tried to persuade Sadat to end the war by producing MiG-25R imagery showing the Israeli counter-offensive building up; he wasn't apparently convinced. We don't know, however, if the Americans did anything similar with the Israelis, although we do know that the Israelis weren't sharing their own information with the Americans. (And we know now that Ted Heath turned off their SR-71 operation out of Lakenheath, so how much did they know?)

There's another problem, though, which is understanding what contribution intelligence actually makes to decisions. Cynically, you might say that giving politicians more data is pointless; they'll either ignore it or pick the bits that suit their preconceptions. John Keegan argued that across history, intelligence was more often misused, ignored, or just irrelevant to the balance of forces on the ground than not. Obviously, having regular deliveries of ULTRA decrypts didn't prevent Dunkirk, although it may have helped bring off the evacuation. Even more obviously, whatever intelligence sources Tony Blair was using in 2002 didn't bring him very much enlightenment. That raises another question - was the intelligence valid even before the upsexers got at it? Why did all the European countries with their own overhead imagery choose to stay out?

These problems are less serious when the events in question were motivated by intelligence interests, rather than by the content of intelligence. Aldrich is good on this - the times when "the SIGINT tail started to wag the policy dog". Notably, this seems to have been a major motivation in the whole sorry story of Diego Garcia, intended as a replacement for the abandoned sites on Mauritius and Ceylon and for the NSA's intelligence-gathering ships after the attack on USS Liberty. Around this time, GCHQ also considered building an enormous, nuclear-powered ship intended to contain a complete overseas station of the size of HMS Anderson on Ceylon or Little Sai Wan in Hong Kong, plus a BBC World Service transmitter site - Harland & Wolff's was commissioned to carry out a design study.

The plan was to have it flagged as a merchantman, but it would have been an enormous and expensive sitting duck.As plans go, at least it didn't involve ethnic cleansing.

Later, when the third Wilson government decided to pull out of the remaining overseas bases in 1976, it was the GCHQ interest, backed up by the NSA, that led them to keep the presence on Cyprus - as well as huge British intelligence facilities, the Americans had transferred numerous organisations there from Turkey when the Turks asked them to leave, which had then moved into the British bases for security after the 1974 invasion.

GCHQ Review, Part 3 - FISH, a case study

So we've discussed GCHQ and broad politics and GCHQ and technology. Now, what about a case study? Following a link from Richard Aldrich's Warwick University homepage, here's a nice article on FISH, the project to break the German high-grade cypher network codenamed TUNNY. You may not be surprised to know that key links in the net were named OCTOPUS (Berlin to Army Group D in the Crimea and Caucasus) and SQUID (Berlin to Army Group South). Everyone always remembers the Enigma break, but FISH is historically important because it was the one for which Bletchley Park invented the COLOSSUS computers, and also because of the extremely sensitive nature of the traffic. The Lorenz cyphersystem was intended to provide secure automated teleprinter links between strategic-level headquarters - essentially, the German army group HQs, OKW and OKH, the U-boat command deployed to France, and key civilian proconsuls in occupied Europe. The article includes a sample decrypt - nothing less than AG South commander von Weichs' strategic appreciation for the battle of Kursk, as sent to OKH, in its entirety.

Some key points, though. It was actually surprisingly late in the day that the full power of FISH became available - it wasn't enough to build COLOSSUS, it was also necessary to get enough of them working to fully industrialise the exploit and break everything that was coming in. This was available in time for Normandy, but a major driver of the project must have been as a form of leverage on the Americans (and the Russians). The fate of the two Colossi that the reorganised postwar GCHQ saved from the parts dump is telling - one of them was used to demonstrate that a NSA project wouldn't work.

Also, COLOSSUS represented a turning point in the nature of British cryptanalysis. It wasn't just a question of automating an existing exploit; the computers were there to implement a qualitatively new attack on FISH, replacing an analytical method invented by Alan Turing and John Tiltman with a statistical method invented by William Tutte. Arguably, this lost something in terms of scientific elegance - "Turingismus" could work on an intercept of any length, Tutte's Statistical Method required masses of data to crunch and machines to crunch it on any practical timescale. But that wasn't the point. The original exploit relied on an common security breach to work - you began by looking for two messages of similar length that began with the same key-indicator group.

Typically, this happened if the message got corrupted by radio interference or the job was interrupted and the German operators were under pressure - the temptation was just to wind back the tape and restart, rather than set up the machine all over again. In mid-1943, though, the Germans patched the system so that the key indicator group was no longer required, being replaced by a codebook distributed by couriers. The statistical attack was now the only viable one, as it depended on the fundamental architecture of FISH. Only a new cypher machine would fix it.

The symbolic figure here is Tommy Flowers, the project chief engineer, a telecoms engineer borrowed from the Post Office research centre who later designed the first all-electronic telephone exchange. Max Newman, Alan Turing's old tutor and the head of the FISH project, had shown Flowers a copy of On Computable Numbers, which Flowers read but didn't understand - he was a hacker rather than a logician, after all. He was responsible for the shift from electromechanical technology to electronics at Bletchley, which set both Newman and Turing off towards their rival postwar stored-program computing projects.

Another key point from the book is the unity of cryptography and cryptanalysis, and the related tension between spreading good technology to allies and hoping to retain an advantage over them. Again, the fate of the machines is telling - not only did the FISH project run on, trying to break Soviet cypher networks set up using captured machines, but it seems that GCHQ encouraged some other countries to use the ex-German technology, in the knowledge that this would make their traffic very secure against everyone but the elect. Also, a major use of the surviving computers was to check British crypto material, specifically by evaluating the randomness of the keystreams involved, a task quite similar to the statistical attack on FISH.

Finally, FISH is exhibit A for the debate as to whether the whole thing has been worthwhile. What could have been achieved had the rest of the Colossi been released from the secret world, fanning out to the universities, like the scientists from Bletchley did themselves? Max Newman took racks of top-quality valves away from Bletchley when he moved to Manchester University, and used them in the very first stored-program, digital, Turing-complete computer; Alan Turing tried to do the same thing, but with a human asset, recruiting Tommy Flowers to work on the Pilot-ACE at NPL. (Flowers couldn't make it - he had to fix the creaking UK telephone network first.) Instead, the machines were broken up and the very existence of the whole project concealed.

On the other hand, though, would either Newman or Turing have considered trying to implement their theories in hardware without the experience, to say nothing of the budget? The fact that Turing's paper was incomprehensible to one of the most brilliant engineers of a brilliant generation doesn't inspire confidence, and of course one of the divides that had to be crossed between Cambridge and GPO Research in Dollis Hill was one of class.

GCHQ Review, Part 2 - GCHQ and the Tech Industry

OK, so some more on Aldrich's GCHQ. Obviously, technology is at the centre of this story. I've said that the signals intelligence world is special among spooks because it guarantees results - they may not be the right results, they may not be helpful, but you can usually depend on it producing something to whack on the PM's desk, that he or she can spring on cabinet ministers later. One of the things that makes it special is its industrial nature; unlike most forms of intelligence, it needs machines, great buildings, thousands of technical staff working shifts, and its performance is heavily dependent on engineering.

From a budget-politics point of view, there's a symbiosis here. Back in 1941, the permanent secretary of the Foreign Office got into the habit of bringing top officials from London to be dazzled by the brilliance on display at Bletchley and terrorised by its security officers. It worked. On the other hand, getting the resources necessary to build the crypto industry required the direct intervention of a group of top scientists around Turing and Gordon Welchman with Churchill. Of course, as someone regularly dosed with their product, he didn't find it hard to give them what they needed, which was money and lots of it. By mid-1942 and the introduction of the third rotor on the Enigma machine, it became very obvious indeed that signals intelligence was now an industrial enterprise. This led directly to the decision to let the US Navy build its own Ultra capability, and hence to the founding treaties of the special relationship.

As soon as the Holden agreement let the Americans get hold of the Ultra secret, however, Bletchley was frantically building up new technology that would maintain a bargaining edge. The huge effort to crack the German on-line cipher known as FISH, for example, which led to the COLOSSUS computers, has to be seen partly in this light. This combination of a sort of fatalism - the Americans would eventually triumph - and a hunt for an edge would colour GCHQ's role in the history of technology from then onwards. Despite its founding achievements in computing, and those of the post-war diaspora of scientists, they were always suspicious of British technology. Post-COLOSSUS, GCHQ joined the long, long queue for IBM 360s and then, oddly enough, veered off to get all its computers from Honeywell into the 1980s.

On the other hand, a number of key research projects were pressed ahead, notably a range of exotic over-the-horizon radars, agent equipment, the Nimrod R-1, and the never-completed Zircon satellite. This combination of cringe and competition was mirrored by the SIGINT tribe's attitude to technology in general; starting in the 1960s, they were both keen to spread good cryptography among NATO and other friends, but also to prevent the development of independent crypto. On the one hand, "free licensing" was meant to let second- and third-tier agencies and Western non-governmental systems get access to effective security; on the other hand, rather like the bundling of MS Internet Explorer, it was meant to secure a monopoly. This put the UK in a difficult position - it strongly intended to develop its own crypto, thanks, and export it, but the companies involved very much wanted to claim royalties on their patents.

This eventually ended up with the incredible effort to subvert Crypto AG of Switzerland's high-end cipher machines (CAG, by the way, owned the intellectual property of Hagelin, the makers of what became the Enigma...), under which the NSA and GCHQ persuaded them to fix certain cryptographic problems, but to leave other security bugs unfixed in order that they could continue to spy on their users. The exploit in question referred to TEMPEST, the now-well known problem where some electronic devices leak information in the clear as radio interference, which strongly suggests that the point was to protect some of the many embassy spying operations.

This couldn't, and didn't, last - by the 1980s, as with general policy, the monopoly of security technology was crumbling as the Europeans (mostly) got better at it. There were efforts to change this - GCHQ was given a special responsibility to keep an eye on Nokia, while other allied agencies got tasked with Ericsson, Siemens, Olivetti, etc (but notably not Alcatel). Another important factor, eventually decisive, was that it was moving from hardware to software. In the light of this, the 1990s crypto wars seem a lot more radical than a bunch of geeks playing at spies; something very important did change back there. On a critical note, I did think Aldrich's book could have done with a good technical reader on software, Internetworking, and related issues - the focus is a bit off here, and he seems to depend more heavily on the civil servants.

Did GCHQ hold back or promote technical progress in the UK? There are various views on this. One is that it's part of a huge cluster of PhDs in the Severn valley that must be having some sort of spin-off benefit to the country - even if it's only that when Thatcher offended them to the extent everyone in the computer division of HEO rank or above quit, a lot of other tech companies filled their boots. Another is that it's a sort of shadow of the British Google that didn't happen, because the potential founders were wasting their time sucking up to the intelligence-administrative complex.

Of course, it's true that they invented public-key cryptography in 1971 and didn't tell anyone for 35 years. But this was largely because nobody could think of a use for it back then. (Apparently, they thought of using it to authenticate nuclear launch orders, until it was pointed out that they didn't have to be sent in real time any more because the nukes were submarine-launched.) On the other hand, much of its purpose in life is to provide a source of clue for the wider government (a sort of infosec Shi'ism, a marja e-taqlid for system administrators and government ministers), and who can say British governments have suffered from too much competence?

GCHQ Review: Part 1, The World's Most Classified Blog and Other Stories

So, Richard Aldrich's book on GCHQ. This looks like it's going to be another in our occasional series of multi-part book reviews that nobody reads, as the book is nothing if not comprehensive. (It's a mere Laundry-esque 666 pages in paperback.) Apart from being packed with good things, like paper and words, as Spike Milligan said about his autobiography, I think it's undeniable that this is the best factual account of British signals intelligence you're likely to get. It practically bursts with detail and is clearly the fruit of an enormous effort of primary research, and a fair bit of the secondary kind too. If you want to know about the continuation of the First World War crypto effort into the inter-war era, the construction of the Hong Kong over-the-horizon radar site on top of a sheer cliff thousands of feet high and the number of Land Rovers the RAF Regiment lost over the edge, or exactly how many index cards Special Branch found in Geoffrey Prime's private database of young girls, it's here. This is in itself quite an achievement, given how much of this stuff remains classified.

Of course, what everyone wants to know about is the intelligence special relationship with the US and the other Commonwealth nations. You will not be disappointed. Aldrich argues that we're unlikely ever to find a smoking document, even after the release of what was described as the UKUSA agreement earlier this year - the terms of the alliance were repeatedly renegotiated, and its content is spread over many different documents. In fact, it might be more interesting to think in terms of the technical documents. He makes the excellent point that the alliance consists, in practice, of a set of shared operating procedures and technical standards, rather like the Internet, with the distinction that here everything is secret. Rather than gaining access to the IETF by making your work public, you gain access to the tribe of SIGINT by submitting to ever greater secrecy, in a sort of masonic career of increasingly complex rites. Crucially, wherever the documentation goes, the internationally agreed security requirements go with it. This, of course, has an impact on parallel technological decisions, but I'll come to those later.

This tribal nature - and in many ways it is tribal, with different agencies' membership in the relationship stemming from their alliance with the founding couple of Bletchley Park and US Naval intelligence - has important and counterintuitive effects on the politics of SIGINT. For example, the tribal leaders have frequently been keen to help their kin succeed in developing new technologies, extracting more funds from national budgets, and securing their secrets from their common enemies. On the other hand, they have also been very keen to prevent them from developing relationships that bypass the central alliance, and to restrict the degree to which they can secure their own traffic against the "level one agencies", GCHQ and NSA. All tribes, however, are in part mythical, and the status of the leader derives in part from the consent of the led.

In the early 1970s, for example, Henry Kissinger ordered the NSA and the National Reconnaissance Office to cut off intelligence sharing with Edward Heath's government (Heath's GCHQ director was, among other things, in the process of negotiating a special link between the Joint Intelligence Committee and the French equivalent). The British were horrified, but it's telling that the NSA itself was very suspicious of the move and took steps to undermine it - it seems that information kept reaching Britain via sharing with Canada and Australia. When the Yom Kippur war broke out, Heath retaliated by refusing to let SR71 reconnaissance flights land in the UK or at Akrotiri, and imposing conditions on U-2 operations from the UK, specifically that the imagery from them could not be shared with Israel.

In the 1980s, the Reagan government imposed a similar "cut-off" on New Zealand to protest their refusal to let US warships call without saying if they were carrying nuclear weapons. The New Zealanders were unexpectedly unimpressed, which was at least in part explained by the fact that the other alliance partners continued to pass information to them, and also by the fact that the New Zealand GCSB was a major analysis centre for traffic from Asia. Notably, GCHQ was collecting French traffic on their behalf as part of the Rainbow Warrior inquiry.

Over the years, the power-relationships within the alliance shifted with the varying scarcity of different resources. To begin with, in the heroic days of Bletchley Park, the UK had a strategic advantage based in its extremely scarce knowledge of cryptanalysis and computing. As the importance of computing and bulk data processing in general grew, this shifted towards the US; they had more money, and their own technology was improving fast. The result was that the Commonwealth partners essentially traded collection for analysis - we had territory, relationships, and collection platforms that the Americans didn't. That included some hideously dangerous overflights, submarine missions, and covert actions around the edge of the Soviet sphere of influence. Again, if you want to know what it was like sailing an old submarine into Polyarnyy harbour in 1959 without asking, it's here.

This oversimplifies; in fact, however much money the Americans threw at the problem, they didn't break the Soviet high-level ciphers between Black Friday in 1948, when the USSR carried out a forklift upgrade of their whole crypto network to end the VENONA codebreak, and the late 1970s. Information had to come, instead, from new forms of collection, targeting networks that weren't encrypted because they were thought to be secure, and by studying the electronic signatures of new weapons. As a result, the inter-allied playing field had a structural skew towards the British, who specialised in forward collection and in ELINT, building up an enormous library of Soviet radars and emplacing microwave listening stations in unlikely places. However, it's unlikely that this was realised at the time - it was all too obvious that Fort Meade was filling up with more and more computers, and it's not clear how honest they were about their successes or failures. There was a sort of technical cultural cringe on the British side.

The other new field was of course space. Starting in the late 1960s, the US began to collect much more of its signals intelligence from satellites, invulnerable to the political turmoil down below. However, this brought about another twist in the political relationship. The Americans had ELINT and COMINT satellites, the allies didn't. But when the RHYOLITE satellites, originally intended to spy on missile telemetry, started to pull in more and more data from the new microwave telecomms backbones, the NSA was forced to rely on its allies to deal with the mountains of data. That meant, among other things, a momentous step - intelligence sharing now included readout, letting the allied agencies point their dishes at the satellites and receive the stuff directly. (Incidentally, this is the purpose of Menwith Hill - it slurps intercept material from satellites and passes it to Cheltenham.)

At the beginning of the 1980s, then, the alliance was undergoing the sort of integration process that the founders of the European Union hoped to see. Rather than painful negotiations in high politics, technical interworking would result in a natural binding together. The system was evolving from the original hierarchical structure into a flatter network, with much greater interdependence. The Americans seem to have been aware that control was slipping away, and made efforts to assert traditional rights, for example by trying to impose the lie detector as part of the common security rules, which even Margaret Thatcher considered illiberal and unscientific. Some tribal practices didn't translate. The New Zealand cut-off was part of this, as was its failure - among other things, what was to happen about the New Zealanders seconded to Canada and the UK, and the Canadians and Brits in New Zealand? What would the US customers for Korean traffic processed at GCSB say when it ceased to arrive?

Interestingly, the US seems to have found continental Europe more interesting as a result. They made efforts to cooperate more closely with West Germany, while the Germans for their part were organising a new European alliance, and the UK was developing close links with the Mitterrand government's intelligence chief (while also helping the New Zealanders get information on his agents).

Yet another shift, possibly even more important than the end of the Cold War for the tribe, was now approaching - the end of the microwave network era and the dawn of widely available strong cryptography. Arguably, what is now scarce is code-breaking of any kind, again, and intelligence analysis; computer power has never been cheaper, while mass collection is much less practical outside one's own borders. In fact, pharaonic proposals like the Intercept Modernisation Programme may be better understood as a sort of atavistic harking back to the microwave era or even to the high Cold War's tunnels under Berlin and Vienna.

However, it's certain that they ain't going away. One thing that SIGINT has which other forms of intelligence don't is that it works, it produces physical output, yer actual primary-source documents - every day, as well as the formal, all-source intelligence reports on particular topics, the prime minister is also sent a wedge of selected quotes from the raw traffic. It's the world's most classified blog! Thatcher's civil servants referred to it as Comic Cuts (in the 1950s and 1960s, similar files were known as Blue Jackets or BJs - another way to make the president feel special, I suppose...), but she lapped it up, like they all do.

Daniel Davies once remarked that secret information is a drug - it alters your perception of reality and makes you feel superior to other people - and that it isn't usually considered wise to make important decisions on drugs. Here's the problem; whether or not the raw matter is actually useful, whether it's typical or misleading, whether GCHQ is breaking a lot of the target's traffic or none of the circuits that matter at all, it's incontrovertibly present. They will produce something rather than nothing.

In our next thrilling instalments: GCHQ and technology, overseas outposts, internal surveillance, and the future...

self-binding admin notice

Coming up on TYR this weekend - we review Richard Aldrich's GCHQ...

writing about Afghanistan, rather than about Brunssum or Qatar

I've finally got around to reading Ahmed Rashid's Taliban and Descent into Chaos. They are as good as everyone says. Specifically, there are perhaps three things that set Rashid apart as a writer on Central Asia. (His contacts book is outstanding, but then, he's not the only one.)

First of all, he writes about Central Asia, rather than about American politics as expressed through the foreign-policy establishment. He writes about Central Asia in the sense that he places the complex regional politics, the competition for power among Pakistan, Iran, and Russia, at the centre of the story. In fact, you could make a case that the Taliban as a phenomenon is almost irrelevant; if it didn't exist, and the political situation was otherwise unchanged, something else would be playing the role of Durrani Pashtun caucus, drugs logistical system, sink for Saudi malcontents, and Pakistani proxy.

He also writes about Central Asia in the sense that he emphasises the intelligent agency of the regional powers, the Afghans of all allegiances, the Pakistani political parties, and the intelligence agencies. This implies taking a very calm view of the actual extent to which the Americans ever controlled anything in the area - in fact, one of his key points is that the current chaos is largely due to the absence of a US policy in the 1990s, and in many ways, its continued absence up to 2005 and even now.

Rashid also provides an interesting view of Taliban sociology - his version of them is essentially another of the child-soldier and refugee camp movements of the 90s, strongly mutually similar across an arc of suffering from Afghanistan to Liberia. He suggests that their ideology is more of a substitute for Islam and for Afghan culture than anything else - a cut-down set of tropes for people brutally removed from the real things.

Of course, there are certain functions that any good tyranny needs to fulfil. You have to have outward signs, so that it is possible to enforce conformity and identify a hated target-group; it may actually be better if they are content-free, so as not to limit flexibility. You have to have exemplary violence, and again, it may help if it isn't actually directed towards victory. Eliminating people for no reason is the ultimate costly signal that anyone could be a target. No tyranny can function without denunciation - arguably, it's more important than all the other functions. And you need the possibility of competitive observance, in order to get individual initiative on your side. "Working towards the Führer" is the classic example. This may actually be a more useful view of the Talibs of the 90s - a sort of minimal dictatorship.

Finally, he provides an integrated systems view of the politics, economics, and societies involved. The creation of the Taliban, in his view, involved many overlaid political networks, those of the Pakistani trucking industry and its partners in organised crime, those of the Sindhi feudal landlords who were frequently investors in the trucking and smuggling business and also powers in the PPP, the Saudi-financed system through which international jihadis were recruited, fetched to the training camps, supplied, and sent out as cannon fodder to pursue Pakistani aspirations in central Asia, and the ISI.

From a purely Anglo-British point of view, it's worth noting that he is very hard on the Americans about the intelligence picture available to the NATO powers in 2005 when Rumsfeld finally dropped his opposition to ISAF deploying outside Kabul. He strongly supports the line that, having maintained practically no presence there and diverted their satellite and other reconnaissance resources to Iraq, the Americans let 16th Air Assault Brigade deploy into a zone of the unknown, which in the way of these things turned out to be full of the enemy. If true, this is the second occasion on which they've welshed on the agreement under which the UK doesn't operate its own imagery satellites. Rashid argues that there was a vital window of opportunity to get a broad-based political settlement in 2002-2003, which the Cheney administration* squandered in the interests of invading Iraq and pleasing the ISI/Saudi intelligence services.

In general, I can't escape the conclusion that Kashmir is still the issue.


(* - Rashid's view of the last US administration is very much a Cheney administration)

Authoritarianism Does Its Thing

This has done the rounds and been roundly done for all the right reasons.

There is almost nothing the Obama administration does regarding terrorism that makes me feel safer. Whether it is guaranteeing captured terrorists that they will not be waterboarded, reciting terrorists their rights, or the legally meandering and confusing rule that some terrorists will be tried in military tribunals and some in civilian courts, what is missing is a firm recognition that what comes first is not the message sent to America's critics but the message sent to Americans themselves. When, oh when, will this administration wake up?

From a purely literary/journalistic point of view, it's the "When, oh when" that gets me. Sometimes, style and content - aesthetics and morality - fuse into one.

More to the point, the astonishing thing here is Bush's lasting achievement - he created a political lobby for torture. It's not just that he let torture happen, or connived at it, or even specifically ordered it. It's that a significant chunk of the body-politic now demands torture - not just 'baggers, but editors of the Washington Post. There isn't a lobbying group with tax-deductible status under 501(3)c yet - unless you count the American Enterprise Institute - but perhaps it would be a more honest world if there was one.

Do I have to quote Vaclav Havel's crack about the man who puts a sign reading "Workers of the world, unite!" in the window all over again? OK. Havel said that obviously, he probably wasn't doing this out of conviction; but if the sign said "I am afraid and therefore obedient", its actual meaning, he might not be so happy to do it.

Perhaps. But I can't help thinking the example may be wrong. Richard Cohen is, after all, not just being willing to turn a blind eye. He's actually yelling for torture, and for specific methods of torture. And the marker of the Bush achievement is that the torture lobby has survived Bush. Here we are, more than a year on, after the US armed forces have been given specific orders against torture. And they're out there wanting it. It's weirdly reminiscent of the last Stasi man and the last suspect.

Also, it's nothing to do with expediency; when the FBI wanted to question Captain Underpants, they got his relatives to talk to him, and it worked. It is usually the case that the purpose of torture is torture; what service, I wonder, does the knowledge of torture provide to these people? After all, Cohen explicitly says that he wants torture because it impresses the public, not because it produces names.

I can't imagine what would have convinced me in 2000 that in 2010, responsible Americans would be lobbying for torture - even after they had succeeded in voting out the torture president. Back then, it used to be a commonplace notion that the power of the state was fundamentally uninteresting; I recall an especially silly newspaper article in which both Bill Clinton and Deng Xiaoping (Deng Xiaoping!) were bracketed together as meaningless figureheads.

Having a considerable lobby that needs a constant drip of draconian rhetoric to maintain their psychological stability is probably very bad for democracy, especially faced with a terrorist group that explicitly aims to destabilise the state through auto-immune warfare. These people have been trained to freak out at the faintest threat and howl for torture - in a sense, it's yet another backdoor into the political system, as well as an example of the unconscious conspiracy between the terrorist and the state.

a single net of conspiracy

Well, this is hardly surprising; the FBI was in the habit of pretending to be on a terrorism case every time they wanted telecoms traffic data. Their greed for call-detail records is truly impressive. Slurp! Unsurprisingly, the lust for CDRs and the telcos' eagerness to shovel them in rapidly got the better of their communications analysis unit's capacity to crunch them.

Meanwhile, Leah Farrell wonders about the problems of investigating "edge-of-network" connections. Obviously, these are going to be the interesting ones. Let's have a toy model; if you dump the CDRs for a group of suspects, 10 men in Bradford, and pour them into a visualisation tool, the bulk of the connections on the social network graph will be between the terrorists themselves, which is only of interest for what it tells you about the group dynamics. There will be somebody who gets a lot of calls from the others, and they will probably be important; but as I say, most of the connections will be between members of the group because that's what the word "group" means. If the likelihood of any given link in the network being internal to it isn't very high, then you're not dealing with anything that could be meaningfully described as a group.

By definition, though, if you're trying to find other terrorists, they will be at the edge of this network; if they weren't, they'd either be in it already, or else they would be multiple hops away, not yet visible. So, any hope of using this data to map the concealed network further must begin at the edge of the sub-network we know about. And the principle that the ability to improve a design occurs primarily at the interfaces - this is also the prime location for screwing it up also points this way.

But there's a really huge problem here. The modelling assumptions are that a group is defined by being significantly more likely to communicate among itself than with any other subset of the phone book, that the group is small relative to the world around it, and that it is boring; everyone has roughly similar phoning behaviour, and therefore who they call is the question that matters. I think these are reasonable.

The problem is that it's exactly at the edge of the network that the numbers of possible connections start to curve upwards, and that the density of suspects in the population falls. Some more assumptions; an average node talks to x others, with calls being distributed among them on a well-behaved curve. Therefore, the set of possibilities is multiplied by x for each link you follow outwards; even if you pick the top 10% of the calling distribution, you're going to fall off the edge as the false positives pile up. After three hops and x=8, we're looking at 512 contacts from the top 10% of the calling distribution alone.

In fact, it's probably foolish to assume that suspects would be in the top 10% of the distribution; most people have mothers, jobs, and the like, and you also have to imagine that the other side would deliberately try to minimise their phoning or, more subtly, to flatten the distribution by splitting their communications over a lot of different phone numbers. Actually, one flag of suspicion might be people who were closely associated by other evidence who never called each other, but the false positive rate for that would be so high that it's only realistically going to be hindsight.

Conclusions? The whole project of big-scale database-driven social network analysis is based on the wrong assumptions, which are drawn either from military signals intelligence or from classical policing. Military traffic analysis works because it assumes that the available signals are a subset of a much bigger total, and that this total is large compared to the world. This makes sense because that's what the battlefield of electronic warfare is meant to look like - cleared of civilian activity, dominated by one side or the other's military traffic. Working from the subset of enemy traffic that gets captured, it's possible to infer quite a lot about the system it belongs to.

Police investigation works because it limits the search space and proceeds along multiple lines of enquiry; rather than pulling CDRs and assuming the three commonest numbers must be suspects, it looks for suspects based on the witness and forensic evidence of the case, and then uses other sources of data to corroborate or refute suspicion.

To summarise, traffic analysis works on the assumption that there is an army out there. We can only see part of it, but we can make inferences about the rest because we know there is an army. Police investigation works on the observation that there has been a crime, and the assumption that probably, only a small number of people are possible suspects.

So, I'm a bit underwhelmed by projects like this. One thing that social network datamining does, undoubtedly, achieve is to create handsome data visualisations. But this is dangerous; it's an opportunity to mistake beauty for truth. (And they will look great on a PowerPoint slide!)

Another, more insidious, more sinister one is to reinforce the assumptions we went into the exercise with. Traffic-analysis methodology will produce patterns; our brains love patterns. But the surge of false positives means that once you get past the first couple of hops, essentially everything you see will be a false positive result. If you've already primed your mind with the idea that there is a sinister network of subversives everywhere, techniques like this will convince you even further.

Unconsciously, this may even be the purpose of the exercise - the latent content of Evan Kohlmann. At the levels of numbers found in telco billing systems, everyone will eventually be a suspect if you just traverse enough links.

Which reminded me of Evelyn Waugh, specifically the Sword of Honour trilogy. Here's his comic counterintelligence officer, Colonel Grace-Groundling-Marchpole:

Colonel Marchpole's department was so secret that it communicated only with the War Cabinet and the Chiefs of Staff. Colonel Marchpole kept his information until it was asked for. To date that had not occurred and he rejoiced under neglect. Premature examination of his files might ruin his private, undefined Plan. Somewhere, in the ultimate curlicues of his mind, there was a Plan.

Given time, given enough confidential material, he would succeed in knitting the entire quarrelsome world into a single net of conspiracy in which there were no antagonists, only millions of men working, unknown to one another, for the same end; and there would be no more war.

Want a positive idea? One reading of this and this would be that the failure of intelligence isn't a failure to collect or analyse information about the world, or rather it is, but it is caused by a failure to collect and analyse information about ourselves.

the problem with NPfIT is the "NP" bit

Something interesting about the NHS NPfIT project. During my recently completed two-week conference binge, I spoke to people from a British telecommunications company who were fresh, if that's the word, from tangling with the NHS IT Zombie, and had apparently escaped before it ate their brains with a spoon. I also heard people from a French telecommunications company who had been working in the same field speak.

They agree on this; national healthcare institutions are too complicated for any one organisation to build the kind of comprehensive, end-to-end workflow system that NPfIT envisaged. This is partly because of the incredible complexity of their business processes; an episode of care can span anything from a GP appointment that ends by the patient being told there is nothing the matter with them, or an immunisation being administered in a single visit by a nurse, to 20 years of treatment for a cancer and subsequent surveillance. There are a hell of a lot of other organisations that interact with the NHS, and who aren't part of the project.

In fact, if they were, the scale and scope of NPfIT would increase to the point at which it encompassed most of the public sector; it would have to integrate with the social security system, and because of all those benefits that are delivered as tax credits, with the Revenue as well, and (because the NHS provides the armed forces' medical care) with the MOD's personnel system and even with tactical communications systems in the RAF, because Selly Oak receives casualties direct from the war. Of course, it no doubt already needs to talk to the Treasury's systems. You might as well just ask the ghost of Stafford Beer to build us a Cybersyn for the whole economy.

But that wasn't the worst of it. The real problem, according to my source, was that the designers of NPfIT believed that there was an organisation called the NHS. In fact, this was a bit like modelling a blue whale as a homeogenous sphere to make the maths easier. The killer wasn't that medicine is complicated; it was that the NHS isn't a monolithic organisation. It is, of course, an institution - a set of social, political, and economic expectations and relationships, a recognisable culture, a way of understanding the world. But it's far from being a single organisation.

Instead, it's an ecosystem, made up of many organisations that sometimes play similar roles (it's a hospital; it's a GP practice) but differ dramatically in their internal structure, rather as a dolphin and a Humboldt squid are both social, pelagic, fast-swimming predators in the subtropical ocean. However, only one of them is even a tetrapod, and only a real idiot would assume they were both sufficiently described by the concept of "shark". And the interactions between the creatures in this ecosystem are deeply complicated.

In that sense, it's quite a lot like the Internet. That, too, consists of a grab-bag of diverse organisations that cooperate with varying success on the basis of a few rules and a rough common culture, which is often honoured more in the breach than the observance. That also has a lot of odd emergent features that arise from its complexity, and would almost certainly be impossible to design as a single organisation. Indeed, an old staple of Internet-related mailing lists is the question of what the word "Internet" actually means.

Cue facile libertarian woofing. Yadda yadda Hayek privatise the BBC. Spare me. Neither does this mean the NHS is disorganised; it may well mean that it's better for its geographically and functionally diverse components to work differently. It would be surprising if they all shared a single optimal strategy. Of course, there is a perfectly good paradigm for building effective information systems in circumstances like these (and another one). What's really deeply depressing about this is that after all the blundering about and the money, there's still not the key element that makes a Web-like approach possible - standard data formats and interconnection procedures.

How much would it have cost to sponsor an effort to fix that, coming up with an XML standard or a Semantic Web ontology and some NHS standards, setting down for example where the canonical data would live and who could get at it in what circumstances?

police and thieves

Bruce Schneier and Jason Sigger, usually sensible sources, both mock a study by some thinktank or other which raises the supposed possibility of hackers "using the Internet to start a nuclear war".

As they both point out, the possibility of anyone getting access to the actual command and control firing chain with metasploit is so remote as to be ridiculous, and we'd do much better to worry about tidying up old radioisotopes in Russia, and perhaps not having quite so many nuclear bombs.

My only objection is that we have, in fact, lived through a serious attempt to do just that, immediately after Lashkar e-Toiba terrorists attacked the centre of Bombay in December, 2008. As you might expect, they didn't try to get control of nuclear weapons from the command line.

Instead, they attempted to use the Internet to influence the political leadership - they placed a call to the Pakistani president's office, spoofing the calling line identification message in order to give credibility to their effort to pose as the Indian foreign minister. My technical analysis is here; the Indian government's investigation later showed that the attackers set up a VoIP network with nodes in the US and Austria for their own use.

Presumably the idea was to provoke the Pakistanis into doing something that would destabilise the situation, causing the Indians to respond and thus triggering Pakistani mobilisation for real. The Guns of August, 2.0, with Princip using a Linksys SIP handset.

Clearly, there is still a need for the existing nuclear states to help the new ones establishing solid command and control procedures, including the communications elements that make them work; one of the problems of international crises is that the system to be secured suddenly gets a whole lot bigger, as other systems - in this case the diplomatic/protocol bureaucracy - become closely connected to it.

It's not the early 80s hackers of War Games we need to worry about - instead it's essentially trolls, provocateurs, empowered by the technology available to today's spammer.

It strikes me that the possibility of ambiguous identity is a hard one to grasp; for a very long time, it was safe to say that such a message was unlikely to be a fake, and if it was, it was probably faked by a proxy for the real enemy. Consider the case of 4chan vs. AT&T.

AT&T null-routed the server which carries the bulk of 4chan's content; everyone freaked; AT&T claimed that a denial of service attack was coming from that IP range. But it was hardly likely that the 4chan crowd, of all people on the Internet, would have been daft enough to launch a denial of service attack from their own machine - DOSs have essentially always been distributed over many, many hacked computers (DDOS, for Distributed Denial of Service) since the first botnets emerged in the early 00s, this being harder to counter, offering much more stolen computing power, and being much more difficult to trace to its source.

A detail in the Ars Technica story explains it all. One of the sources cited mentions "persistent ACK scans" - when a computer wants to start a TCP connection, as used for the Web, to another, it sends a message called a SYN to the receiving party, which if it gets the message and wants to reply, sends a message called an ACK to the address provided in the SYN. If received, the sender replies with a SYN-ACK and then starts transferring data.

4chan was experiencing a DDOS attack itself at the time. Putting these bits together, it's clear that the attackers were altering the source header in the packets they threw at 4chan to point to a machine somewhere in AT&T's network, so that every one received generated a further packet thrown at the AT&T machine. This is a classic; it gets you two attacks for the price of one, it conceals your own position, and it brings the possibility that AT&T might go ape and do the job for you. If the first target is especially big, you could also use it to magnify the volume of traffic, in a so-called reflector attack.

It's surprising and depressing that they weren't aware of that; no more surprising and depressing, however, than the way so many people have been willing to believe patently false information just because it's "secret".

know your spies

A field guide to sinister Middle Eastern spy chiefs.