Now that's what I call lobbying

In the recent case of Liam Fox and Adam Werritty, there was an issue that the news media spent an enormous amount of time and effort dancing around with innuendo, newspaper code, and carefully lawyered prose. It is a fact that the word "lawyered" is to the word "lawyer" as the word "doctored" is to the word "doctor". Without understanding this hidden and sordid side of the issue, you would have been seriously misinformed. The matter was very sensitive, and there was an excellent chance of getting sued and probably also demonised as being deranged by shameful prejudices.

I refer, of course, to whether or not the Defence Secretary's private office was having unprotected sex with other defence secretaries' private offices.

It took a while to surface this at all - the Guardian let a wee squeak out on Thursday, and eventually it was the Sindy that took the plunge and surfaced it in the same way you surface a submarine, with an enormous roar of compressed air thundering into the ballast tanks under pressure while the nuclear reactor cranks up to full power. It's a must read.

The fact that Werritty's freebies included trips to the Herzliya Security Conference paid for by pro-Israeli lobbying groups should have been a screaming giveaway, but then, that's what a good cover story is for. I presume that was what the Sindy eventually followed up.

I mentioned this element of the story to Daniel Davies earlier in the week. I can offer no special insight except for the enduring value of pattern recognition. This has, after all, happened before in recent memory, with really bad consequences.

Consider Mr. Michael Ledeen and the affair of the weapons of mass destruction. Mr. Ledeen, a professional neoconservative, claimed to have intelligence about Iraqi efforts to acquire uranium and various other things, which came from his contacts in Iran, some of whom were recommended to him by his contacts in Israel, one of whom, Larry Franklin, was convicted of spying for Israel in the US State Department. Ledeen believed these contacts to be renegade members of the Iranian secret service. (He had never visited Iran, and I think to this day never has, and he doesn't to the best of my knowledge speak Persian, so how he would have known is beyond me.) The CIA, for its part, believed that this was partly true. They just disagreed with the "renegade" bit. But Donald Rumsfeld had deliberately decided to ignore the CIA, so Ledeen's intelligence was accepted. However, that wasn't the end of the story. At some point, the Department of Defense became suspicious and called in its own Counter-Intelligence Field Activity to investigate.

At this point, a thick curtain of secrecy was drawn down on the story, even if we did eventually get the Phase IIA report. Whatever CIFA found out, Ledeen was able to introduce the famous forged documents on uranium from Niger, which seem to have come from the Italian secret service, as being Iranian information with Israeli approval, and this was used in the even more famous dossier.

I wouldn't be at all surprised if old blogging chum from way back in the day, 2004, Laura Rozen hasn't also had this thought, as she was instrumental in digging into the whole Ledeen affair and she's too smart to miss it. Also, hilariously, she and Spencer Ackerman had the honour of being targeted by Ledeen's mates in Silvio Berlusconi's intelligence service with a scurrilous smear-campaign. I should probably hat-tip the lady's Twitter feed.

Note the elements of the story. Ledeen is a semi-official adviser with special, privileged access to policymakers. He is outside the formal requirements of government service, but has access inside it. He is seen to have special access to an important ally, and therefore to be trustworthy. A third party observed this, and took advantage of it to introduce information (or rather, disinformation) into the policymaking system. Does anybody see a pattern here? Similarly, Werritty was offered privileged access from outside the government firewall because he was ideologically congenial. It seems that this was considered acceptable because the influence exerted came from a country considered friendly. But then, there were the rogue Iranian intelligence agents, or were they just ordinary Iranian intelligence agents?

In May 2009, Mr Werritty arranged a meeting in Portcullis House between Mr Fox and an Iranian lobbyist with close links to President Ahmadinejad's regime. In February this year, Mr Werritty arranged a dinner with Mr Fox, Britain's ambassador to Israel, Matthew Gould, and senior political figures – understood to include Israeli intelligence agents – during an Israeli security conference in Herzliya, during which sanctions against Iran were discussed. Despite Mr Werritty having no official MoD capacity, an Israeli source said there was "no question" that Mr Werritty was regarded as anyone other than Mr Fox's chief of staff who was able to fix meetings at the highest levels, and was seen as an "expert on Iran".

Well, at least Werritty actually went to Iran. Unfortunately this is the worst of the story, as it seems he was going round encouraging Iranian dissidents, or people he thought were Iranian dissidents, and promising them British support. This is really incredibly, shamefully irresponsible - he could have got people killed, and it cannot be ruled out that he did, although it's also quite possible that the whole affair was just a massive exercise in bullshitting and wanktankery.

Probably he really believes that he was in contact with the opposition. I'm fairly sure Ledeen doesn't think he's an Iranian agent either. This is where this classic Onion article comes into play. As I said at the time, why *do* all these Iranian agents keep sucking Michael Ledeen's cock?

It is all reminiscent of Bruce Schneier's thoughts on what happens if you create a backdoor into some computer system, so people like us can get in and out without anyone noticing. The problem is that once you do that, it immediately becomes the biggest security threat to the system as anyone else can use it too. Once this new interface to the MoD was created, with Werritty accepting connections from the wider Internet and forwarding them to Fox, of course it attracted dubious actors. Hence the parade of various people trying to sell aircraft spares and dodgy encryption software to the military or to get someone's knighthood expedited.

For my next trick, what parallels do you see between Werritty's role with Liam Fox and those of Andy Coulson and Neil Wallis with No.10 Downing Street and the Metropolitan Police (and of course the Conservative Central Office) respectively? Remember that both of them were at various times funded by third parties. Further, is it not interesting that the same key Conservatives who defended Coulson to the bitter end - George Osborne and Michael Gove - also tried to save Liam Fox? (Jonathan Freedland seems to have sensed something here - check out the reference to "Cheneyite Tories".) And is it not even more interesting that George Osborne actually recommended Andy Coulson for the job? And is it not completely fucking outrageous that William Hague, Atlantic Bridge board member and Foreign Secretary (I think this is the right order of precedence), dares to claim that proper Cabinet government is back in the midst of this berserk threat-chaos?

Enduring stare: Donal Blaney edition

Here's something interesting, from a post at Owen Jones's. Has Donal Blaney's "Young Britons' Foundation", the Tory wanktank that was too wanky for the Tories, been reactivated?

I will not distance myself from the right wing spin machine which attempts to bring down NUS’ credibility, but tackle it head on. Both of those occasions have taken NUS’ communications outrageously out of context and have been blatant attempts to weaken NUS. I also think these kinds of attacks will appear in the same kinds of newspapers consistently over the coming months in the run up to a report which is being published soon by the right wing “Young Britons Foundation” which will call NUS and Students’ Unions ‘dangerous organisations’ and attempt (again) to discredit and shut us down.

Over at the YBF website, it looks increasingly like they're being absorbed into the whole ex-Decent industry. Speakers from the Henry Jackson Society, projects on "Students' Rights" and "radicalisation". I suspect part of the reason for this is that the Department for Communities & Local Government budget is relatively big compared to your average wanktank and therefore wanktanks are irresistibly drawn towards it, by a sort of financial gravity. I presume the planned "report" is going to be an effort to accuse anti-cuts protestors of secretly being jihadis, perhaps with a side effort to get on the DCLG counter-radicalisation gravy train.

Also, here's the rundown for their next event:

Confirmed speakers:

Andrew Stephenson MP, Vice-Chairman of the Conservative Party

Steve Baker MP,Wycombe

Jonathan Isaby, ConservativeHome

Robert Halfon MP, Harlow

Shane Greer, Executive Editor, Total Politics

Eamonn Butler, Adam Smith Institute

Iain Dale, Broadcaster & Publisher

Dr Alan Mendoza, Executive Director, Henry Jackson Society

Raheem Kassam, Henry Jackson Society

Douglas Murray, Center for Social Cohesion

Simon Richards, The Freedom Association

Michael Fallon MP, Sevenoaks

Matthew Sinclair, Director TaxPayers‘ Alliance

Sajid Javid MP, Bromsgrove

Daniel Hamilton, Big Brother Watch

Lord Flight

Nigel Evans MP, Deputy Speaker

Rt Hon Peter Lilley MP, Hitchin & Harpenden

James Delingpole, Polemicist

Chris Kelly MP, Dudley South

Tim Evans, Director, Libertarian Alliance

Andrew Rosindell MP, Romford

Douglas Carswell MP, Clacton

Rt Hon Theresa Villiers MP, Minister for Transport

Mark Wallace, Senior Fellow TaxPayers’ Alliance

Conor Burns MP, Bournemouth West

Now there's a party even I wouldn't try to get into. What a bunch. For an organisation that was publicly disavowed as barking extremists during the last election campaign by the current prime minister, it's not a bad turn-out. There's a fair amount of filler, but even so. It also looks like I was right about "Big Brother Watch", the anti-ID card campaign that suddenly appeared five minutes before the repeal bill got Royal assent to claim the credit for six years of NO2ID - it's yet another Tory wanktank.

I was also amused by this:

Raheem has served on the National Executive for Conservative Future and began his political life running the blog ‘KeepRightOnline’, which swiftly became one of the best read political blogs in the country (now closed).

Yeah. One of the best read blogs in the country. Now closed. (Alexa can't find any scoreboard data on it at all.) It's apparently become a twitter feed.

Hey fat chicks; colouring and straightening your hair doesn't make you attractive. It makes you a fat chick with straight, coloured hair.

That's "quintessentially British commentary", apparently. It actually sounds more like a poor effort at pretending to be one of the crappier US trolls - Jeff Goldstein or Adam Yoshida - but then, crappy fake American trolling is Blaney's MO down to the ground. I wouldn't be at all surprised if they try to pull some sort of cod James O'Keefe trick in the run up to the 26th March demo.

Dumb Blond

The Philip Blond test: if you have a new policy idea, ask if Philip Blond would support it, and if so, bin it.

Exactly what is Communication Strategy & Management Ltd?

So I scraped the government meetings data and rescraped it as one-edge-per-row. And then, obviously enough, I tidied it up in a spreadsheet and threw it at ManyEyes as a proof-of-concept. Unfortunately, IBM's otherwise great web site is broken, so although it will preview the network diagram, it fails to actually publish it to the web. Oh well, ticket opened, etc.

Anyway, I was able to demonstrate the thing to Daniel Davies on my laptop, on the bar of the Nelson's Retreat pub in Old Street. This impressed him excessively. Specifically, we were interested by an odd outlier on the chart. Before I get into that, though, here are some preliminary findings.

1 - Clegg's Diary

At first sight, Nick Clegg appears to be unexpectedly influential. His calender included meetings with NATO, the World Bank, the Metropolitan Police, the Gates Foundation, and oddly enough, Lord Robertson of Port Ellen. Not only that, he had one-to-one meetings with all of them. However, he also got The Elders (i.e. retired politicos playing at shop) and the leader of the Canadian opposition, one Michael Ignatieff, Esq. God help us, is Clegg turning out to be a Decent?

2 - Dave from PR's surprisingly dull world

The Prime Minister, no less, meets with some remarkably dull people. In fact, he met quite a lot of people who you'd expect to be left to flunkies while leaving quite a lot of important people to Nick Clegg. He did get BP, Shell, Pfizer, Rupert Murdoch, the TUC general secretary, and Ratan Tata (twice!) as one-on-ones, but he also met a surprising number of minor worthies from Cornwall and vacuous photocalls with people from Facebook.

3 - Francis Maude, evil genius of the coalition

Secretary of State for the Cabinet Office and Paymaster-General, Francis Maude MP, is the surprise hit, as far as I can make out. He seems to have a special responsibility for anything that smacks of privatisation - therefore, the monetary value of meeting him is probably high. Of course, if your evil genius is Francis Mediocritus, you've got problems. No wonder we're in such a mess. All these points are also true of Oliver Letwin.

4 - Communication and Strategy Management Ltd

This is our far outlier. Some of the least significant people on the chart appear to be government whips, which is obviously an artefact of the data set. The data release does not cover intra-governmental or parliamentary meetings, nor does it cover diplomatic activity. Whips, of course, are a key institution in the political system. Given their special role with regard to both the government and parliament, it's not surprising that they appear to be sheltered from external lobbying - access to the Whips' Office would be such a powerful and occult influence that it must be held closely.

So what on earth is Communication and Strategy Management Ltd., a company which had one-on-one access to the Government Chief Whip, the Rt. Hon. Patrick McLoughlin MP, and which according to Companies House was founded on the 11th of April? It has no web site or perceptible public presence. It is located in what looks like a private house, here, not far from Stratford upon Avon:


View Larger Map

Evidently the hub of political influence, but those are the facts. The directors are Elizabeth Ann Murphy and Richard Anthony Cubitt Murphy*, ignoring a company-formation agent who was a director for one day when setting up the company. It's not as if C&SM Ltd is a constituent of McLoughlin's - he's MP for the Derbyshire Dales. Actually, either the directors are related or else there was a cockup, as Murphy's name on the books was amended from Bromley the day after the company was formed and both were born in 1963. The Companies House filing* doesn't give any other information - accounts aren't due for a while - except that the one share issued is held by Norman Younger, who is a partner in the company formation service that was used.

Anyway, the next stop is to learn how this works and put up a nice little dashboard page to help watch the lobbysphere. I'd be happier doing something with python - such as nodebox - but the diagram is already too big to be useful without interactivity, and you can't stick a NodeBox window in a web page.

*Not the Richard Murphy, who is too young.
*WebCheck - it's not an ugly website, it's a way of life...

so you want to know who's lobbying?

So I was moaning about the Government and the release of lists of meetings with external organisations. Well, what about some action? I've written a scraper that aggregates all the existing data and sticks it in a sinister database. At the moment, the Cabinet Office, DEFRA, and the Scottish Office have coughed up the files and are all included. I'm going to add more departments as they become available. Scraperwiki seems to be a bit sporky this evening; the whole thing has run to completion, although for some reason you can't see all the data, and I've added the link to the UK Open Government Licence twice without it being saved.

A couple of technical points: to start with, I'd like to thank this guy who wrote an alternative to Python's csv module's wonderful DictReader class. DictReader is lovely because it lets you open a CSV (or indeed anything-separated value) file and keep the rows of data linked to their column headers as python dictionaries. Unfortunately, it won't handle Unicode or anything except UTF-8. Which is a problem if you're Chinese, or as it happens, if you want to read documents produced by Windows users, as they tend to use Really Strange characters for trivial things like apostrophes (\x92, can you believe it?). This, however, will process whatever encoding you give it and will still give you dictionaries. Thanks!

I also discovered something fun about ScraperWiki itself. It's surprisingly clever under the bonnet - I was aware of various smart things with User Mode Linux and heavy parallelisation going on, and I recall Julian Todd talking about his plans to design a new scaling architecture based on lots of SQLite databases in RAM as read-slaves. Anyway, I had kept some URIs in a list, which I was then planning to loop through, retrieving the data and processing it. One of the URIs, DEFRA's, ended like so: oct2010.csv.

Obviously, I liked the idea of generating the filename programmatically, in the expectation of future releases of data. For some reason, though, the parsing kept failing as soon as it got to the DEFRA page. Weirdly, what was happening was that the parser would run into a chunk of HTML and, obviously enough, choke. But there was no HTML. Bizarre. Eventually I thought to look in the Scraperwiki debugger's Sources tab. To my considerable surprise, all the URIs were being loaded at once, in parallel, before the processing of the first file began. This was entirely different from the flow of control in my program, and as a result, the filename was not generated before the HTTP request was issued. DEFRA was 404ing, and because the csv module takes a file object rather than a string, I was using urllib.urlretrieve() rather than urlopen() or scraperwiki.scrape(). Hence the HTML.

So, Scraperwiki does a silent optimisation and loads all your data sources in parallel on startup. Quite cool, but I have to say that some documentation of this feature might be nice, as multithreading is usually meant to be voluntary:-)

TODO, meanwhile: at the moment, all the organisations that take part in a given meeting are lumped together. I want to break them out, to facilitate counting the heaviest lobbyists and feeding visualisation tools. Also, I'd like to clean up the "Purpose of meeting" field so as to be able to do the same for subject matter.

Update: Slight return. Fixed the unique keying requirement by creating a unique meeting id.

Update Update: Would anyone prefer if the data output schema was link-oriented rather than event-oriented? At the moment it preserves the underlying structure of the data releases, which have one row for each meeting. It might be better, when I come to expand the Name of External Org field, to have a row per relationship, i.e. edge in the network. This would help a lot with visualisation. In that case, I'd create a non-unique meeting identifier to make it possible to recreate the meetings by grouping on that key, and instead have a unique constraint on an identifier for each link.

Update Update Update: So I made one.

coffee and stupidity

Deeply unprofitable Crooked Timber thread produces an interesting point. Not so much "why do libertarians hate Fairtrade coffee?", but why on earth do they insist on lying? From the link:

By guaranteeing a minimum price, Fairtrade also encourages market oversupply, which depresses global commodity prices. This locks Fairtrade farmers into greater Fairtrade dependency and further impoverishes farmers outside the Fairtrade umbrella. Economist Tyler Cowen describes this as the “parallel exploitation coffee sector”.

Coffee farms must not be more than 12 acres in size and they are not allowed to employ any full-time workers. This means that during harvest season migrant workers must be employed on short-term contracts. These rural poor are therefore expressly excluded from the stability of long-term employment by Fairtrade rules.

The problem here is paragraph two; I've not attempted to estimate the cross-elasticity of demand for Fairtrade and non-Fairtrade coffee, and it's not the kind of thing I'd try unless someone was paying me to do it. But it's perfectly simple to consult the Fairtrade Labelling Organisation International standardisation documents and see if they actually forbid full-time employment or set a maximum limit on the size of farms. Here's the standard on the use of hired labour. Requirement 1.5.1.13:

All regular work is undertaken by permanent workers...The objective is that as often as possible, work is undertaken by permanent workers. Only work that is added to usual work levels during peak seasons may be undertaken by seasonal workers.

So - no. Not even close. 180 degrees out, in fact. The point arises downthread that the great majority of work on a coffee farm is seasonal in nature, so Cowan couldn't possibly have been right even if you let him have his own facts. This is marginally unfair to him; it's not his prose, but rather something he's (very) favourably quoting. The original source is here; note the cites back to Cowan and to people at George Mason University, the Koch Industries school.

For some reason, I still followed a link to his blog after this, to this interesting sociological result. Apparently, if you've been to jail, you're more likely to self-identify as black when you leave jail then when you went in. This is interesting, but then numskull racist twit Steve Sailer showed up in the comments. As far as I can make out from his semi-coherent comments, he's very angry about this, which is odd for someone who believes in biological racism (surely it wouldn't matter? your DNA, after all, doesn't care which census box you tick).

Further, Sailer's freakout reminded me of the old joke about the Wee Free minister who hears that a hippie commune has moved into the next island along. He listens icily as the various novel features are described - the long music, the loud hair, the ruthlessly commercial communists, etc. Eventually they tell him the newcomers believe in free love. "Free love?" he explodes. "That could lead to mixed dancing!" Similarly, it's clear from Sailer's comments that he's opposed to rape in prison because he thinks it might lead to multi-culturalism. And does he think about it. All the time, it seems.

Perhaps his mate Anthony Browne from PolEx could offer him counselling as part of the Big Society?

tha mun laike wi't gangling-iron, Councillor Duxbury

Well, well, well.

The Conservative chairman, Eric Pickles, last night appeared to disown the leadership of the Young Britons' Foundation, a rightwing training organisation for young Conservatives whose officials have described the NHS as "the biggest waste of money in the UK" and suggested the waterboarding of prisoners can be justified.

Pickles spoke last week at a YBF rally at the House of Commons and the group is working with Conservative Future, the party's official youth wing, on pre-election training of young Tory activists. But yesterday that relationship came under serious strain.

"We don't agree with these views," a spokesman for Pickles said in a brief statement. "The YBF organisation is independent of the Conservative party."

Sometimes, what you need is a Yorkshireman. But it would be very interesting to know if the Conservative Party actually knows what, legally, the YBF is, how much money it's received, and how much VAT has been paid.

This is interesting, too:

Cutts said that, as an organiser for Conservative Future's East Midlands region, he came under pressure from the national organisation to ensure there were sufficient paying customers for the YBF courses.

If this is a commercial arrangement in the meaning of Section 50 of the Act, it's a slightly unusual one.

(If anyone gets the joke in the title, I'll be delighted.)

a donation of minus £25

Before moving on to more serious issues, does anyone know what, exactly, the Young Britons' Foundation is? I mean, if you were to sign a cheque to it, what kind of a legal entity is it that cashes it? (I'm tempted to cut them a £10 cheque, just to see what comes up on my bank statement. Or possibly a whole lot of 1p cheques. But that would be infantile. Wouldn't it?) This is an interesting question. Is it a commercial company? Is it a charity? Is it a club of some sort? Is it an unincorporated association? If it was a company limited by guarantee, the usual set-up for a wanktank, it would have to file accounts somewhere, but the funds passing through it would be tax deductible.

I ask because despite its aims:

The Young Britons’ Foundation is recognized as the leading provider of state-of-the-art political training to conservative activists in Britain.

YBF offers a number of one-day specialist training workshops in London as well as bespoke half-day campus campaign workshops throughout Britain.

It is conspicuous by its absence from here. Neither is it a company or a registered charity.

I can see a couple of possibilities. One is that whatever-it-is claims that by charging participants a fee, it's providing services to the Conservative Party on commercial terms and is therefore exempt from registration under Section 50(2) f of the Political Parties, Elections, and Referendums Act 2000. This has some interesting consequences - if this is commercial activity, then surely someone is making a profit from it. If these services were supplied at a loss, the terms would not be commercial ones.

Another is that Section 52(1) g applies, which excludes from registration

the provision by any individual of his own services which he provides voluntarily in his own time and free of charge;

It's possible that the YBF is merely a collection of activists who voluntarily provide these services to the Tories. But the services are not provided free of charge (and in that case, what becomes of the money?).

My favourite explanation, however, is that Section 50(4) b states that:

(4) In determining—

(a) for the purposes of subsection (2)(e), whether any money lent to a registered party is so lent otherwise than on commercial terms, or

(b) for the purposes of subsection (2)(f), whether any property, services or facilities provided for the use or benefit of a registered party is or are so provided otherwise than on such terms,

regard shall be had to the total value in monetary terms of the consideration provided by or on behalf of the party in respect of the loan or the provision of the property, services or facilities.

I think I see the pattern. The argument is clearly that his courses are worthless, and therefore have no market value whatsoever, and therefore, cannot possibly constitute a political donation subject to registration under the 2000 Act?

This is not impossible in practical terms. Consider the YouGov polling of marginal seats; the difference between the famous Ashcroft-targeted marginals and the uniform national swing has narrowed sharply over the last 18 months, which seems to suggest it's possible to spend and spend on a Tory campaign and actually do worse.

a decline into local navel-gazing

Can we, can we, can we have better thinktanks already? I'm not so sure whether the worst bit is the fact they got the number wrong by a factor of four, or that they didn't know that the National Rivers Authority doesn't exist and hasn't for 14 years, or just that it's so obvious that the whole thing was based on the following reaction: "Whoo! Jetboats!!"

This project of not spending all my time Tory-slapping is going to be tough.

a single net of conspiracy

Well, this is hardly surprising; the FBI was in the habit of pretending to be on a terrorism case every time they wanted telecoms traffic data. Their greed for call-detail records is truly impressive. Slurp! Unsurprisingly, the lust for CDRs and the telcos' eagerness to shovel them in rapidly got the better of their communications analysis unit's capacity to crunch them.

Meanwhile, Leah Farrell wonders about the problems of investigating "edge-of-network" connections. Obviously, these are going to be the interesting ones. Let's have a toy model; if you dump the CDRs for a group of suspects, 10 men in Bradford, and pour them into a visualisation tool, the bulk of the connections on the social network graph will be between the terrorists themselves, which is only of interest for what it tells you about the group dynamics. There will be somebody who gets a lot of calls from the others, and they will probably be important; but as I say, most of the connections will be between members of the group because that's what the word "group" means. If the likelihood of any given link in the network being internal to it isn't very high, then you're not dealing with anything that could be meaningfully described as a group.

By definition, though, if you're trying to find other terrorists, they will be at the edge of this network; if they weren't, they'd either be in it already, or else they would be multiple hops away, not yet visible. So, any hope of using this data to map the concealed network further must begin at the edge of the sub-network we know about. And the principle that the ability to improve a design occurs primarily at the interfaces - this is also the prime location for screwing it up also points this way.

But there's a really huge problem here. The modelling assumptions are that a group is defined by being significantly more likely to communicate among itself than with any other subset of the phone book, that the group is small relative to the world around it, and that it is boring; everyone has roughly similar phoning behaviour, and therefore who they call is the question that matters. I think these are reasonable.

The problem is that it's exactly at the edge of the network that the numbers of possible connections start to curve upwards, and that the density of suspects in the population falls. Some more assumptions; an average node talks to x others, with calls being distributed among them on a well-behaved curve. Therefore, the set of possibilities is multiplied by x for each link you follow outwards; even if you pick the top 10% of the calling distribution, you're going to fall off the edge as the false positives pile up. After three hops and x=8, we're looking at 512 contacts from the top 10% of the calling distribution alone.

In fact, it's probably foolish to assume that suspects would be in the top 10% of the distribution; most people have mothers, jobs, and the like, and you also have to imagine that the other side would deliberately try to minimise their phoning or, more subtly, to flatten the distribution by splitting their communications over a lot of different phone numbers. Actually, one flag of suspicion might be people who were closely associated by other evidence who never called each other, but the false positive rate for that would be so high that it's only realistically going to be hindsight.

Conclusions? The whole project of big-scale database-driven social network analysis is based on the wrong assumptions, which are drawn either from military signals intelligence or from classical policing. Military traffic analysis works because it assumes that the available signals are a subset of a much bigger total, and that this total is large compared to the world. This makes sense because that's what the battlefield of electronic warfare is meant to look like - cleared of civilian activity, dominated by one side or the other's military traffic. Working from the subset of enemy traffic that gets captured, it's possible to infer quite a lot about the system it belongs to.

Police investigation works because it limits the search space and proceeds along multiple lines of enquiry; rather than pulling CDRs and assuming the three commonest numbers must be suspects, it looks for suspects based on the witness and forensic evidence of the case, and then uses other sources of data to corroborate or refute suspicion.

To summarise, traffic analysis works on the assumption that there is an army out there. We can only see part of it, but we can make inferences about the rest because we know there is an army. Police investigation works on the observation that there has been a crime, and the assumption that probably, only a small number of people are possible suspects.

So, I'm a bit underwhelmed by projects like this. One thing that social network datamining does, undoubtedly, achieve is to create handsome data visualisations. But this is dangerous; it's an opportunity to mistake beauty for truth. (And they will look great on a PowerPoint slide!)

Another, more insidious, more sinister one is to reinforce the assumptions we went into the exercise with. Traffic-analysis methodology will produce patterns; our brains love patterns. But the surge of false positives means that once you get past the first couple of hops, essentially everything you see will be a false positive result. If you've already primed your mind with the idea that there is a sinister network of subversives everywhere, techniques like this will convince you even further.

Unconsciously, this may even be the purpose of the exercise - the latent content of Evan Kohlmann. At the levels of numbers found in telco billing systems, everyone will eventually be a suspect if you just traverse enough links.

Which reminded me of Evelyn Waugh, specifically the Sword of Honour trilogy. Here's his comic counterintelligence officer, Colonel Grace-Groundling-Marchpole:

Colonel Marchpole's department was so secret that it communicated only with the War Cabinet and the Chiefs of Staff. Colonel Marchpole kept his information until it was asked for. To date that had not occurred and he rejoiced under neglect. Premature examination of his files might ruin his private, undefined Plan. Somewhere, in the ultimate curlicues of his mind, there was a Plan.

Given time, given enough confidential material, he would succeed in knitting the entire quarrelsome world into a single net of conspiracy in which there were no antagonists, only millions of men working, unknown to one another, for the same end; and there would be no more war.

Want a positive idea? One reading of this and this would be that the failure of intelligence isn't a failure to collect or analyse information about the world, or rather it is, but it is caused by a failure to collect and analyse information about ourselves.

Profiles in Wanktankery: Alexander Meleagrou-Hitchens

How could I forget this?

The Obscurer's coverage of the Undabomber has been marked by one man. Here he is:

Peter Hoekstra, the senior Republican on the House intelligence committee, said it was examining Mutallab's links with the radical Yemeni imam, Anwar al-Awlaki, who has inspired a number of terrorists.

Awlaki had contacts with Major Nidal Hasan, the Army psychiatrist who is accused of carrying out the massacre at Fort Hood, Texas, in November in which 13 people were murdered. According to government officials, Awlaki was also the spiritual adviser to two of the 9/11 hijackers, Khalid al Mihdhar and Nawaf al-Hazmi, while he was an imam at a mosque in northern Virginia. The FBI investigated him in 1999 and 2000, believing him to be a possible procurement agent for Osama bin Laden.

In Toronto, a terror cell watched videos of Awlaki at a makeshift training camp where an attack was planned on the Canadian parliament and prime minister. "He's a star attraction as a recruiter to young Americans and Canadians," one former American intelligence official told the US media.

This month, in an interview with Al Jazeera, Awlaki expressed surprise that the US military had failed to uncover Hasan's plan, to which he gave his backing. "My support to the operation was because the operation brother Nidal carried out was a courageous one, and I endeavoured to explain my position regarding what happened because many Islamic organisations and preachers in the west condemned the operation," he said.

Awlaki left the US and moved to Yemen in 2002 where he established an English-language website that has thousands of followers around the world. In January 2009, he published an online essay, 44 Ways to Support Jihad, in which he asserts that all Muslims must participate in jihad, whether in person, by funding mujahideen or by fighting the west.

There's something missing here...can you spot it?

Concerns about his influence in the UK have been expressed by experts on community cohesion. In August, the Observer reported anger that Awlaki was due to speak via a video link at Kensington town hall. The broadcast was dropped after the local council stepped in. He has also been invited to give talks via video link at several London universities. "Mutallab is the latest in a long list of terrorists [Awlaki] has inspired and encouraged," said Alexander Meleagrou-Hitchens of the Centre for Social Cohesion.

"The preacher has long been a highly respected figure within a number of British university Islamic societies because, unlike most other radical preachers, Awlaki speaks English as a first language, and being born and raised in America has given him a good understanding of western culture. This makes him very appealing to young western Muslims."

Meleagrou-Hitchens called for British universities to increase their vigilance. "This incident should act as a wake-up call to university authorities," he said. "It is crucial that they now accept the central role they must play in resisting extremists and preventing student groups from promoting hate preachers."

Did you spot it? The Obscurer didn't actually say that he had any connection with the pants bomber. They didn't even quote Hoekstra saying so - and Hoekstra is a comedy rightwing buffoon anyway. They didn't adduce any evidence of his connections with him in any way - just cut straight to Alexander Meleagrou-Hitchens. Whose kid are you?

Oh, right. He's "worked" for Standpoint, the Centre for Social Cohesion, Policy Exchange, and the Henry Jackson Society. I think he gets a free cup of coffee and 200 air miles if he can punch another content-free wanktank funded by the Tories' neocon wing on his loyalty card.

PolEx's Web site has an "Alumni" page, but mysteriously it bears no trace of him. Google, however, knows:

He has also worked at the Stanford University based think tank, the Hoover Institution for War, Revolution and Peace, and the Washington DC based think tank, Foundation for Defence of Democracies (FDD).

He holds an MA in International Relations from Brunel University, and a BA in Classics from King’s College London.

Alexander researched for publications providing policy recommendations on creating a robust defence against the threat of terrorism in the UK and abroad.

FDD as well! Free cuppa for you! There is, of course, no suggestion of or link to any work on terrorism he ever did.

Today, he's in the Obscurer again. Let's roll the tape.

Recordings of Anwar al-Awlaki, an al-Qaida sympathiser who is believed to have inspired Abdulmutallab in Yemen, can be bought through British-based websites and bookshops. Three shops in London and Manchester were contacted by this newspaper last week. Staff said they could sell DVDs of the speeches by the cleric, who is banned from the UK.

As recently as last April, students at London's City University Islamic Society's annual dinner were invited to hear the words of al-Awlaki being broadcast live into Britain.

So why is he "believed" to have inspired pants boy? Where is the evidence? It's not even the electioneering torture fan Hoekstra this time.

Alexander Meleagrou-Hitchens, a research fellow for the think-tank the Centre for Social Cohesion,

For it is he.

said that al-Awlaki has become an increasingly influential figure. "For well over a year now, organisations such as ours have repeatedly warned about the dangerous influence of this man, with most of our warnings falling on deaf ears," he said.

Call now and buy your anti-terrorist water ioniser - 20 per cent off before the end of this broadcast! And don't forget to donate now and claim Gift Aid!

"They had no objection to his giving a video sermon to a gathering at Kensington and Chelsea town hall. We are also often told that, although al-Awlaki's views may be unsavoury, he has never been convicted of any crime. Clearly, this excuse is simply not good enough."

The excuse that he hasn't done anything wrong.

Further, Hitchens Minor seems to be missing someone in his laudable crusade on the home front. I refer, of course, to the current and past tenants of Kensington & Chelsea Town Hall, or in other words, the Conservative Party in London. Could this perhaps have something to do with the fact that his boss at Policy Exchange is now the Conservative Mayor of London's director of policy?

wanktanks against racial profiling

Whilst we're on the subject, a couple of other terrorism things. Kingpin of Comedy Gladio, intimate of Patrick Mercer MP, and self-made spy Glen Jenvey has been arrested, ironically on charges of inciting hatred against Jews. He's finally convinced somebody that he's a real jihadi! Not that he's likely to convince anyone else of anything at all...except Tories. The government must, indeed, look at these coincidences!

Meanwhile, in this season of prodigies, Ed Husain astonishingly says something quite sensible.

In both Britain and America demands for profiling all Muslims at airports are increasing in volume. This mindset not only fails to understand that most Muslims around the world detest al-Qaeda, but this outlook also cannot comprehend how terrorists are always one step ahead of the game. If it is Muslim-sounding names that are to be stopped, would a name like Richard Reid – the infamous shoe bomber – have been detected? If it is Asian men that are to be stopped, then we will see an increase in white men recruited for terror?

After all, al-Qaeda's English spokesperson is Adam Gadahn, a white American. If it is men who are stopped, we will see women terrorists emerge. Let us not forget Palestinian groups' repeated use of single women as suicide bombers. Do not underestimate the power of terrorists to recruit serving airline pilots and other aviation personnel. Where there is a will, there will always be a way.

The profiling of ordinary Muslims not only opens other avenues for al-Qaeda, but results in the harassment and potential loss of support from the very people we need on our side to contain al-Qaeda: ordinary Muslims.

He also spends quite a lot of time romanticising Sufism, retailing whines about Obama bowing to the Saudi king, and saying things like this:

Nearly a decade after 9/11, when compared with military budgets, where is investment in these soft-power, counter radicalisation projects? The silence says it all.

Indeed it does, indeed it does. Strangely, he didn't include a PayPal DONATE button. But it's a start, I suppose, and we should reward good behaviour when we see it, even from the wanktank community. We Are Reasonable People, after all.

Someone who isn't reasonable, however, is Con Coughlin:

It is easy to imagine that the authorities at UCL took quiet pride in the fact that they had a radical Nigerian Muslim running their Islamic Society. You can’t get more politically correct than that? They would therefore have had little interest in monitoring whether he was using a British university campus as a recruiting ground for al-Qaeda terrorists such as himself. The authorities at UCL should hang their heads in shame – or better than that, perhaps they should resign?

It may be easy to imagine that, but what are your offensive imaginings doing in something that claims to be a newspaper? Good job you only imagined it or else you could have been sued. It's a bit like adumbrating that way.

Did Con Coughlin, perhaps, imagine the story about those Iraqi chemical weapons that he reckoned would fit on RPGs and be issued to air defence personnel so they could be launched in 45 minutes? The truth was considerably worse than that.

CCC vs Wanktanks

Other people's TDLs. I note that this year's CCC has devoted some time to the wanktank phenomenon in Germany, and also elsewhere. Volker Bilk's slides are here; it makes a lot of sense, but there's quite a bit of German philosophical maundering in there, and I'd have liked to see more case studies/howtos. As far as his call to action to map the wanktanks goes, I fully agree.

In other CCC presentations, it looks like there's going to be a lot of GSM hacking in 2010. See here, here and here.