the missing link between slimming tea and tactical electronic warfare

Well, speak of the devil. Peter Foster makes his appearance in the Murdoch scandal and fingers the Sun directly.

He said he then received an email from a Dublin-based private investigator calling himself ''Autarch'', who told Mr Foster he tapped into his mother's phone in December 2002.

That month, The Sun published the ''Foster tapes'', which featured transcripts of Mr Foster talking about selling the story of his links with Tony Blair's wife, Cherie. Yesterday, Mr Foster said he had since had a Skype conversation with the investigator in Dublin, in which Autarch described how he tapped into Mr Foster's mother's phone.

''He said she was using an analogue telephone which they were able to intercept,'' Mr Foster said. Autarch said he discussed the hacking with Sun journalists.

However, this story - at least this version of it - probably isn't true. It is true that the first-generation analogue mobile phone systems like TACS in the UK and AMPS in the States were unencrypted over the air, and therefore could be trivially intercepted using a scanner. (They were also frequency-division duplex, so you needed to monitor two frequencies at once in order to capture both parties to the call.) It is also true that they were displaced by GSM very quickly indeed, compared to the length of time it is expected to take for the GSM networks to be replaced. In the UK, the last TACS network (O2's) shut down in December 2000. It took a while longer in the Republic of Ireland, but it was all over by the end of 2001.

So Foster is bullshitting...which wouldn't be a surprise. Or is he? TACS wasn't the only analogue system out there. There were also a lot of cordless phones about using a different radio standard. Even the more modern DECT phones are notorious for generating masses of radio noise in the 2.4GHz band where your WiFi lives. It may well be the case that "Autarch" was referring to an analogue cordless phone. Because a lot of these were installed by individual people who bought them off the shelf, there was no guarantee that they would be replaced with newer devices. (Readers of Richard Aldrich's history of GCHQ will note that his take on the "Squidgygate" tape is that it was probably a cordless intercept.)

This would have required a measure of physical surveillance, but then again so would an attempt to intercept mobile traffic over-the-air as opposed to interfering with voicemail or the lawful intercept system.

The Daily Beast has a further story, which points out that the then editor David Yelland apologised after being censured by the Press Complaints Commission (no wonder he didn't go further in the Murdoch empire) and makes the point that such an interception was a crime in both the UK and Ireland at the time. They also quote Foster as follows:

According to Foster, the investigator told him that, for four days at the height of Cheriegate, he had been sitting with another detective outside Foster’s mother’s flat in the Dublin suburbs, intercepting and recording the calls to her cordless landline

The Sun hardly made any effort to conceal this - they published what purports to be a transcript, as such.

The intersection of electronic warfare and mall management

Here's something interesting. You may remember this story from back in November about the CIA spy network in Lebanon that met at a Pizza Hut they codenamed PIZZA, and which was rolled up by a joint Hezbollah-Lebanese military intelligence investigation. The key detail is as follows:

U.S. officials also denied the source's allegation that the former CIA station chief dismissed an email warning that some of his Lebanese agents could be identified because they used cellphones to call only their CIA handlers and no one else.
...
Lebanon's security service was able to isolate the CIA informants by analyzing cellphone company records that showed the numbers called, duration of each call and location of the phone at the time of the call, the source said.

Using billing and cell tower records for hundreds of thousands of phone numbers, software can isolate cellphones used near an embassy, or used only once, or only on quick calls. The process quickly narrows down a small group of phones that a security service can monitor.

If the top paragraph is true, it would have been catastrophically ill-advised. Even somebody special, like a CIA agent under diplomatic cover, has a relatively large number of weak ties to normal people. This is the reverse of the small-world principle, and is a consequence of the fact that the great majority of people are real human beings rather than important persons. As a result, things like STELLAR WIND, the illegal Bush-era effort to analyse the whole pile of call-detail records at AT&T and Verizon in the hope that this would find terrorists, face a sort of Bayesian doom. We've gone over this over and over again.

However, phone numbers that only talk to special people are obviously suspicious. Most numbers with a neighbourhood length of 1 will be things like machine-to-machine SIMs in vending machines and cash points, but once you'd filtered those out, the remaining pool of possibles would be quite small. It is intuitive to think of avoiding surveillance, or keeping a low profile, but what is required is actually camouflage rather than concealment.

There are more direct methods - which is where electronic warfare and shopping mall management intersect.

Path Intelligence, a Portsmouth-based startup, will install a network of IMSI-catchers, devices which act as a mobile base station in order to identify mobile phones nearby, in your shopping centre so as to collect really detailed footfall information.

Similarly, you could plant such a device near that Pizza Hut to capture which phones passed by and when, and which ones usually coincided. Alternatively, you could use it in a targeted mode to confirm the presence or absence of a known device. Which makes me wonder about the famous Hezbollah telecoms network, and whether it was intended at least in part to be an electronic-intelligence network - as after all, nothing would be a better cover for a huge network of fake mobile base stations than a network of real ones.

Meanwhile, this year's CCC (like last year's) was just stuffed with GSM exploits. It really is beginning to look a lot like "time we retired that network".

The RQ-170 hack and the drone bubble

The fact that a majority of this year's graduates from USAF basic pilot training are assigned to drone squadrons has got quite a bit of play in the blogosphere. Here, via Jamie Kenny, John Robb (who may still be burying money for fear of Obama or may not) argues that the reason they still do an initial flight training course is so that the pilot-heavy USAF hierarchy can maintain its hold on the institution. He instead wants to recruit South Korean gamers, in his usual faintly trendy dad way. Jamie adds the snark and suggests setting up a call centre in Salford.

On the other hand, before Christmas, the Iranians caught an RQ-170 intelligence/reconnaissance drone. Although the RQ-170 is reportedly meant to be at least partly stealthy, numerous reports suggest that the CIA was using it among other things to get live video of suspected nuclear sites. This seems to be a very common use case for drones, which usually have a long endurance in the air and can be risked remaining over the target for hours on end, if the surveillance doesn't have to be covert.

Obviously, live video means that a radio transmitter has to be active 100% of the time. It's also been reported that one of the RQ-170's main sensors is a synthetic-aperture radar. Just as obviously, using radar involves transmitting lots of radio energy.

It is possible to make a radio transmitter less obvious, for example by saving up information and sending it in infrequent bursts, and by making the transmissions as directional as possible, which also requires less power and reduces the zone in which it is possible to detect the transmission. However, the nature of the message governs its form. Live video can't be burst-transmitted because it wouldn't be live. Similarly, real-time control signalling for the drone itself has to be instant, although engineering telemetry and the like could be saved and sent later, or only sent on request. And the need to keep a directional antenna pointing precisely at the satellite sets limits on the drone's manoeuvring. None of this really works for a mapping radar, though, which by definition needs to sweep a radio beam across its field of view.

Even if it was difficult to acquire it on radar, then, it would have been very possible to detect and track the RQ-170 passively, by listening to its radio emissions. And it would have been much easier to get a radar detection with the advantage of knowing where to look.

There has been a lot of speculation about how they then attacked it. The most likely scenario suggests that they jammed the command link, forcing the drone to follow a pre-programmed routine for what to do if the link is lost. It might, for example, be required to circle a given location and wait for instructions, or even to set a course for somewhere near home, hold, and wait for the ground station to acquire them in line-of-sight mode.

Either way, it would use GPS to find its way, and it seems likely that the Iranians broadcast a fake GPS signal for it. Clive "Scary Commenter" Robinson explains how to go about spoofing GPS in some detail in Bruce Schneier's comments, and points out that the hardware involved is cheap and available.

Although the military version would require you to break the encryption in order to prepare your own GPS signal, it's possible that the Iranians either jammed it and forced the drone to fall back on the civilian GPS signal, and spoofed that, or else picked up the real signal at the location they wanted to spoof and re-broadcast it somewhere else, an attack known as "meaconing" during the second world war when the RAF Y-Service did it to German radio navigation. We would now call it a replay attack with a fairly small time window. (In fact, it's still called meaconing.) Because GPS is based on timing, there would be a limit to how far off course they could put it this way without either producing impossible data or messages that failed the crypto validation, but this is a question of degree.

It's been suggested that Russian hackers have a valid exploit of the RSA cipher, although the credibility of this suggestion is unknown.

The last link is from Charlie Stross, who basically outlined a conceptual GPS-spoofing attack in my old Enetation comments back in 2006, as a way of subverting Alistair Darling's national road-pricing scheme.

Anyway, whether they cracked the RSA key or forced a roll-back to the cleartext GPS signal or replayed the real GPS signal from somewhere else, I think we can all agree it was a pretty neat trick. But what is the upshot? In the next post, I'm going to have a go at that...

Liam Fox: Not Fit For Purpose

OK, so "Not All That" Foxy Liam Fox is in trouble.

"He is an odd bloke," said one fellow minister. "He has fingers in so many pies that you kind of think one of them will land him in trouble somewhere along the line."

Another Tory MP said Fox's tendency to name-drop and brag about his close friendships with Republicans in the US, media magnates such as David and Frederick Barclay (owners of the Daily Telegraph), and his endless globe-trotting, even before he entered the cabinet, has made many bristle and help explain why he has plenty of enemies in the Tory party and in Whitehall. "I think you either roll with the bluster or find it repellent," said a Tory MP.

Ah, one of them. Anyway. Part of the problem is this famous meeting where his bestie Adam Werritty just happened to turn up. What was on offer? Well, a product called Cellcrypt, whose makers were trying to sell it to the MoD to stop evilly-disposed persons from eavesdropping on British soldiers' phone calls back to the UK. (Note: this is going to be long. Technical summary: voice encryption apps for GSM-style mobile networks can guarantee that your call will not be overheard, but not that your presence cannot be monitored, and not necessarily that the parties to your calls cannot be identified.)

Back in the early days of Iraq, the CPA permitted one mobile phone operator in each of its three zones to set up. The British zone, CPA-South/Multinational Division South-East, let the Kuwaiti national telco, MTC (now Zain and busy running Mo Ibrahim's old Celtel business into the ground) set up there with a partner some of us may have heard of. It's from Newbury and it's not a pub or an estate agency and its logo is a big red comma...funny how Vodafone never talked that particular investment up, innit? Anyway. Later the Iraqi government did a major tender for permanent licences and Orascom got most of it, but that's another story.

One thing that did happen was that soldiers took their mobiles with them to Iraq, and some of them pretty soon realised that buying a local SIM card in the bazaar was much cheaper than making roaming calls back to the UK. Either way, lots of +44 numbers started showing up in their VLR, the big database that keeps track of where phones are in a GSM network so it can route incoming calls.

Pretty soon someone who - presumably - worked for the MTC-Voda affiliate and whose purposes were not entirely aligned with Iraq The Model realised that you could use the VLR to follow the Brits (and the Yanks and the Danes and the Dutchmen and Kiwis and all sorts of contractors) around. Not only that, you could ring up their families in the UK and make threats with the benefit of apparently supernatural knowledge.

This obviously wasn't ideal. Efforts were made to mitigate the problem; soldiers were discouraged from using local GSM networks, more computers and public phones were made available. The eventual solution, though, was to get some nice new ruggedised small-cell systems from companies like Private Mobile Networks Ltd., which basically pack a small base station and a base station controller and a satellite backhaul terminal into a tough plastic box of a suitably military colour. You open it up, unfold the antenna, turn on the power, and complete some configuration options. It logs into the mobile operator who's providing service to you via the satellite link.

Now, because radio signals like all radiation lose intensity with the inverse square of the distance, you'll be vastly louder than everyone else. So any mobile phone nearby will roam onto your private mobile network and will be in the UK for mobile phone purposes, a bit like the shipping container that's technically in Egypt at the end of Four Lions. And none of this will touch any other mobile network that might be operating in your area. Obviously you can also use these powers for evil, by snarfing up everyone else's traffic, and don't for a moment think this isn't also done by so-called IMSI catchers.

You're not meant to do this, normally, because you probably don't have a licence to use the GSM, GSM/PCS, or UMTS frequencies. But, as the founder of PMN Ltd. told a colleague of mine, the answer to that is "we've got bigger tanks".

So, where were we? Well, the problem with trying to do...something...with Cellcrypt is that it doesn't actually solve this problem, because the problem wasn't originally that the other side could listen to the content of voice calls. Like all telecoms interception stories, it was about the traffic analysis, not the content. Actually, they probably could listen in as well because some of the Iraqi and Afghan operators may not have been using up-to-date or even *any* air interface encryption.

But if you're going to fix this with an encryption app like Cellcrypt, you've got to make sure that every soldier (and sailor and diplomat and journo and MoD civilian) installs it, it works on all the phones, and you absolutely can't make calls without it. Also, you've got to make sure all the people they talk to install it.

And the enemy can still follow you because the phones are still registering in the VLRs!

So, there's not much point relying on OTA voice encryption to solve a problem that's got nothing to do with the voice bearer channel. However, bringing your own small cell network certainly does solve the problem, elegantly, and without needing to worry about what kind of phones people bring along or buy locally.

And the military surely understand this, as by the time of the famous meeting, they'd already started deploying them. Also, back when this was a big problem, 19 year-old riflemen usually didn't have the sort of phones that would run a big hefty application like Cellcrypt, which also uses the mobile data link and therefore would give them four figure phone bills.

To sum up, Werritty was helping someone market gear that the MoD didn't need, that was hopelessly unfit for purpose, wouldn't actually do what the MoD wanted, and would cost individual soldiers a fortune, by providing privileged access to the Secretary of State for Defence.

discovering the axis of barking

The Grauniad Dabatlog has produced a rather fancy network visualisation of the sources cited in Anders Behring Breivik's personal manifesto/horse-shit compendium. This is great as I now don't need to worry that I perhaps should have made one. It's very pretty and you can click on stuff, and see that some of the sources are thinktanks and some of them are newspapers, and well, it's very pretty and you can click on stuff. It also comes with a piece by Andrew Brown reprising his "Don't be beastly to the creationists!" shtick but with Melanie Phillips, for some reason.

Unfortunately it's almost completely intransparent, and gives little indication of what data is being visualised or on what basis, and there is really no obvious conclusion to draw from it. But did I mention pretty and click? If forced to take a view, I would reckon that the underlying data is probably a matrix of which sources appear together with others and the layout algo is a force-directed graph (aka the default in pretty much any visualisation toolkit), probably weighted by appearance count. There's some sort of proprietary metric called "linkfluence" which appears to be given by(indegree/outdegree)*len(neighbourhood) or words to that effect.

As a result, the only information I got from it was that he linked to Wikipedia, the BBC, and big news sites a lot. Well yes; Wikipedia, bbc.co.uk, etc, generate a hell of a lot of web pages and people read them a lot. Obviously, to say the least, you need to normalise the data with regard to sheer bulk, or you'd end up concluding that Google (or Bing or Yahoo) was his inspiration because he did a lot of web searches, or that he was a normal man twisted by SMTP because he used e-mail.

In fact, I thought they actually did that until I realised that RSS.org is about the other RSS, the Indian extreme-right movement, not the popular Internet syndication standard. Harrowell fail. Anyway, it does show up rather nicely that the groups "European nationalists", "Counter-Jihad", and "American Right-Wing" overlap. However, I feel there's something missing in the characterisation of MEMRI and various other sites as just "Think Tanks" as if they were just like, say, IPPR.

Also, an emergent property of the data is that there is an Axis of Barking running vertically through it: the nearer you are to the top of the diagram, the more extreme and crazy. MEMRI, FrontPage, Gates of Vienna, Melanie Phillips are near the top; the Wikipedia article on the Russo-Turkish War of 1878 is at the bottom. And the MSM is somewhere in the middle. (Although I do wonder if they allocated the sources to groups before or after running the force-directed graph.)

It seems to be one of those command the exciting world of social media with just one click! things.

Anyway, upshot. I want to avoid Project Lobster producing a diagram like this one. It's too impressionistic and fluffy and reliant on basically aesthetic reasoning. (I think we've had this point before.) Of course, that's partly the difference between the underlying data sets; it was at least thinkable if unlikely that there would be no grouping in Breivik's sources, while presumably political lobbying is nonrandom and subject to intelligent design.

Elsewhere, a reader passed this along which I need to actually watch (isn't video time consuming?). There's a shindig in Warsaw in late October. And I want this on a T-shirt.

it could be you, but not while she's around

This is fascinating, especially when you remember GTech's role in our own dear sordid raffle. Great reporting, too.

your call could not be connected - please check the number and try again

The Obscurer has possibly the first intelligent article on the whole "turn off their Facebook! that'll learn em!" furore. Notably, they interviewed one-man UK mobile industry institution Mike Short. Go, read, and up your clue. I especially liked that the piece provided some facts about the 7th July 2005 terrorist incident and the mobile networks.

There is only one reported case of a UK network being closed by police. During the 7/7 London suicide bombings, O2 phone masts in a 1km square area around Aldgate tube station were disconnected for a number of hours.

Police have an emergency power to order masts to be put out of action known as MTPAS – Mobile Telecommunication Privileged Access Scheme. The move has to be approved by Gold Command, by the officers in highest authority during a major incident, and is designed to restrict all but emergency service phones with registered sim cards from making calls. But a shutdown can have dangerous knock-on effects. Short says that phones within the Aldgate zone automatically sought a signal from live masts outside it, overloading them and causing a network failure that rippled out "like a whirlpool".

On the day, other networks were simply overloaded as Londoners sought reassurance and information. Vodafone alone experienced a 250% increase in call volumes

MTPAS is the GSM-land equivalent of the old fixed phone Telephone Preference Scheme (not to be confused with the new one that blocks cold-callers), which permitted The Authorities to turn off between 1% and 90% of phone lines in order to let official traffic through. As far as I know, the Met never asked for it and it was City of London Police who initiated it without asking the Met or anyone else, and in fact O2 UK's network had been keeping up with demand up to that point, before the closure caused the cascade failure Short describes.

The significance of O2 is that it used to be "Surf the Net, Surf the BT Cellnet" and some residual gaullist/spook reflex in the government tried to keep official phones on what was then one of two British-owned networks.

Anyway, this weekend seems to have the theme "The Intersection of Charlie Stross and the August 2011 Riots". Charlie's talk at USENIX is sensibly sceptical about some tech dreams as they apply to networking.

This leaves aside a third model, that of peer to peer mesh networks with no actual cellcos as such – just lots of folks with cheap routers. I’m going to provisionally assume that this one is hopelessly utopian, a GNU vision of telecommunications that can’t actually work on a large scale because the routing topology of such a network is going to be nightmarish unless there are some fat fibre optic cables somewhere in the picture. It’s kind of a shame – I’d love to see a future where no corporate behemoths have a choke hold on the internet – but humans aren’t evenly distributed geographically.

Especially as the theoretical maximum bandwidth of one fibre is about the same as the entire radio spectrum. And the point about routing table size and complexity is a very good one, especially as it's assumed that the routers aren't CRS-1s but rather Linksys fifty quidders or mobile phones.

However, one thing the liberation technologists should take away from the riots is that you shouldn't get hung up on bandwidth. It's great to be able to post the photos on Flickr, but it's more useful to have your own secure voice and messaging. When the Egyptian government relented on its GSM cut-off, the Egyptian Twitter feeds lit up with calls for more people to this or that exit of Tahrir Square or medical supplies to the clinic or (and I remember this) that a lost child was waiting at the press tent.

It was what NANOG users would call operational content. There was of course no need whatsoever for it to go via a Bay Area website - all Twitter provided was the one-to-many element, very important, and the publicity on the Web. The latter is a nice-to-have feature, the former, critical. Text, or even voice, is not a high bandwidth application and doesn't necessarily need access to the global Internet.

So yes - perhaps there is in fact quite a bit of angular momentum to be had in a mobile mesh-WLAN client as an instrument of democracy, as long as you're willing to accept that it's not the sort of thing that can be exclusive to people who agree with you. But then, that's the test of whether or not you actually believe in democracy.

Something else, between Charlie's USENIX talk and the riots. Isn't one of the biggest disappointments, from a police point of view, the performance of CCTV? No doubt it will help put some of the rioters in jail. But it didn't prevent the riots and neither did it seem to help quell them much. It's possible that the whole idea that potential surveillance (like the original panopticon) is a policing influence isn't as strong as it's made out to be.

Another point; not all crimes are punished or even taken notice of. This is obvious. Less obvious is that the degree to which the police ignore crime is an important political fact. Is it possible that CCTV, by forcing them to make at least a token response to everything that passes in camera range, actually contributed to using up the police strength? In a riot, the police aim is to demonstrate public, mass control. They are usually willing to ignore quite a lot of individual criminality in the process. It's possible that surveillance culture and technology are opposed to strategy.

a city wired for sound

Am I right in thinking that Andy Hayman's testimony yesterday fingered Met press chief Dick Fedorcio? Hayman admitted he'd regularly had dinner with News International executives while he was meant to be investigating them. He mentioned that he had done this in the company of the head of communications of the Met, presumably with his approval, although Hayman was also acting in his capacity as ACPO media lead.

Fedorcio has had the same job since 1997. He was named by Nick Davies as having been present in the meeting where the Met demanded to know why Dave Cook was being followed by News International private detectives, and apparently intervened with senior police officers to get them to go easy on NI. Surely the guy in charge of police-press-political relations is a key figure in a scandal that's all about relations between the press, the police, and politics?

Like the key News International men, Alex Marunchak and Greg Miskiw, there's no sign of him. The Home Affairs committee, and indeed anyone else who wants the truth about this, must call Fedorcio without delay. Oh, and is Greg Miskiw in the UK?

Second point. Yesterday's New York Times claims that Miskiw and others on the NOTW were able to locate mobile phones by paying £500 a shot to a corrupt police officer. That is to say, this policeman had access to the lawful intercept systems that are part of all GSM and UMTS cellular networks, or at least he could task people who did. ETSI Specification 01.33 defines this as a standard element of all GSM networks and the corresponding 3GPP TS 33.106 does so for UMTS ones.

If this is so, they could certainly also get pen-register information - lists of calls to and from given phone numbers - and even tap the calls themselves.

This is a massive violation of the UK's critical national infrastructure security, of the Regulation of Investigatory Powers Act, and of the Data Protection Act. News International, their police contact, and the police force responsible (not necessarily the Met) should all be prosecuted.

There is an urgent need to audit the lawful interception systems' logs, among other things to find out if there are other unauthorised users out there. International standards foresee a detailed audit trail as part of these systems in order to preserve the legal chain-of-evidence. If the Interception Request message was submitted in proper form from the police to the telcos, the operators are legally in the clear, but if I was in charge of their network security I'd suspend processing the requests until such an audit was carried out as we now know that an unknown but significant percentage of them are illegal.

Thank fuck we didn't build that giant national ID card database.

Third point. Not that anyone will answer this, but were any of the Prime Minister's designated deputies for nuclear retaliation subject to illegal telecoms surveillance?

Fourth point. Circling back to the Defence Vetting Agency and Andy Coulson, the vetting procedure as described on the DVA Web site states that in some cases, the decision may be taken to issue a security clearance subject to risk management measures taken by the department involved. In these cases, the DVA will disclose information to the sponsoring department that it would usually keep confidential. Did they make such a recommendation to the Prime Minister's office, and if so, what was the information?

If you're out of luck and out of work, we could send you to the western mountains of Libya

The Libyan rebels are making progress, as well as robots. Some of them are reported to be within 40 miles of Tripoli, those being the ones who the French have been secretly arming, including with a number of light tanks. Now that's what I call protecting civilians.

They are also about to take over the GSM network in western Libya like they did in the east. How do I know? I'm subscribed to the Telecom Tigers group on LinkedIn and so I get job adverts like these two.

ZTE BSC Job: URGENT send cv at [e-mail] for the job position or fw to your friends : Expert Telecom Engineer ZTE BSC.Location:Lybia,Western Area,1300USD/day,start immediate

URGENT send cv at [e-mail] for the job position or fw to your friends : ERICSSON MGW/BSS/BSC 2G/RAN Implementation Senior Expert Engineer.Location:Lybia,Gherian,Western Mountains,1300-1500 USD/day

In fact, one of the ads explicitly says that the job is in the rebel zone and the other is clear enough. What the rebels are planning to do is clear from the job descriptions:

must be able to install a ZTE latest generation BSC - platform to be integrated with 3rd party switching platform,solid knowledge of ZTE BSC build out and commissioning to connect up to 200 existing 2G/3G sites

To put it another way, they want to unhook the existing BTSs - the base stations - from Libyana and link them to a core system of their own, and in order to do this they need to install some Chinese-made Base Station Controllers (BSCs - the intermediary between the radio base stations and the central SS7 switch in GSM).

Here's the blurb for the Ericsson post:

Responsible for commissioning and integrating an Ericsson 2G BSS network (2048-TRX Ericsson BSC plus Ericsson BTSs) in a multi-vendor environment. Will be responsible for taking the lead and ownership of all BSS commissioning and integration, leading the local team of BSS engineers, and managing the team through to completion of integration.

Experience of Ericsson MGW implementation, and integration of MGW with BSS, is highly desirable. Experience of optical transmission over A-interface.

Compilation, creation and coordination of BSC Datafill. This will include creating, generating, seeking and gathering of all Datafill components (Transport, RF Frequencies, neighbor relations, handovers, Switch parameters, ABIS mapping, etc.) based on experience and from examination of existing network configuration and data. Loading of Datafill into the BSC to facilitate BTS integration.

Working with the MSC specialists to integrate the BSC with the MSC. Providing integration support to BTS field teams; providing configuration and commissioning support to the BSC field team.

So they've got some Ericsson BSCs, the base stations are Ericsson too, and an MSC (Mobile Switching Centre, the core voice switch) has been found from somewhere - interesting that they don't say who made it. That'll be the "3rd party switching platform" referred to in the first job. They're doing VoIP at some point, though, because they need a media gateway (MGW) to translate between traditional SS7 and SIP. They need engineers to integrate it all and to work out what the various configurations should be by studying what Gadhafi's guys left. (It's actually fairly typical that a mobile network consists of four or so different manufacturers' kit, which keeps a lot of people in pies dealing with the inevitable implementation quirks.)

The successful candidate will also have some soft skills, too:

Willing to work flexible hours, excellent interpersonal skills and the ability to work under pressure in a challenging, diverse and dynamic environment with a variety of people and cultures.

You can say that again. Apparently, security is provided for anyone who's up for the rate, which doesn't include full board and expenses, also promised.

They already have at least one candidate.

MGIs for cleaner skies, and Power Tool of the Week

Swinging off a discussion at Jamie Kenny's of climate deniers, I wonder what Jamie thinks about Steve Levine's thesis here that China's emerging culture of mass protest, the famous Mass-Group Incidents or MGIs, may have major and positive consequences for Chinese energy policy and therefore for the world.

It's surely time we started calling the MGIs a movement; they are big, they are angry, they are common and increasingly so. Also, they seem to be getting more simultaneous as well as more frequent. The range of issues involved is enormous, from pay to police violence via public corruption and land appropriation. And they're effective - the Chinese Communist Party, although it has more than enough brute force to crush them, often seems to semi-tolerate mass protests by trimming policy or sacking discredited officials. I've suggested before that the top level of the Party may actually see them as a useful force in disciplining the industrial bosses and territorial proconsuls who rule below it. The emperor may be far and the mountains may be high, but that's the last thing you want when an enraged mob is trying to burn down the Public Security Bureau offices.

Beyond that, it's conventional to say that the Party wants stability above all and that the organising principle of Chinese politics is Hobbesian fear of chaos. JK would probably point out that they're damn right - if you had China's history, you'd be obsessed by chaos because there's been so much of it and it was so fucking chaotic. Anyway, Jamie is the blogosphere's MGI expert and therefore I'd like his opinion.

Levine's argument is that forecasts of China's economic and energy future tend to arrive at an enormous and prolonged boom in coal-fired generation. They do this by projecting current rates of growth into the future. This scares the shit out of everyone with any sense, as it's this huge, epochal belch of CO2 (and a lot of other stuff besides) that will eventually fuck us all up. Of course, if the CAGRs for coal consumption were wrong quite a few assumptions would need to be reviewed.

Levine argues that it's the other stuff you get with coal, especially the low grade brown coal China uses a lot of, that will intervene. Basically, he reckons, air pollution, power-plant development, and mining will become a major and rising source of serious MGIs and will result in the Party restraining the coal industry before the mob does it for them. L

Levine points out that Chinese interests were quite restrained during last year's rush of coal-related mergers and acquisitions - which is interesting when you think that if the Party wanted them to, they could bid almost without limit thanks to SAFE's enormous foreign exchange reserves.

Further, and I seem to remember James Hansen making this point, there are real constraints on how much coal the Chinese economy can get through, in that moving that much coal from mines and ports to power stations will fairly soon use up most of the State Railways' freight capacity. As most of this coal is going to drive the machine tools in all those export processing factories...well, either the bulk haul trainload of coal moves or the intermodal linertrain of containers of exports moves. Are you feeling lucky, punk? Building a completely new railway is of course the sort of thing that gets people in an MGI mood.

From a technocratic perspective, as Joe Romm explains here, restrictions on all the other stuff coal-fired power stations shit into the atmosphere are basically as good as a ban on them.

The question is therefore whether "green MGIs" are a serious possibility. It's not actually necessary that the MGIs be specifically about what Greenpeace would call a green issue, of course. Rioting over pay or safety down the mines, over ethnic resentment in the coalfields, or over land appropriation for new power stations or railway lines would do as well. But it's worth noting that environmental protests happened in the 1980s in Eastern Europe and the Soviet Union and acted as a sort of gateway drug to dissidence more broadly. Not that people who are willing to burn down the police headquarters and run the mayor out of town when they feel their interests are insufficiently recognised need one.

Relatedly, and also via LeVine, meet the Unitec Model 5 pneumatic hacksaw, guaranteed by the manufacturer to slice through a 24" pipeline in one blow and only 16lbs dead weight to tote away from the scene of the crime. And it's nothing but good American workmanship, too. Mesh wireless is so pre-Iraq by comparison, don't you think?

meet Project Lobster

My lobbying project has been entered in the Open Data Challenge! Someone posted this to the MySociety list, with rather fewer than the advertised 36 hours left. I was at a wedding and didn't read it at the time. After my partner and I had tried to invent a tap routine to the back end of Prince's "Alphabet Street" and had got up at 8am to make it for the sadistic bed & breakfast breakfast and gone back to help clean up and drink any unaccountably unconsumed champagne, and the only thing left to look forward to was the end of the day, I remembered the message and noted that I had to get it filed before midnight.

So it was filed in the Apps category - there's an Ideas category but that struck me as pathetic, and after all there is some running code. I pushed on to try and get something out under the Visualisation category but ManyEyes was a bit broken that evening and anyway its network diagram view starts to suck after a thousand or so vertices.

As a result, the project now has a name and I have some thin chance of snagging an actual Big Society cheque for a few thousand euros and a trip to Brussels. (You've got to take the rough with the smooth.)

The most recent experiment with the Lobster Project - see, it's got a name! It's got you its grips before you're born...it lets you think you're king when you're really a prawn...whoops, wrong shellfish - was to try out a new centrality metric, networkx.algorithms.centrality.betweenness_centrality. This is defined as the fraction of the shortest paths between all the pairs of nodes in the network that pass through a given node. As you have probably guessed, this is quite an inefficient metric to compute and the T1700 lappy took over a minute to crunch it compared to 7 seconds to complete the processing script without it. Perhaps the new KillPad would do better but the difference is big enough that it's obviously my fault.



As far as I can see, though, it's also not very useful. The results are correlated (R^2 = 0.64) with the infinitely faster weighted graph degree. (It also confirms that Francis Maude is the secret ruler of the world, though.)

The NX functions I'm really interested in, though, are the ones for clique discovery and blockmodelling. It's obvious that with getting on for 3,000 links and more to come, any visualisation is going to need a lot of reduction. Blockmodelling basically chops your network into groups of nodes you provide and aggregates the links between those groups - it's one way, for example, to get department level results.

But I'd be really interested to use empirical clique discovery to feed into blockmodelling - the API for the one generates a python list of cliques, which are themselves lists of nodes, and the other accepts a list of nodes or a list of lists (of nodes). Another interesting option might be to blockmodel by edge attribute, which would be a way of deriving results for the content of meetings via the "Purpose of meeting" field. However, that would require creating a list of unique meeting subjects and then iterating over it creating lists of nodes with at least one edge having that subject, and then shoving the resulting list-of-lists into the blockmodeller.

That's a lorra lorra iteratin' by anybody's standards, even if, this being Python, most of it will end up being rolled up in a couple of seriously convoluted list comps. Oddly enough, it would be far easier in a query language or an ORM, but I've not heard of anything that lets you do SQL queries against a NX graph.

Having got this far, I notice that I've managed to blog my enthusiasm back up.

Anyway, I think it's perhaps time for a meetup on this next week with Who's Rob-bying.

lobbyists lobbying for lobbying

Best lobby metrics (lobbylyzer? lobster, for LoBbying Social Topology ExploreR?) result yet. Today I implemented my gatekeeper vs. flakcatcher metric - it averages the edge weights of all the neighbours of a minister, and returns a ratio of the difference between this and the average weight and the difference between the minister's weighting and the average. The principle is that if you lobby a minister, and then get access to another, your lobbying effort should gain or lose impact depending on the difference between the minister's own importance and the average. In the null hypothesis, where no minister is better or worse as a lobbying target than predicted by their department and their individual rank, you'd get the difference between the minister's weighting and the average.

If some ministers are gatekeepers, though, you would see a greater boost to your efforts at influence than the null case. Similarly, if some of them are flak catchers who mainly exist to turn away lobbying efforts, and you happened on one of those, you'd get a lesser boost. This metric should be greater than unity if the minister is a gatekeeper, 1 if they are perfectly mediocre, and less than unity if they are a flak catcher.

Interestingly, the Scotland and Wales Offices score highly. The highest value recorded is for David Jones MP, Parliamentary Under-Secretary of State for Wales, with 2.75. His closest rival is David Mundell MP, from the Scotland Office, on 1.94. The highest scoring Cabinet minister is the Scottish Secretary, Michael Moore. Some of this is down to the magic of low expectations. Nobody thinks these departments are great offices of state, not even the Welsh or the Scots - real power there has long since shifted to the devolved administrations. So if you meet any other minister, you're likely to do better. But the gatekeeper metric should handle this, as it measures influence relative to the structural difference between the minister and the average weighting. Arguably, this is a valid measurement. These ministries' role really is as a gatekeeper, something like a diplomatic representation in both directions.

So, who's the all-UK champ? It turns out to be Mark Harper MP, Minister for Political and Constitutional Reform, with an impressive 1.65, contrasting with his network degree of 0.08. Hilariously, one of his lobbies turns out to be the UK Public Affairs Council, the lobbyists' trade union, which wanted to see him in July, 2010 on the pressing matter of "lobbying".

He's followed home by Education PUSS Tim Loughton MP on 1.228, Defence PUSS Lord Astor on 1.2, Education's Lord Hill on 1.1, Justice PUSS Crispin Blunt MP with 1.09, and the sinister intellectual force that is Oliver Letwin MP, Minister of State for Government Policy, on 1.05. Francis Maude, whose horrific rise to national influence has been tracked with interest, turns out to be quite the flak catcher, on 0.43 - or is it that he claims to be a figure of authority in his own right? Does the buck stop there? After all, and as expected, once you meet the prime minister you can't really go anywhere but down.

Of course, what everyone will surely want to know is who gets the wooden spoon. Step forward Andrew Stunell MP, PUSS in the Department for Communities and Local Government, with a mighty 0.21 to go with his network degree of 0.3. Stunell held a large number of meetings around the country, notably in London, Bristol, and Bradford, as "Big Society Roundtables" with a wide range of community organisations. It would appear that nobody was more shortchanged than these. Meeting Mr. Stunell reduced one's average lobbying impact by a smacking 80%. Such was the coalition's contempt for, among other organisations, Operation Black Vote, the Stephen Lawrence Trust, and basically everyone in Bradford who showed up. The list is here.

It will surprise nobody that meeting a coalition minister would increase the UK Public Affairs Council's members' influence by 65%, but cut that of council tenants, Muslims, blacks, single mothers, young people (to list just a few) by 80%. But it's worth making it hideously explicit. And here's a lesson from all this obscure science that is easy enough to operationalise: if you see Andrew Stunell coming towards you through the Strangers' Bar with a smile on his face, don't make eye contact, don't shake hands, don't offer him your business card. Run. Spill a pint. Create a diversion. Trigger the fire alarm. Do not, in any circumstances, lobby him.

Here's the really sad bit. Stunell's dance card, from TWFY.

Voted very strongly against introducing foundation hospitals.
Voted strongly against Labour's anti-terrorism laws.
Voted very strongly against the Iraq war.
Voted very strongly for an investigation into the Iraq war.
Voted moderately against allowing ministers to intervene in inquests.
Voted a mixture of for and against greater autonomy for schools.
Voted moderately against replacing Trident.
Voted very strongly for the hunting ban.
Voted moderately for more EU integration.
Voted very strongly against introducing ID cards.
Voted very strongly for laws to stop climate change.
Voted very strongly against a stricter asylum system.
Voted moderately for removing hereditary peers from the House of Lords.
Voted strongly for a wholly elected House of Lords.
Voted very strongly for equal gay rights.
Voted moderately for a transparent Parliament.

I remember the Lib Dems. Do you? I wonder if Andrew Stunell remembers Andrew Stunell.

Routine maintenance

After this post and the outstanding response to it, I've just been working on the lobby project's underpinnings, specifically to backport some data cleaning from the analyser script into the original scraper, and to fix the one-edge-per-row version of the scraper. As a result I've had to flush the datastore and also search out some URIs that have changed. So far we've recreated 931 out of 1,721 meetings, although we're getting the dreaded "Execution status: run interrupted by a timeout". Actually, we've got 1,233 meetings back, and we've got rid of some crap. Anyone wanting the dataset can get it from the Scraperwiki API here or here for linkwise rather than meetingwise (coming soon) as either json-dict or csv. A full SQL syntax is available.

With luck, there will also be some more data quite soon. On the analysis score, notably, this and also this seem useful. The first estimates the value of a node based on its edges, which is fundamentally what I'm trying to achieve, and the second finds the cliques in the network a given node belongs to.

Regarding visualisation issues, I think one of my mistakes last time out was to visualise the data as a multi-graph - i.e. a structure with zero or more links between each node, permitting the existence of multiple links between the same pair of nodes. This invariably means a lot of links. The nature of the data - multiple meetings are absolutely central to the whole project, and lobbies meet ministers at different times and on different issues - enforces an underlying multigraph structure. But it would be possible to condense it for visualisation purposes - if we rolled up all links between the same nodes into one, we could tot up their weights and perhaps show that in the visualisation, as a thicker line for example.

lobby: update

I'm beginning to make some progress with the lobbying project. Last week I got it spitting out data; in mid-week, I optimised the process of loading the meetings from the ScraperWiki API into NetworkX. Hint: the obj_hook keyword argument in python's json.load() function is really useful!

This weekend it's producing information about lobbies, ministers, and government departments. I've got implementations nearly ready for a couple more dimensions of data - providing each actor's network degree by month, and trying to measure the extent to which ministers act as gatekeepers or flak-catchers. The first of those involves reimplementing a bit of NetworkX - you can't ask for node properties excluding certain edges by attribute, or at least you can't do so without creating a new subgraph, which seems ugly. The second, at the moment, counts the edges of a node if they have a higher weight than that of the node itself and expresses the sum of those edges' weights as a percentage of the total meetings that minister had. That doesn't take any account of time, yet.

I'm thinking of using Google App Engine to deploy it, running the data generator as a cron job and using the bulk uploader utility to slurp the results.

As a taster, the biggest single private interest lobbying Government is Barclays Bank, followed by Shell, the World Bank, the London Stock Exchange, BP, RBS, BAE, Standard Chartered, Lloyds, and Ratan Tata. This may not be that surprising. Neither is it very surprising, if somehow comforting in an old-fashioned way, that the two biggest lobbies of all are the Confederation of British Industry and the TUC, which is achieving about two-thirds the lobbying effort of the CBI and about twice that of Barclays. I was surprised to find that lobby 26 is Facebook, above Tesco, Microsoft, or UNISON. (Google is far, far down the list.) The highest placed individual trade union is the CWU at 24, between HSBC and the Electoral Commission. The littlest lobby is a nursery school in Leeds that got herded into a Big Society meeting with Nick Hurd MP.

I'm not so sure about using this model to assess the ministers, as we're using a priori weightings on them. But the decision to lobby a given minister must contain some information about the lobbyist's perceptions of their power and influence. Britain's most lobbied minister is Chris Grayling MP, Minister of State for Employment, who achieves a weighted degree of 4.2, not far off twice the prime minister. David Willetts, Vince Cable, Nick Clegg, and Francis Maude are the next four before the prime minister. They range between 2.8 and 2.6 with the PM on 2.3. Britain's least influential minister appears to be Baroness Warsi, minister without portfolio, on a score of 0.057.

BIS is the most lobbied department on 12.42, followed by the Department for Work and Pensions on 9.65, the Treasury on 7.065, the Cabinet Office on 5.62, and the DCLG on 3.825. Delight to the econophysicists (are they still around?): the distributions seem to show a nice power-law relationship! Which tells us what precisely? Well....not much except that it's a social network and they usually have them!

There were 2,073 nodes, either ministers or lobbies, in the graph at the last data upload. 2,848 interactions between them were analysed.

Does anyone have any ideas for other metrics that might be interesting?

of course, Gauck tells me my file is probably in Moscow

Well, if you're the prime minister, you're not allowed to fight back against your enemies in case you win because Gus O'Donnell says so, as long as the enemies aren't the right kind of enemies like Coronation Street editors and UNISON hospital porters. All clear so far? When are the Tory apologies to O'Donnell going to show up - they thought he was biased against them....

Further, interesting subplot - Brown told Peter Mandelson his phone might have been compromised. Mandelson asked the Information Commissioner, who had nothing to say. Of course he didn't - as far as I know he had nothing to do with the case. If Mandelson had wanted to know, he needed to ask the police. Was Mandelson trying not to find out, so as not to burn his bridges with the Murdochs? Or was he just ill-informed?

Also, did the central government have any communications security at all? Did CESG or MI5 not have anything at all to say about this? Didn't any of them just change their damn password, or even change their damn number?

I cannot begin to theorise what may have caused such a catastrophic malfunction

So, a free, jetlagged afternoon by the pool in Palo Alto, after this experience. What to do? Obviously, hack on some code. I dragged out the lobby analyzer project and got it to actually spit out ministers, lobbyists, and MPs, with their weighted degrees in the network, onto the command line. The conclusions are dreadful and confirm all my preliminary work. We are being ruled by Francis Maude and David Willetts. They both have significantly higher scores than the Prime Minister, with Willetts topping the poll. Of course, this is using the idea the wrong way up, but presumably the lobbyists' choice of who to lobby contains information about their perception of ministerial importance and influence.

Self-binding note: lobby metrics

Things to get out of the data in this scraper of mine: for each lobby, the monthly meeting counts, degrees in the weighted multigraph, impact factor (i.e. graph degree/meetings to give an idea of productivity), most met ministers, most met departments, topics. For each ministry, meeting counts, most met lobbies, most discussed topics. For each PR agency (Who's Lobbying had or has a list of clients for some of them), the same metrics as for lobbies. Summary dashboard: top lobbies, top lobbyists, top topics, graph visualisation, top 10 rising and falling lobbies by impact.

Things I'd like to have but aren't sure how to implement: a metric of gatekeeper-ness for ministers, for example, how often a lobby met a more powerful minister after meeting this one, and its inverse, a metric of how many low-value meetings a minister had. I've already done some scripting for this, and NetworkX will happily produce most of the numbers, although the search for an ideal charting solution goes on. Generating the graph and subgraphs is computationally expensive, so I'm thinking of doing this when the data gets loaded up and storing the results, rather than doing the sums at runtime.

Where's that Django tutorial? Unfortunately it's 7.05 pm on Sunday and it's looking unlikely I'll do it this weekend...

a patron, Sir?

Sensible piece about US State Department funding for mobile anonymity projects, and some interesting stuff. The crack about looking with disfavour on the drowning man and then encumbering him with help once he reaches ground is relevant.

The real prize (as alluded to here) would be a mesh network application that works either instead of the PLMN or alongside it. The only way to avoid leaving traces in the enormous billing/rating/charging infrastructure of your average cellular network is not to use it. According to Comptel, the Finnish OSS/BSS software house, operators spend about €32bn a year on software, of which €11.5bn is in the revenue management segment, another €5.5bn in business analytics, and another €4bn in CRM - €21bn worth of data-mangling kit that could theoretically be repurposed. It's probably better to just leave a GPRS datacall than a phone call to the person you want to speak to in there, though.

On the other hand, there's an API for the US Army.

Fun with nonsense and hash functions

Churnalism is a brilliant idea - no surprise that it was originally one of Chris Lightfoot's. Basically, it allows you to determine how much of a given newspaper article was copied from which press release. There's a nice graphic visualisation, and a diff, so you can see precisely what was altered and what taken over in its entirety. It's right up there with Piggipedia and SukeyDating as a brilliant piece of geek activism.

However, here's something amusing. There's a basic API here; I chucked the text of the GSMA final press release from this year's MWC at it, and I was quite surprised at the results. The first article it extracted from Journalisted was none other than this piece in the Guardian from...February 2008. One consequence of churnalism is that your newspaper is likely to get repetitive.

As far as I can see, if there's anything missing here it's that the comparison is mostly the wrong way - having a newspaper article and wanting to know what vacuous NIB-fodder got regurgitated into it is a much more common use-case than having a press release and wanting to know which newspaper articles it got into. Actually, the latter use-case is far more likely if you're a PR and you're trying to measure how the talking-points are spreading. But once it has more press releases on file, it'll work better in that sense. And that's just a question of hoovering Businesswire, PRNewsWire etc up.

i haz bin in yr AR standardz, facilitatin yr interop. kthx!

So I had the opportunity to take part in an Augmented Reality standardisation meeting on the fringe of this year's 3GSM Mobile World Congress. First of all, it was the year the heavens opened (someone on twitter said it was as if the show had turned into Glastonbury) and I got drenched and my shoes went bad, and my cab didn't take me to the Telefonica R&D building in Via Augusta but instead to the main switching centre, this amazingly domineering building...

Telcos - they live in places like this, they know where your dog goes to school, but can they tell you if it's really your bank on the line?

So I got soaked again, and eventually arrived, and spent the first session listening to my shoes rotting. I acted as scribe for the session on AR browser implementations, markup language vs. JSON, native application vs. browser plugin and the like. I hope I contributed something of value. I have a Flickr set of the annotated flip charts here; I've been asked to help prepare the final report. Which just goes to show the enduring truth that if you want to influence something, wait until the very end and sum up with a balanced account. Supposedly this used to be the way to pass the Diplomatic Service exams - buy a pipe, puff on it occasionally during the team exercise, then "sum up with a balanced account". But you're not allowed to smoke these days.