So we've looked at how they're dreadful and why. The stakes are important; a huge chunk of the economy is made up of services, and some of the places where they are located are becoming almost as much one-industry towns as they were before their one industry shut down. What if this sector was as productive and as valued as Rolls-Royce? (Especially as, all things considered, it is quite difficult to use them as a weapon of war, rather as the role of the orchestra in counter-insurgency is limited at best.)
We have the technology. Ticketing systems are as mature as anything gets, and a reader of this blog was moved to say that every software developer has at least once tried to write their own. Web-voice integration is a hugely creative field at the moment. Things like Fonolo and the Networked Helpdesk Protocol (API docs are here) show what can be done.
But the big issue is management, and I think expectations. People expect the experience to be terrible. People expect the job to be status-reducing and generally horrible. People expect that because it's a cost-centre, there's no way to improve it other than flogging the slaves harder.
Blogging a noisy and socialistic view on politics, security, and whatever may take my fancy. "All the world now is in the Ranting humour" - Samuel Sheppard, 1647
Saturday, January 21, 2012
The politics of call centres, part two: sources of failure
So, why did we get here? Back in the mists of time, in the US Bell System, there used to be something called a Business Office, by contrast to a Central Office (i.e. what we call a BT Local Exchange in the UK), whose features and functions were set down in numerous Bell System Practice documents. Basically, it was a site where the phone company took calls from the public, either for its own account or on behalf of a third party. Its practices were defined by Bell System standardisation, and its industrial relations were defined by the agreement between AT&T and the unions, which specified the pay and conditions for the various trades and workplace types inside the monster telco. If something was a Business Office according to the book, the union agreement covering those offices would apply.
In the Reaganite 80s, after the Bell System was broken up, someone realised that it would be possible to get rid of the union rules if they could re-define the site as something else. Not only could they change the rules, but they could move the site physically to a right-to-work state or even outside the USA. This is, it turns out, the origin of the phrase "call centre".
In the UK, of course, call centres proliferated in parallel with utility privatisation and financial deregulation. A major element in the business case for privatisation was getting rid of all those electricity showrooms and BT local offices and centralising customer service functions into `all centres. At the same time, of course, privatisation created the demand for customer service in that it was suddenly possible to change provider and therefore to generate a shit-load of admin. Banks were keen to get rid of their branches and to serve the hugely expanding credit card market. At another level, IT helpdesks made their appearance.
On the other hand, hard though it is to imagine it now, there was a broader vision of technology that expected it all to be provided centrally - in the cloud, if you will - down phone lines controlled by your favourite telco, or by the French Government, or perhaps Rupert Murdoch. This is one of the futures that didn't happen, of course, because PCs and the web happened instead, but you can bet I spent a lot of time listening to people as late as the mid-2000s still talking about multimedia services (and there are those who argue this is what stiffed Symbian). But we do get a sneak-preview of the digital future that Serious People wanted us to have, every time we have to ring the call centre. In many ways, call centres are the Anti-Web.
In Britain, starting in the 1990s, they were also part of the package of urban regeneration in the North. Along with your iconic eurobox apartments and AutoCAD-shaped arts centre, yup, you could expect to find a couple of gigantic decorated sheds full of striplighting and the precariat. Hey, he's like a stocky, Yorkshire Owen Hatherley. After all, it was fairly widely accepted that even if you pressed the button marked Arts and the money rolled in, there was a limit to the supply of yuppies and there had to be some jobs in there as well.
You would be amazed at the degree of boosterism certain Yorkshire councils developed on this score, although you didn't need top futurist Popcorn Whatsname to work out that booming submarine cable capacity would pretty quickly make offshoring an option. Still, if Bradford didn't make half-arsed attempts to jump on every bandwagon going, leaving it cluttered with vaguely Sicilian failed boondoggles, it wouldn't be Bradford.
Anyway, I think I've made a case that this is an institution whose history has been pathological right from the start. It embodies a fantasy of managing a service industry in the way the US automakers were doing at the same time - and failing, catastrophically.
In the Reaganite 80s, after the Bell System was broken up, someone realised that it would be possible to get rid of the union rules if they could re-define the site as something else. Not only could they change the rules, but they could move the site physically to a right-to-work state or even outside the USA. This is, it turns out, the origin of the phrase "call centre".
In the UK, of course, call centres proliferated in parallel with utility privatisation and financial deregulation. A major element in the business case for privatisation was getting rid of all those electricity showrooms and BT local offices and centralising customer service functions into `all centres. At the same time, of course, privatisation created the demand for customer service in that it was suddenly possible to change provider and therefore to generate a shit-load of admin. Banks were keen to get rid of their branches and to serve the hugely expanding credit card market. At another level, IT helpdesks made their appearance.
On the other hand, hard though it is to imagine it now, there was a broader vision of technology that expected it all to be provided centrally - in the cloud, if you will - down phone lines controlled by your favourite telco, or by the French Government, or perhaps Rupert Murdoch. This is one of the futures that didn't happen, of course, because PCs and the web happened instead, but you can bet I spent a lot of time listening to people as late as the mid-2000s still talking about multimedia services (and there are those who argue this is what stiffed Symbian). But we do get a sneak-preview of the digital future that Serious People wanted us to have, every time we have to ring the call centre. In many ways, call centres are the Anti-Web.
In Britain, starting in the 1990s, they were also part of the package of urban regeneration in the North. Along with your iconic eurobox apartments and AutoCAD-shaped arts centre, yup, you could expect to find a couple of gigantic decorated sheds full of striplighting and the precariat. Hey, he's like a stocky, Yorkshire Owen Hatherley. After all, it was fairly widely accepted that even if you pressed the button marked Arts and the money rolled in, there was a limit to the supply of yuppies and there had to be some jobs in there as well.
You would be amazed at the degree of boosterism certain Yorkshire councils developed on this score, although you didn't need top futurist Popcorn Whatsname to work out that booming submarine cable capacity would pretty quickly make offshoring an option. Still, if Bradford didn't make half-arsed attempts to jump on every bandwagon going, leaving it cluttered with vaguely Sicilian failed boondoggles, it wouldn't be Bradford.
Anyway, I think I've made a case that this is an institution whose history has been pathological right from the start. It embodies a fantasy of managing a service industry in the way the US automakers were doing at the same time - and failing, catastrophically.
The politics of call centres, part one
What is it that makes call centres so uniquely awful as social institutions? This is something I've often touched on at Telco 2.0, and also something that's been unusually salient in my life recently - I moved house, and therefore had to interact with getting on for a dozen of the things, several repeatedly. (Vodafone and Thames Water were the best, npower and Virgin Media the worst.) But this isn't just going to be a consumer whine. In an economy that is over 70% services, the combination of service design, technology, and social relations that makes these things so awful is something we need to understand.
For example, why does E.ON (the electricity company, a branch of the German utility Rhein-Westfälische Elektrizitätswerke) want you to tell their IVR what class you are before they do anything else? This may sound paranoid, but when I called them, the first question I had to answer was whether I owned my home or was a tenant. What on earth did they want to know that for?
Call centres provide a horrible experience to the user. They are famously awful workplaces. And they are also hideously inefficient - some sites experience levels of failure demand, that is to say calls generated due to a prior failure to serve, over 50% of the total inbound calls. Manufacturing industry has long recognised that rework is the greatest enemy of productivity, taking up disproportionate amounts of time and resources and inevitably never quite fixing the problems.
So why are they so awful? Well, I'll get to that in the next post. Before we can answer that, we need to think about how they are so awful. I've made a list of anti-patterns - common or standard practices that embody error - that make me angry.
Our first anti-pattern is queueing. Call centres essentially all work on the basis of oversubscription and queueing. On the assumption that some percentage of calls will go away, they save on staff by queueing calls. This is not the only way to deal with peaks in demand, though - for example, rather than holding calls, there is no good technical reason why you couldn't instead have a call-back architecture, scheduling a call back sometime in the future.
Waiting on hold is interesting because it represents an imposition on the user - because telephony is a hot medium in McLuhan's terminology, your attention is demanded while you sit pointlessly in the queue. In essence, you're providing unpaid labour. Worse, companies are always tempted to impose on you while you wait - playing music on hold (does anybody actually like this?), or worse, nagging you about using the web site. We will see later on that this is especially pointless and stupid.
And the existence of the queue is important in the social relations of the workplace. If there are people queueing, it is obviously essential to get to them as soon as possible, which means there is a permanent pressure to speed up the line. Many centres use the queue as an operational KPI. It is also quality-destroying, in that both workers and managers' attention is always focused on the next call and how to get off the current call in order to get after the queue.
A related issue is polling. That is to say, repeatedly checking on something, rather than being informed pro-actively when it changes. This is of course implicit in the queueing model. It represents a waste of time for everyone involved.
Repetition is one of the most annoying of the anti-patterns, and it is caused by statelessness. It is always assumed that this interaction has never happened before, will never happen again, and is purely atomised. They don't know what happened in the last call, or even earlier in the call if it has been transferred. As a result, you have to provide your mother's maiden name and your account number, again, and they have to retype it, again. The decontextualised nature of interaction with a call centre is one of the worst things about it.
Pretty much every phone system these days uses SIP internally, so there is no excuse for not setting a header with a unique identifier that could be used to look up data in all the systems involved, and indeed given out as a ticket number to the user in case they need to call again, or - why not - used to share the record of the call.
That point leads us to another very important one. Assymetric legibility characterises call centres, and it's dreadful. Within, management tries to maintain a panopticon glare at the staff. Without, the user faces an unmapped territory, in which the paths are deliberately obscure, and the details the centre holds on you are kept secret. Call centres know a lot about you, but won't say; their managers endlessly spy on the galley slaves; you're not allowed to know how the system works.
So no wonder we get failure demand, in which people keep coming back because it was so awful last time. A few companies get this, and use first-call resolution (the percentage of cases that are closed first time) as a KPI rather than call rates, but you'd be surprised. Obviously, first-call resolution has a whole string of social implications - it requires re-skilling of the workforce and devolution of authority to them. No wonder it's rare.
Now, while we were in the queue, the robot voice kept telling us to bugger off and try the Web site. But this is futile. Inappropriate automation and human/machine confusion bedevil call centres. If you could solve your problem by filling in a web form, you probably would have done. The fact you're in the queue is evidence that your request is complicated, that something has gone wrong, or generally that human intervention is required.
However, exactly this flexibility and devolution of authority is what call centres try to design out of their processes and impose on their employees. The product is not valued, therefore it is awful. The job is not valued by the employer, and therefore, it is awful. And, I would add, it is not valued by society at large and therefore, nobody cares.
So, there's the how. Now for the why.
For example, why does E.ON (the electricity company, a branch of the German utility Rhein-Westfälische Elektrizitätswerke) want you to tell their IVR what class you are before they do anything else? This may sound paranoid, but when I called them, the first question I had to answer was whether I owned my home or was a tenant. What on earth did they want to know that for?
Call centres provide a horrible experience to the user. They are famously awful workplaces. And they are also hideously inefficient - some sites experience levels of failure demand, that is to say calls generated due to a prior failure to serve, over 50% of the total inbound calls. Manufacturing industry has long recognised that rework is the greatest enemy of productivity, taking up disproportionate amounts of time and resources and inevitably never quite fixing the problems.
So why are they so awful? Well, I'll get to that in the next post. Before we can answer that, we need to think about how they are so awful. I've made a list of anti-patterns - common or standard practices that embody error - that make me angry.
Our first anti-pattern is queueing. Call centres essentially all work on the basis of oversubscription and queueing. On the assumption that some percentage of calls will go away, they save on staff by queueing calls. This is not the only way to deal with peaks in demand, though - for example, rather than holding calls, there is no good technical reason why you couldn't instead have a call-back architecture, scheduling a call back sometime in the future.
Waiting on hold is interesting because it represents an imposition on the user - because telephony is a hot medium in McLuhan's terminology, your attention is demanded while you sit pointlessly in the queue. In essence, you're providing unpaid labour. Worse, companies are always tempted to impose on you while you wait - playing music on hold (does anybody actually like this?), or worse, nagging you about using the web site. We will see later on that this is especially pointless and stupid.
And the existence of the queue is important in the social relations of the workplace. If there are people queueing, it is obviously essential to get to them as soon as possible, which means there is a permanent pressure to speed up the line. Many centres use the queue as an operational KPI. It is also quality-destroying, in that both workers and managers' attention is always focused on the next call and how to get off the current call in order to get after the queue.
A related issue is polling. That is to say, repeatedly checking on something, rather than being informed pro-actively when it changes. This is of course implicit in the queueing model. It represents a waste of time for everyone involved.
Repetition is one of the most annoying of the anti-patterns, and it is caused by statelessness. It is always assumed that this interaction has never happened before, will never happen again, and is purely atomised. They don't know what happened in the last call, or even earlier in the call if it has been transferred. As a result, you have to provide your mother's maiden name and your account number, again, and they have to retype it, again. The decontextualised nature of interaction with a call centre is one of the worst things about it.
Pretty much every phone system these days uses SIP internally, so there is no excuse for not setting a header with a unique identifier that could be used to look up data in all the systems involved, and indeed given out as a ticket number to the user in case they need to call again, or - why not - used to share the record of the call.
That point leads us to another very important one. Assymetric legibility characterises call centres, and it's dreadful. Within, management tries to maintain a panopticon glare at the staff. Without, the user faces an unmapped territory, in which the paths are deliberately obscure, and the details the centre holds on you are kept secret. Call centres know a lot about you, but won't say; their managers endlessly spy on the galley slaves; you're not allowed to know how the system works.
So no wonder we get failure demand, in which people keep coming back because it was so awful last time. A few companies get this, and use first-call resolution (the percentage of cases that are closed first time) as a KPI rather than call rates, but you'd be surprised. Obviously, first-call resolution has a whole string of social implications - it requires re-skilling of the workforce and devolution of authority to them. No wonder it's rare.
Now, while we were in the queue, the robot voice kept telling us to bugger off and try the Web site. But this is futile. Inappropriate automation and human/machine confusion bedevil call centres. If you could solve your problem by filling in a web form, you probably would have done. The fact you're in the queue is evidence that your request is complicated, that something has gone wrong, or generally that human intervention is required.
However, exactly this flexibility and devolution of authority is what call centres try to design out of their processes and impose on their employees. The product is not valued, therefore it is awful. The job is not valued by the employer, and therefore, it is awful. And, I would add, it is not valued by society at large and therefore, nobody cares.
So, there's the how. Now for the why.
Labels:
callcentre,
class,
dayjob,
engineering,
history,
managerialism,
politics,
socialism,
special relationships,
surveillance
Sunday, January 15, 2012
lazyweb: old budget forecasts
Dear Lazyweb, has anyone seen a data series showing the forecast for the UK government budget? Or will I have to download all the Treasury statements and re-chew it?
links...
Quick-hit update to the Baluchistan/US/Iran post; Daniel Drezner has a crack at rounding up the news and comes pretty close to arguing that the Americans are trying to stop the Israelis getting them into a war with Iran. Akbar Ahmed argues, in a must-read, that things in Baluchistan have been getting much worse lately and that this is very bad news for Pakistan, and it's all the government's fault. And US-Israel anti-missile live fire exercise gets called off.
A quick look back to the riots
Reading through tehgrauniad's riots deep-dive, the impression that I get is that the whole "riots as an insurgency" idea wasn't that far off. I've been indisciplined in that I took notes but didn't keep links (a problem with paying for and reading the actual newspaper), so you'll have to trust me on this. Obviously, blaming the whole thing on "criminality" is about as useful as blaming rain on "water falling from the sky".
The first common factor that struck me was that pretty much everyone they interviewed had a grudge against the police. Not in any broad theoretical sense, but a grudge - a specific and personal memory of perceived injustice and especially incivility, cherished over time. Now, it's in the nature of policing as a public service that nobody enjoys it. If you're interacting with policemen on duty, it's either because they suspect you of being a criminal, or because something bad has happened to you. Generally, everybody would quite like to minimise their lifetime consumption of policing.
There is something that motivates people to put up with it, though, and that something is legitimacy.
The second common factor was the attitude towards property. Quite a lot of the people the Guardian spoke to reported looting goods from shops, and then giving them away, or witnessing others doing so. Stealing goods is one thing, but immediately giving them away is rather different and very much a political act. So much so that there is a word for it (and I'm not the only one to notice this).
Of course, police legitimacy comes in a very large degree from their role as protectors of property, so this was a way of directly challenging their claim to provide security and to employ legitimate force.
Eyewitnesses often described a tactical, practical implementation of this - small groups of rioters harassing the police, in a sort of screening or covering operation, while many more looted or destroyed property. It's very interesting that this could all happen so quickly.
The first common factor that struck me was that pretty much everyone they interviewed had a grudge against the police. Not in any broad theoretical sense, but a grudge - a specific and personal memory of perceived injustice and especially incivility, cherished over time. Now, it's in the nature of policing as a public service that nobody enjoys it. If you're interacting with policemen on duty, it's either because they suspect you of being a criminal, or because something bad has happened to you. Generally, everybody would quite like to minimise their lifetime consumption of policing.
There is something that motivates people to put up with it, though, and that something is legitimacy.
The second common factor was the attitude towards property. Quite a lot of the people the Guardian spoke to reported looting goods from shops, and then giving them away, or witnessing others doing so. Stealing goods is one thing, but immediately giving them away is rather different and very much a political act. So much so that there is a word for it (and I'm not the only one to notice this).
Of course, police legitimacy comes in a very large degree from their role as protectors of property, so this was a way of directly challenging their claim to provide security and to employ legitimate force.
Eyewitnesses often described a tactical, practical implementation of this - small groups of rioters harassing the police, in a sort of screening or covering operation, while many more looted or destroyed property. It's very interesting that this could all happen so quickly.
404
Following up on the earlier post about IMSI catchers and shopping malls and Hezbollah, I wanted to link to a really excellent piece in Le Monde about mining call-detail records ("fadettes" in French, from "facture détaillée téléphonique"). The URI, here now leads to an annoyingly cutesy 404 page. However, the search function turns it up and even shows it as being free...but the link it returns doesn't work.
Jack Straw, still repellent after all these years
How much of a bastard was Jack Straw again? This much.
Scotland Yard has opened a criminal investigation into secret MI6 rendition operations that resulted in leading Libyan dissidents being abducted and flown to Tripoli where they were subsequently tortured in Muammar Gaddafi's prisons....The year after the joint UK-Libyan operations were mounted, Straw told MPs they must disbelieve allegations of UK involvement in rendition "unless we all start to believe in conspiracy theories and that the officials are lying, that I am lying, that behind this there is some kind of secret state which is in league with some dark forces in the United States".
a short telegram, or a very long tweet
Everyone's linked to Mark Perry (of Conflicts Forum/Alistair Crooke fame)'s piece on Israeli spooks running around Baluchistan posing as the CIA already, but I will too as it's very interesting indeed. I'm not sure what their bag in this is, other than the notion of "always escalate" and hope to profit from the general confusion.
But what's really interesting is what the story is doing out there now. Here's Laura Rozen's write-up, which introduces the suggestion that they may have represented themselves as being from NATO and notes that a leader of the organisation said as much on Iranian TV before being executed. Meanwhile, the Iranians write to the Americans accusing the CIA of being behind the assassination of another nuclear scientist.
On Twitter, she suggests that the scientist wasn't killed by the Americans (i.e. presumptively by the Israelis, or by people working for them wittingly or otherwise), and that this was staged specifically to queer the possibility of reviving the Iran-Turkey uranium swap deal. (You do wonder what George F. Kennan would have made of diplomatic tweeting.) Further, we know that a back-channel has been set up.
Disclosing information about the Israeli operation in Baluchistan might be a smart way of establishing trust between the US and Iran. Obviously, information about terrorists running about blowing stuff up and killing people is of value to Iran. Information that it's the Israelis is obviously congenial to Iran. Crucially, burning an Israeli spy network is costly to the Americans and not something they would do lightly (the Perry piece is a monument to important people trying all they could to do nothing). In that sense, it is a meaningful signal - much more convincing than mere words. Presumably, Perry's role at Conflicts Forum and with Arafat makes him a convincing postman into the bargain. And third-party spies are just the sort of thing that enemies can bond over. I recall reading about the IRA and the UVF staging a joint investigation to find informers in the early 1970s.
Another dose of speculation - if Baluch rebels were meeting with people who they thought were from NATO, was this plausible because NATO was in fact paying them off to leave the Karachi-Quetta-Kandahar supply route alone?
But what's really interesting is what the story is doing out there now. Here's Laura Rozen's write-up, which introduces the suggestion that they may have represented themselves as being from NATO and notes that a leader of the organisation said as much on Iranian TV before being executed. Meanwhile, the Iranians write to the Americans accusing the CIA of being behind the assassination of another nuclear scientist.
On Twitter, she suggests that the scientist wasn't killed by the Americans (i.e. presumptively by the Israelis, or by people working for them wittingly or otherwise), and that this was staged specifically to queer the possibility of reviving the Iran-Turkey uranium swap deal. (You do wonder what George F. Kennan would have made of diplomatic tweeting.) Further, we know that a back-channel has been set up.
Disclosing information about the Israeli operation in Baluchistan might be a smart way of establishing trust between the US and Iran. Obviously, information about terrorists running about blowing stuff up and killing people is of value to Iran. Information that it's the Israelis is obviously congenial to Iran. Crucially, burning an Israeli spy network is costly to the Americans and not something they would do lightly (the Perry piece is a monument to important people trying all they could to do nothing). In that sense, it is a meaningful signal - much more convincing than mere words. Presumably, Perry's role at Conflicts Forum and with Arafat makes him a convincing postman into the bargain. And third-party spies are just the sort of thing that enemies can bond over. I recall reading about the IRA and the UVF staging a joint investigation to find informers in the early 1970s.
Another dose of speculation - if Baluch rebels were meeting with people who they thought were from NATO, was this plausible because NATO was in fact paying them off to leave the Karachi-Quetta-Kandahar supply route alone?
The intersection of electronic warfare and mall management
Here's something interesting. You may remember this story from back in November about the CIA spy network in Lebanon that met at a Pizza Hut they codenamed PIZZA, and which was rolled up by a joint Hezbollah-Lebanese military intelligence investigation. The key detail is as follows:
If the top paragraph is true, it would have been catastrophically ill-advised. Even somebody special, like a CIA agent under diplomatic cover, has a relatively large number of weak ties to normal people. This is the reverse of the small-world principle, and is a consequence of the fact that the great majority of people are real human beings rather than important persons. As a result, things like STELLAR WIND, the illegal Bush-era effort to analyse the whole pile of call-detail records at AT&T and Verizon in the hope that this would find terrorists, face a sort of Bayesian doom. We've gone over this over and over again.
However, phone numbers that only talk to special people are obviously suspicious. Most numbers with a neighbourhood length of 1 will be things like machine-to-machine SIMs in vending machines and cash points, but once you'd filtered those out, the remaining pool of possibles would be quite small. It is intuitive to think of avoiding surveillance, or keeping a low profile, but what is required is actually camouflage rather than concealment.
There are more direct methods - which is where electronic warfare and shopping mall management intersect.
Path Intelligence, a Portsmouth-based startup, will install a network of IMSI-catchers, devices which act as a mobile base station in order to identify mobile phones nearby, in your shopping centre so as to collect really detailed footfall information.
Similarly, you could plant such a device near that Pizza Hut to capture which phones passed by and when, and which ones usually coincided. Alternatively, you could use it in a targeted mode to confirm the presence or absence of a known device. Which makes me wonder about the famous Hezbollah telecoms network, and whether it was intended at least in part to be an electronic-intelligence network - as after all, nothing would be a better cover for a huge network of fake mobile base stations than a network of real ones.
Meanwhile, this year's CCC (like last year's) was just stuffed with GSM exploits. It really is beginning to look a lot like "time we retired that network".
U.S. officials also denied the source's allegation that the former CIA station chief dismissed an email warning that some of his Lebanese agents could be identified because they used cellphones to call only their CIA handlers and no one else.
...
Lebanon's security service was able to isolate the CIA informants by analyzing cellphone company records that showed the numbers called, duration of each call and location of the phone at the time of the call, the source said.
Using billing and cell tower records for hundreds of thousands of phone numbers, software can isolate cellphones used near an embassy, or used only once, or only on quick calls. The process quickly narrows down a small group of phones that a security service can monitor.
If the top paragraph is true, it would have been catastrophically ill-advised. Even somebody special, like a CIA agent under diplomatic cover, has a relatively large number of weak ties to normal people. This is the reverse of the small-world principle, and is a consequence of the fact that the great majority of people are real human beings rather than important persons. As a result, things like STELLAR WIND, the illegal Bush-era effort to analyse the whole pile of call-detail records at AT&T and Verizon in the hope that this would find terrorists, face a sort of Bayesian doom. We've gone over this over and over again.
However, phone numbers that only talk to special people are obviously suspicious. Most numbers with a neighbourhood length of 1 will be things like machine-to-machine SIMs in vending machines and cash points, but once you'd filtered those out, the remaining pool of possibles would be quite small. It is intuitive to think of avoiding surveillance, or keeping a low profile, but what is required is actually camouflage rather than concealment.
There are more direct methods - which is where electronic warfare and shopping mall management intersect.
Path Intelligence, a Portsmouth-based startup, will install a network of IMSI-catchers, devices which act as a mobile base station in order to identify mobile phones nearby, in your shopping centre so as to collect really detailed footfall information.
Similarly, you could plant such a device near that Pizza Hut to capture which phones passed by and when, and which ones usually coincided. Alternatively, you could use it in a targeted mode to confirm the presence or absence of a known device. Which makes me wonder about the famous Hezbollah telecoms network, and whether it was intended at least in part to be an electronic-intelligence network - as after all, nothing would be a better cover for a huge network of fake mobile base stations than a network of real ones.
Meanwhile, this year's CCC (like last year's) was just stuffed with GSM exploits. It really is beginning to look a lot like "time we retired that network".
Labels:
GSM,
hacker,
Hezbollah,
intelligence and stupidity,
lobbying,
networks,
surveillance
Wednesday, January 11, 2012
Konsidered a waste of time
OK, so I eventually finished listening to the 793 songs in the 2011 SXSW torrent and rating them all. This was a while ago, but it was only yesterday that I reorganised some stuff in the collection and remembered that the couple of gigabytes of mediocrity was sitting there. It was clearly time to implement the TYR Band-Pass filter, my objective methodology for filtering musical slushpiles.
So I frobbed around Amarok until I found the "Automated Playlist Generator" hiding under a rock, and then fiddled with it until I understood the UI-only-a-hacker-could-imagine. Seriously, it would have been easier to just provide a command prompt on the underlying database. (Does a "Match All" Constraint Group match both any rules of its own and also the output of a "Match Any"? Search me, guv, because you can search your hard disk and not find any documentation.)
And it gave me 32 tracks, all with a rating of zero. Now that is a valid output from the filter. Or it would be if there were no tracks rated above the upper limit, 3.5. And I gave out quite a few 5s. So I check in the pile. All the ratings are gone. This isn't quite as bad as the phase KAddressBook and Akonadi went through a couple of years ago when they regularly, randomly, truncated my contacts file from 269KB to 10.8KB - always exactly the same - and inserted helpful invalid characters. (Fortunately they also left a renamed copy of the original file, so you could just restore from backup.) But it's pretty shit. Any software that randomly destroys user data has failed and failed horribly. It's the antithesis of polite software.
But it did produce 32 tracks, so there must be a wrong copy of the data somewhere, which suggests that there might also be a right one.
Meanwhile, I've been reading the traffic on kdepim-l about KMail 2 with horror and an increasing sense that KDE is going spongy. Even without anything related to Akonadi actually working, long after the last lot of performance bugs were closed, it still has a nasty habit of keeping the hard disk active for half an hour at a time, doing what? KM2 users report rampant loss of data and of meta-data. And I don't have a working desktop search utility despite years of promises about Akonadi and Nepomuk and Strigi and "semantic desktop".
Think about it like this - a new era KDE application that needs to read data from your contacts file, a vCard sitting somewhere in your .kde4 directory, is meant to go to an "akonadi_vcal_resource" that's mediated by the common Akonadi API and no less than two RDF triplestore databases (Redland and Virtuoso). What happened to the filesystem?
So, I'm going to initiate a new, innocent laptop into the twisted cult this week. And I think I'm quitting the KDE world. I'm not the only one - from 25 killer Linux apps to When you first launch KMail,
Well, I've never had that error but my install crashes every time it launches, and only ever works on the second time of asking. Of course, I could spend all my time maintaining this particular e-mail client. Don't all write at once.
So I frobbed around Amarok until I found the "Automated Playlist Generator" hiding under a rock, and then fiddled with it until I understood the UI-only-a-hacker-could-imagine. Seriously, it would have been easier to just provide a command prompt on the underlying database. (Does a "Match All" Constraint Group match both any rules of its own and also the output of a "Match Any"? Search me, guv, because you can search your hard disk and not find any documentation.)
And it gave me 32 tracks, all with a rating of zero. Now that is a valid output from the filter. Or it would be if there were no tracks rated above the upper limit, 3.5. And I gave out quite a few 5s. So I check in the pile. All the ratings are gone. This isn't quite as bad as the phase KAddressBook and Akonadi went through a couple of years ago when they regularly, randomly, truncated my contacts file from 269KB to 10.8KB - always exactly the same - and inserted helpful invalid characters. (Fortunately they also left a renamed copy of the original file, so you could just restore from backup.) But it's pretty shit. Any software that randomly destroys user data has failed and failed horribly. It's the antithesis of polite software.
But it did produce 32 tracks, so there must be a wrong copy of the data somewhere, which suggests that there might also be a right one.
Meanwhile, I've been reading the traffic on kdepim-l about KMail 2 with horror and an increasing sense that KDE is going spongy. Even without anything related to Akonadi actually working, long after the last lot of performance bugs were closed, it still has a nasty habit of keeping the hard disk active for half an hour at a time, doing what? KM2 users report rampant loss of data and of meta-data. And I don't have a working desktop search utility despite years of promises about Akonadi and Nepomuk and Strigi and "semantic desktop".
Think about it like this - a new era KDE application that needs to read data from your contacts file, a vCard sitting somewhere in your .kde4 directory, is meant to go to an "akonadi_vcal_resource" that's mediated by the common Akonadi API and no less than two RDF triplestore databases (Redland and Virtuoso). What happened to the filesystem?
So, I'm going to initiate a new, innocent laptop into the twisted cult this week. And I think I'm quitting the KDE world. I'm not the only one - from 25 killer Linux apps to When you first launch KMail,
it will terminate with a 'Failed to fetch the resource collection' error. KMail doesn't have a default incoming mail directory configured, which causes this error. The workaround involves using Akonadi to specify a maildir location for KMail. To do this, launch the Akonadi Configuration tool and point the Local Folders to /.kde4/share/apps/kmail2/.
Well, I've never had that error but my install crashes every time it launches, and only ever works on the second time of asking. Of course, I could spend all my time maintaining this particular e-mail client. Don't all write at once.
Sunday, January 08, 2012
RQ-170 upshot, part 2: the bubble
Is there a drone bubble? It's not clear whether this is more like the .com bubble, when a lot of useful stuff was built but a couple of years too early, or more like the housing bubble, when a lot of stuff was built in the wrong places to the wrong standards at the wrong prices and will probably never be worth much. It's the nature of a bubble, of course, that it's precisely at the top of the bubble that the commitment to it is greatest.
One of the things the RQ-170 incident tells us about is some of the operational limitations of the drones. Typically, they are piloted in the cruise from locations that may be a long way off, using satellite communication links, but when they land, they do so under local control via line-of-sight radio link from their base. This allows us to set some bounds on how much of a problem link latency really is, which will take us circling back to John Robb's South Korean gamers.
Gamers are famous for being obsessed with ping-times - the measurement of round-trip latency on the Internet - because it's really, really annoying to see the other guy on your screen, go to zap'em, and get zapped yourself because it took longer for your zap to cross the Internet than theirs. Typically you can expect 40 or so milliseconds nationally, 60-80 inter-continentally...or several hundred if a satellite or an old-school cellular operator with a hierarchical network architecture is involved. A sat hop is always clearly identifiable in traceroute output because latency goes to several hundred ms, and there's a great RIPE NCC paper on using the variations in latency over a year to identify the satellite's geosynchronous (rather than geostationary) orbit as the slant-range changes.
On the other hand, roundtrip latency across an airfield circuit a couple of miles wide will be negligible. So we can conclude that tolerable latency for manoeuvring, as opposed to cruising, is very little. Now, check out this post on David Cenciotti's blog from January 2010. Some of the Israeli air force's F-15s have received a new communications radio suite specifically for controlling UAVs.
You might now be able to guess why even drone pilots are going through basic flight training. Also, this post of Cenciotti's describes the causes of six recent hull losses, all of which are classic airmanship accidents - the sort of thing pilot training is designed to teach you to avoid.
That said, why did all those drones get built? The original, 1980s UAV concepts were usually about the fact that there was no pilot and therefore the craft could be treated as expendable, usually in order to gain intelligence on the (presumably) Soviet enemy's air defences by acting as a ferret aircraft, forcing them to switch on the radars so the drone could identify them. But that's not what they've been doing all these years.
The main reason for using them has been that they are lightweight and have long endurance. This is obviously important from an intelligence gathering perspective, whether you're thinking of over-watching road convoys or of assassinating suspected terrorists (and there are strong arguments against that, as Joshua Foust points out). In fact, long endurance and good sensors are so important that there are even so-called manned drones - diesel-engined, piloted light aircraft stuffed with sensors, with the special feature that they fly with intelligence specialists aboard and provide a much faster turn-around of information for the army.
Their limitations - restricted manoeuvre, limited speed and payload, and high dependence on communications infrastructure - haven't really been important because they have been operating in places and against enemies who don't have an air force or ground-based air defences and don't have an electronic warfare capability either. Where the enemy have had man-portable SAMs available, as sometimes in Iraq, they have chosen to save them for transport aircraft and the chance of killing Americans, which makes sense if anti-aircraft weapons are scarce (and surely, the fact of their scarcity has to be one of the major unreported news stories of the decade).
But then, the war in Iraq is meant to be over even if the drones are still landing in Kurdistan, and the US may be on its way to a "pre-1990" military posture in the Gulf. This week's strategic fashion is "Air-Sea Battle" and the Pacific, and nobody expects anything but the most hostile possible environment in the air and in the electromagnetic spectrum. And the RQ-170 incident is surely a straw in the wind. Also, the Bush wars were fought in an environment of huge airfields in the desert, and the ASB planners expect that the capacity of US bases in Japan and Guam and the decks of aircraft carriers will be their key logistical constraint. (The Russians aren't betting everything on them either.)
I think, therefore, it's fair to suggest that a lot of big drones are going to end up in the AMARC stockpile. After the Americans' last major counter-insurgency, of course, that's what happened. The low-tech ones are likely to keep proliferating, though, whether as part of the Royal Engineers' route clearance system or annoying the hell out of Japanese whalers or even playing with lego.
One of the things the RQ-170 incident tells us about is some of the operational limitations of the drones. Typically, they are piloted in the cruise from locations that may be a long way off, using satellite communication links, but when they land, they do so under local control via line-of-sight radio link from their base. This allows us to set some bounds on how much of a problem link latency really is, which will take us circling back to John Robb's South Korean gamers.
Gamers are famous for being obsessed with ping-times - the measurement of round-trip latency on the Internet - because it's really, really annoying to see the other guy on your screen, go to zap'em, and get zapped yourself because it took longer for your zap to cross the Internet than theirs. Typically you can expect 40 or so milliseconds nationally, 60-80 inter-continentally...or several hundred if a satellite or an old-school cellular operator with a hierarchical network architecture is involved. A sat hop is always clearly identifiable in traceroute output because latency goes to several hundred ms, and there's a great RIPE NCC paper on using the variations in latency over a year to identify the satellite's geosynchronous (rather than geostationary) orbit as the slant-range changes.
On the other hand, roundtrip latency across an airfield circuit a couple of miles wide will be negligible. So we can conclude that tolerable latency for manoeuvring, as opposed to cruising, is very little. Now, check out this post on David Cenciotti's blog from January 2010. Some of the Israeli air force's F-15s have received a new communications radio suite specifically for controlling UAVs.
You might now be able to guess why even drone pilots are going through basic flight training. Also, this post of Cenciotti's describes the causes of six recent hull losses, all of which are classic airmanship accidents - the sort of thing pilot training is designed to teach you to avoid.
That said, why did all those drones get built? The original, 1980s UAV concepts were usually about the fact that there was no pilot and therefore the craft could be treated as expendable, usually in order to gain intelligence on the (presumably) Soviet enemy's air defences by acting as a ferret aircraft, forcing them to switch on the radars so the drone could identify them. But that's not what they've been doing all these years.
The main reason for using them has been that they are lightweight and have long endurance. This is obviously important from an intelligence gathering perspective, whether you're thinking of over-watching road convoys or of assassinating suspected terrorists (and there are strong arguments against that, as Joshua Foust points out). In fact, long endurance and good sensors are so important that there are even so-called manned drones - diesel-engined, piloted light aircraft stuffed with sensors, with the special feature that they fly with intelligence specialists aboard and provide a much faster turn-around of information for the army.
Their limitations - restricted manoeuvre, limited speed and payload, and high dependence on communications infrastructure - haven't really been important because they have been operating in places and against enemies who don't have an air force or ground-based air defences and don't have an electronic warfare capability either. Where the enemy have had man-portable SAMs available, as sometimes in Iraq, they have chosen to save them for transport aircraft and the chance of killing Americans, which makes sense if anti-aircraft weapons are scarce (and surely, the fact of their scarcity has to be one of the major unreported news stories of the decade).
But then, the war in Iraq is meant to be over even if the drones are still landing in Kurdistan, and the US may be on its way to a "pre-1990" military posture in the Gulf. This week's strategic fashion is "Air-Sea Battle" and the Pacific, and nobody expects anything but the most hostile possible environment in the air and in the electromagnetic spectrum. And the RQ-170 incident is surely a straw in the wind. Also, the Bush wars were fought in an environment of huge airfields in the desert, and the ASB planners expect that the capacity of US bases in Japan and Guam and the decks of aircraft carriers will be their key logistical constraint. (The Russians aren't betting everything on them either.)
I think, therefore, it's fair to suggest that a lot of big drones are going to end up in the AMARC stockpile. After the Americans' last major counter-insurgency, of course, that's what happened. The low-tech ones are likely to keep proliferating, though, whether as part of the Royal Engineers' route clearance system or annoying the hell out of Japanese whalers or even playing with lego.
Labels:
4GW,
Afghanistan,
aviation,
GPS,
logistics,
special relationships,
strategy,
stupid procurement,
surveillance,
UAV,
US
The RQ-170 hack and the drone bubble
The fact that a majority of this year's graduates from USAF basic pilot training are assigned to drone squadrons has got quite a bit of play in the blogosphere. Here, via Jamie Kenny, John Robb (who may still be burying money for fear of Obama or may not) argues that the reason they still do an initial flight training course is so that the pilot-heavy USAF hierarchy can maintain its hold on the institution. He instead wants to recruit South Korean gamers, in his usual faintly trendy dad way. Jamie adds the snark and suggests setting up a call centre in Salford.
On the other hand, before Christmas, the Iranians caught an RQ-170 intelligence/reconnaissance drone. Although the RQ-170 is reportedly meant to be at least partly stealthy, numerous reports suggest that the CIA was using it among other things to get live video of suspected nuclear sites. This seems to be a very common use case for drones, which usually have a long endurance in the air and can be risked remaining over the target for hours on end, if the surveillance doesn't have to be covert.
Obviously, live video means that a radio transmitter has to be active 100% of the time. It's also been reported that one of the RQ-170's main sensors is a synthetic-aperture radar. Just as obviously, using radar involves transmitting lots of radio energy.
It is possible to make a radio transmitter less obvious, for example by saving up information and sending it in infrequent bursts, and by making the transmissions as directional as possible, which also requires less power and reduces the zone in which it is possible to detect the transmission. However, the nature of the message governs its form. Live video can't be burst-transmitted because it wouldn't be live. Similarly, real-time control signalling for the drone itself has to be instant, although engineering telemetry and the like could be saved and sent later, or only sent on request. And the need to keep a directional antenna pointing precisely at the satellite sets limits on the drone's manoeuvring. None of this really works for a mapping radar, though, which by definition needs to sweep a radio beam across its field of view.
Even if it was difficult to acquire it on radar, then, it would have been very possible to detect and track the RQ-170 passively, by listening to its radio emissions. And it would have been much easier to get a radar detection with the advantage of knowing where to look.
There has been a lot of speculation about how they then attacked it. The most likely scenario suggests that they jammed the command link, forcing the drone to follow a pre-programmed routine for what to do if the link is lost. It might, for example, be required to circle a given location and wait for instructions, or even to set a course for somewhere near home, hold, and wait for the ground station to acquire them in line-of-sight mode.
Either way, it would use GPS to find its way, and it seems likely that the Iranians broadcast a fake GPS signal for it. Clive "Scary Commenter" Robinson explains how to go about spoofing GPS in some detail in Bruce Schneier's comments, and points out that the hardware involved is cheap and available.
Although the military version would require you to break the encryption in order to prepare your own GPS signal, it's possible that the Iranians either jammed it and forced the drone to fall back on the civilian GPS signal, and spoofed that, or else picked up the real signal at the location they wanted to spoof and re-broadcast it somewhere else, an attack known as "meaconing" during the second world war when the RAF Y-Service did it to German radio navigation. We would now call it a replay attack with a fairly small time window. (In fact, it's still called meaconing.) Because GPS is based on timing, there would be a limit to how far off course they could put it this way without either producing impossible data or messages that failed the crypto validation, but this is a question of degree.
It's been suggested that Russian hackers have a valid exploit of the RSA cipher, although the credibility of this suggestion is unknown.
The last link is from Charlie Stross, who basically outlined a conceptual GPS-spoofing attack in my old Enetation comments back in 2006, as a way of subverting Alistair Darling's national road-pricing scheme.
Anyway, whether they cracked the RSA key or forced a roll-back to the cleartext GPS signal or replayed the real GPS signal from somewhere else, I think we can all agree it was a pretty neat trick. But what is the upshot? In the next post, I'm going to have a go at that...
On the other hand, before Christmas, the Iranians caught an RQ-170 intelligence/reconnaissance drone. Although the RQ-170 is reportedly meant to be at least partly stealthy, numerous reports suggest that the CIA was using it among other things to get live video of suspected nuclear sites. This seems to be a very common use case for drones, which usually have a long endurance in the air and can be risked remaining over the target for hours on end, if the surveillance doesn't have to be covert.
Obviously, live video means that a radio transmitter has to be active 100% of the time. It's also been reported that one of the RQ-170's main sensors is a synthetic-aperture radar. Just as obviously, using radar involves transmitting lots of radio energy.
It is possible to make a radio transmitter less obvious, for example by saving up information and sending it in infrequent bursts, and by making the transmissions as directional as possible, which also requires less power and reduces the zone in which it is possible to detect the transmission. However, the nature of the message governs its form. Live video can't be burst-transmitted because it wouldn't be live. Similarly, real-time control signalling for the drone itself has to be instant, although engineering telemetry and the like could be saved and sent later, or only sent on request. And the need to keep a directional antenna pointing precisely at the satellite sets limits on the drone's manoeuvring. None of this really works for a mapping radar, though, which by definition needs to sweep a radio beam across its field of view.
Even if it was difficult to acquire it on radar, then, it would have been very possible to detect and track the RQ-170 passively, by listening to its radio emissions. And it would have been much easier to get a radar detection with the advantage of knowing where to look.
There has been a lot of speculation about how they then attacked it. The most likely scenario suggests that they jammed the command link, forcing the drone to follow a pre-programmed routine for what to do if the link is lost. It might, for example, be required to circle a given location and wait for instructions, or even to set a course for somewhere near home, hold, and wait for the ground station to acquire them in line-of-sight mode.
Either way, it would use GPS to find its way, and it seems likely that the Iranians broadcast a fake GPS signal for it. Clive "Scary Commenter" Robinson explains how to go about spoofing GPS in some detail in Bruce Schneier's comments, and points out that the hardware involved is cheap and available.
Although the military version would require you to break the encryption in order to prepare your own GPS signal, it's possible that the Iranians either jammed it and forced the drone to fall back on the civilian GPS signal, and spoofed that, or else picked up the real signal at the location they wanted to spoof and re-broadcast it somewhere else, an attack known as "meaconing" during the second world war when the RAF Y-Service did it to German radio navigation. We would now call it a replay attack with a fairly small time window. (In fact, it's still called meaconing.) Because GPS is based on timing, there would be a limit to how far off course they could put it this way without either producing impossible data or messages that failed the crypto validation, but this is a question of degree.
It's been suggested that Russian hackers have a valid exploit of the RSA cipher, although the credibility of this suggestion is unknown.
The last link is from Charlie Stross, who basically outlined a conceptual GPS-spoofing attack in my old Enetation comments back in 2006, as a way of subverting Alistair Darling's national road-pricing scheme.
Anyway, whether they cracked the RSA key or forced a roll-back to the cleartext GPS signal or replayed the real GPS signal from somewhere else, I think we can all agree it was a pretty neat trick. But what is the upshot? In the next post, I'm going to have a go at that...
Labels:
4GW,
Afghanistan,
electronics,
engineering,
GPS,
hacker,
Iran,
radar,
RAF,
surveillance,
UAV,
US
Subscribe to:
Posts (Atom)