Category Archives: Security economics

Social-science angles of security

When safety and security become one

Academic papers, Legal issues, Open-source security, Operating systems, Security economics, Security engineering

What happens when your car starts getting monthly upgrades like your phone and your laptop? It’s starting to happen, and the changes will be profound. We’ll be able to improve car safety as we learn from accidents, and fixing a flaw won’t mean spending billions on a recall. But if you’re writing navigation code today that will go in the 2020 Landrover, how will you be able to ship safety and security patches in 2030? In 2040? In 2050? At present we struggle to keep software patched for three years; we have no idea how to do it for 30.

Our latest paper reports a project that Éireann Leverett, Richard Clayton and I undertook for the European Commission into what happens to safety in this brave new world. Europe is the world’s lead safety regulator for about a dozen industry sectors, of which we studied three: road transport, medical devices and the electricity industry.

Up till now, we’ve known how to make two kinds of fairly secure system. There’s the software in your phone or laptop which is complex and exposed to online attack, so has to be patched regularly as vulnerabilities are discovered. It’s typically abandoned after a few years as patching too many versions of software costs too much. The other kind is the software in safety-critical machinery which has tended to be stable, simple and thoroughly tested, and not exposed to the big bad Internet. As these two worlds collide, there will be some rather large waves.

Regulators who only thought in terms of safety will have to start thinking of security too. Safety engineers will have to learn adversarial thinking. Security engineers will have to think much more about ease of safe use. Educators will have to start teaching these subjects together. (I just expanded my introductory course on software engineering into one on software and security engineering.) And the policy debate will change too; people might vote for the FBI to have a golden master key to unlock your iPhone and read your private messages, but they might be less likely to vote them a master key to take over your car or your pacemaker.

Researchers and software developers will have to think seriously about how we can keep on patching the software in durable goods such as vehicles for thirty or forty years. It’s not acceptable to recycle cars after seven years, as greedy carmakers might hope; the embedded carbon cost of a car is about equal to its lifetime fuel burn, and reducing average mileage from 200,000 to 70,000 would treble the car industry’s CO2 emissions. So we’re going to have to learn how to make software sustainable. How do we do that?

Our paper is here; there’s a short video here and a longer video here. The full report is available from the EU here.

RIP smart meters

Legal issues, News coverage, Politics, Security economics

The Telegraph has just run an op-ed they asked me to write over the weekend, after I pointed out here on Friday that the Conservative manifesto had quietly downgraded the smart meter programme to a voluntary one.

Regular readers of Light Blue Touchpaper will have followed the smart meter story for almost a decade, back through the dishonest impact assessment to the fact that they pose a threat to critical infrastructure.

Manifestos and tech

Authentication, Cybercrime, Internet censorship, Legal issues, News coverage, Politics, Security economics

The papers went to town yesterday on the Conservative manifesto but missed some interesting bits.

First, no-one seems to have noticed that the smart meter programme is being quietly put to death. We read on page 60 that everyone will be offered a smart meter by 2020. So a mandatory national programme has become voluntary, just like that. Regular readers of this blog will recall that the programme was sold in 2008 by Ed Milliband using a dishonest impact assessment, yet all the parties backed it after 2010, leaving no-one to point out that it was going to cost us all a fortune and never save any carbon. May says she wants to reduce energy costs; this was surely a no-brainer.

That was the good news for England. The good news for friends in rural Scotland is high-speed broadband for all by 2020. But there are some rather weird things in there too.

What on earth is “the right of businesses to insist on a digital signature”? Digital signatures are very 1998, and we already have the electronic signature directive. From whom will businesses be able to insist on a signature, and if I’m one of the legislated victims, how much do I have to pay to buy the apparatus?

All digital businesses will have “to support new digital proofs of identification”. That presumably means forcing firms to use Verify, a dysfunctional online authentication service whose roots lie in Blair’s obsession with identity. If a newspaper currently identifies its subscribers via a proprietary logon, will they have to offer Verify as an option? Will it have to be the only option, displacing Facebook and Twitter? The manifesto also says that local government will have to use Verify; and elsewhere that councils must publish planning applications and bus routes “without the hassle and delay that currently exists.” OK, so some councils could so with more competent webmasters, but don’t worry: “hundreds of leaders from the world of tech can come into government to help deliver better public services.”

The Land Registry, the Ordnance Survey and other quangos that do geography (our leader’s degree subject) will all band together to create the largest open repository of land data in the world. So where will the Ordnance Survey get its money from then? That small question killed the same idea in 2010 after Tim Berners-Lee sold it to Cameron.

There will be a levy on social media companies, like on gambling companies, to support awareness and preventive activity. And they must not direct users, even unintentionally, to hate speech. So will Facebook be fined whenever they let users like a xenophobic article in the Daily Mail?

No doubt in view of the delicacy of such regulatory decisions, Leveson II is killed; there will be a Data Use and Ethics Commission instead. It will advise regulators and develop the principles and rules that will give people confidence their data are being handled properly. Wow. We now have the Charter of Fundamental Rights to give us principles, the GDPR to give us rules, and the ECJ to hammer out the case law. Now the People don’t have confidence in such experts we’re going to let the Prime Minister of the day appoint a different lot.

The next government will further strengthen cyber security standards for government and public services, so presumably all such services will have to use expensive networks such as the NHS-wide network from BT which will expect them to manage their own firewalls without telling them how to.

But don’t worry. It will become “as difficult to commit a crime digitally as it is physically”. There is text about working “with international law enforcement agencies to ensure perpetrators are brought to justice” but our local police force isn’t allowed to do anything effective about online accommodation fraud committed by a gang in Germany. They have to work through the NCA – who don’t care. The manifesto signals more of the same: the NCA will get to eat the SFO, which does crimes over £100m, leaving them even less interested in online crooks who steal a thousand pounds of deposit from dozens of students a year.

In fact there is no signal anywhere in the manifesto that May understands the impact of volume cybercrime, even though it’s now most of the property crime in the UK. She rather prefers to boast of the falling crime over the past seven years, as if it were her achievement as Home Secretary. The simple fact is that crime has been going online like everything else, and until 2015 the online part of it wasn’t recorded properly. This was not the doing of Theresa May, but of Margaret Hodge.

The manifesto rather seems to have been drafted in a geek-free room. And let’s not spoil the party by mentioning the impact that tight immigration targets will have on the IT industry, or for that matter on higher education. Perhaps they want us to hope that they don’t really mean that part of it, but perhaps we’d better make a plan to open a campus in India or Canada, just in case.

Bad malware, worse reporting

Cybercrime, Legal issues, News coverage, Security economics, Security psychology

The Wannacry malware that has infected some UK hospital computers should interest not just security researchers but also people interested in what drives fake news.

Some made errors of fact: the Daily Mail inititally reported the ransom demand as 300 bitcoin, or £415,000, rather than $300 in bitcoin. Others made errors of logic: the Indy, for example, reported that “Up to 90 percent of NHS computers still run XP, released in 2001”, citing as its source a BMJ article which stated that 90% of trusts run this version of Windows. And some made errors of concurrency. After dinner I found inquiries from journalists about my fight with the Prime Minister. My what? Eventually I found that the Guardian had followed something Mrs May’s spokesman had said (“not aware of any evidence that patient data has been compromised”) with something I’d said a couple of hours earlier (“The NHS are saying that patient privacy hasn’t been compromised, but if significant numbers of hospitals have been negligently running unpatched computers for two months after the patch came out, how do they know?”). The Home Secretary later helpfully glossed the PM’s stonewall as “No patient data has been accessed or transferred in any way” but leaving the get-out-of-jail card “that’s the information we’ve been given.”

Many papers caught the international political aspect: that the vulnerability was discovered by the NSA, kept secret rather than fixed (contrary to the advice of Obama’s NSA review group), then stolen from the CIA by the Russians and published via wikileaks. Scary stuff, eh? And we read of some surprising overreactions, such as the GP who switched off his networking as a precaution and found he couldn’t access any of his patients’ records.

As luck would have it, yesterday was the day that I gave my talk on entomology – the classification of software bugs and other security vulnerabilities – to my first-year security and software engineering class. So let’s try to look at it calmly as I’d expect of a student writing an assignment.

The first point is that there’s not a really lot of this malware. The NHS has over 200 hospitals, and the typical IT director is a senior clinician supported by technicians. Yet despite having their IT run by well-meaning amateurs, only 16 NHS organisations have been hit, according to the Register and Kaspersky – including several hospitals.

So the second point is that when the Indy says that “The NHS is a perfect combination of sensitive data and insecure storage. And there’s very little they can do about it” the answer is simple: in well over 90% of NHS organisations, the well-meaning amateurs managed perfectly well. What they did was to keep their systems patched up-to-date; simple hygiene, like washing your hands after going to the toilet.

The third takeaway is that it’s worth looking at the actual code. A UK researcher did so and discovered a kill switch.

Now I am just listening on the BBC morning news to a former deputy director of GCHQ who first cautions against alarmist headlines and argues that everyone develops malware; that a patch had been issued by Microsoft halfway through March; that you can deal with ransomware by keeping decent backups; and that paying ransom will embolden the bad guys. However he claims that it’s clearly an organised criminal attack. (when it could be one guy in his bedroom somewhere) and says that the NCSC should look at whether there is some countermeasure that everyone should have taken (for answer see above).

So our fourth takeaway is that although the details matter, so do the economics of security. When something unexpected happens, you should not just get your head down and look at the code, but look up and observe people’s agendas. Politicians duck and weave; NHS managers blame the system rather than step up to the plate; the NHS as a whole turns every incident into a plea for more money; the spooks want to avoid responsibility for the abuse of their stolen cyberweaponz, but still big up the threat and get more influence for a part of their agency that’s presented as solely defensive. And we academics? Hey, we just want the students to pay attention to what we’re teaching them.

Hope this helps!

Video on Edge

Banking security, Cybercrime, Internet censorship, Legal issues, News coverage, Politics, Security economics, Security engineering, Social networks

John Brockman of Edge interviewed me in London in March. The video of the interview, and a transcript, are now available on the Edge website. Edge runs big interviews with several dozen scientists a year, with particular interest in people who do cross-disciplinary work. For me, the interaction of economics, psychology and engineering is one of the things that makes security so fascinating, as well as the creativity driven by adversarial behaviour.

The topics covered include the last thirty years of progress (of lack of it) in information security, from the early beginnings, through the crypto wars and crime moving online, to the economics of security. We talked about how cryptography can help less developed countries; about managing complexity in big projects; about how network effects lead firms to design insecure products; about whether big data can undermine democracy by empowering elites; and about how in a future world of intelligent things, security may become more about safety than anything else. Finally I talk about our current big project, the Cambridge Cybercrime Centre.

John runs a literary agency, and he’s worked on books by many of the scientists who feature on his site. This makes me wonder: on what topic should I write my next book?

Banks biased against black fraud victims

Banking security, Legal issues, News coverage, Security economics

The following is an op-ed I wrote in today’s Times. It appeared in their Thunderer column.

You’re less likely to be treated fairly by your bank if you’re elderly, poor, female or black. We’ve suspected this for years, and finally The Times has dug up the numbers to prove it.

Fraud victims who’re refused compensation often contact our security research group at Cambridge after they find we work on payment fraud. We call this stream of complaints our ‘fraud telescope’ as it gives us early warning of what the bad guys are up to. We’ve had more than 2,000 cases over 25 years.

In recent years we’ve started to realise what we weren’t seeing. The “dark matter” in the fraud universe is the missing victims: we don’t see that many middle-class white men. The victims who do come to us are disproportionately elderly, poor, female, or black. But crime surveys tell us that the middle classes and the young are more likely to be victims of fraud, so it’s hard to avoid the conclusion that banks are less generous to some of their customers.

We raised the issue of discrimination in 2011 with one of the banks and with the Commission for Racial Equality, but as no-one was keeping records, nothing could be proved, until today.

How can this discrimination happen? Well, UK rules give banks a lot of discretion to decide whether to refund a victim, and the first responders often don’t know the full story. If your HSBC card was compromised by a skimmer on a Tesco ATM, there’s no guarantee that Tesco will have told anyone (unlike in America, where the law forces Tesco to tell you). And the fraud pattern might be something entirely new. So bank staff end up making judgement calls like “Is this customer telling the truth?” and “How much is their business worth to us?” This in turn sets the stage for biases and prejudices to kick in, however subconsciously. Add management pressure to cut costs, sometimes even bonuses for cutting them, and here we are.

There are two lessons. First, banks need to train staff to be aware of unconscious bias (as universities do), and monitor their performance.

Second, the Financial Conduct Authority needs to protect all customers properly. It seems to be moving in the right direction; after the recent fraud against tens of thousands of Tesco Bank account holders, it said it expected fraud victims to be made good immediately. This has been the law in the USA since the 1980s and it must become a firm rule here too.

Security Economics MOOC

Cybercrime, Politics, Security economics, Security psychology

In two weeks’ time we’re starting an open course in security economics. I’m teaching this together with Rainer Boehme, Tyler Moore, Michel van Eeten, Carlos Ganan, Sophie van der Zee and David Modic.

Over the past fifteen years, we’ve come to realise that many information security failures arise from poor incentives. If Alice guards a system while Bob pays the cost of failure, things can be expected to go wrong. Security economics is now an important research topic: you can’t design secure systems involving multiple principals if you can’t get the incentives right. And it goes way beyond computer science. Without understanding how incentives play out, you can’t expect to make decent policy on cybercrime, on consumer protection or indeed on protecting critical national infrastructure

We first did the course last year as a paid-for course with EdX. Our agreement with them was that they’d charge for it the first time, to recoup the production costs, and thereafter it would be free.

So here it is as a free course. Spread the word!