Camouflage or scary monsters: deceiving others about risk

I have just been at the Cambridge Risk and Uncertainty Conference which brings together people who educate the public about risks. They include public-health doctors trying to get people to eat better and exercise more, statisticians trying to keep governments honest about crime statistics, and climatologists trying to educate us about global warming – an eclectic and interesting bunch.

Most of the people in this community see their role as dispelling ignorance, or motivating the slothful. Yet in most of the cases we discussed, the public get risk wrong because powerful interests make a serious effort to scare them about some of life’s little hazards, or to reassure them about others. When this is put to the risk communication folks in a question – whether after a talk or in the corridor – they readily admit they’re up against a torrent of misleading marketing. But they don’t see what they’re doing as adversarial, and I strongly suspect that many risk interventions are less effective as a result.

In my talk (slides) I set this out as simply and starkly as I could. We spend too much on terrorism, because both the terrorists and the governments who’re supposed to protect us from them big up the threat; we spend too little on cybercrime, because everyone from the crooks through the police and the banks to the computer industry has their own reason to talk down the threat. I mentioned recent cases such as Wannacry as examples of how institutions communicate risk in self-serving, misleading ways. I discussed our own study of browser warnings, which suggests that people at least subconsciously know that most of the warnings they see are written to benefit others rather than them; they tune out all but the most specific.

What struck me with some force when preparing my talk, though, is that there’s just nobody in academia who takes a holistic view of adversarial risk communication. Many people look at some small part of the problem, from David Rios’ game-theoretic analysis of adversarial risk through John Mueller’s studies of terrorism risk and Alessandro Acquisti’s behavioural economics of privacy, through to criminologists who study pathways into crime and psychologists who study deception. Of all these, the literature on deception might be the most relevant, though we should also look at politics, propaganda, and studies of why people stubbornly persist in their beliefs – including the excellent work by Bénabou and Tirole on the value people place on belief. Perhaps the professionals whose job comes closest to adversarial risk communication are political spin doctors. So when should we talk about new facts, and when should we talk about who’s deceiving you and why?

Given the current concern over populism and the role of social media in the Brexit and Trump votes, it might be time for a more careful cross-disciplinary study of how we can change people’s minds about risk in the presence of smart and persistent adversaries. We know, for example, that a college education makes people much less susceptible to propaganda and marketing; but what is the science behind designing interventions that are quicker and cheaper in specific circumstances?

Second Annual Cybercrime Conference

The Cambridge Cybercrime Centre is organising another one day conference on cybercrime on Thursday, 13th July 2017.

In future years we intend to focus on research that has been carried out using datasets provided by the Cybercrime Centre, but just as last year (details here, liveblog here) we have a stellar group of invited speakers who are at the forefront of their fields:

They will present various aspects of cybercrime from the point of view of criminology, policy, security economics, law and policing.

This one day event, to be held in the Faculty of Law, University of Cambridge will follow immediately after (and will be in the same venue as) the “Tenth International Conference on Evidence Based Policing” organised by the Institute of Criminology which runs on the 11th and 12th July 2016.

Full details (and information about booking) is here.

When safety and security become one

What happens when your car starts getting monthly upgrades like your phone and your laptop? It’s starting to happen, and the changes will be profound. We’ll be able to improve car safety as we learn from accidents, and fixing a flaw won’t mean spending billions on a recall. But if you’re writing navigation code today that will go in the 2020 Landrover, how will you be able to ship safety and security patches in 2030? In 2040? In 2050? At present we struggle to keep software patched for three years; we have no idea how to do it for 30.

Our latest paper reports a project that Éireann Leverett, Richard Clayton and I undertook for the European Commission into what happens to safety in this brave new world. Europe is the world’s lead safety regulator for about a dozen industry sectors, of which we studied three: road transport, medical devices and the electricity industry.

Up till now, we’ve known how to make two kinds of fairly secure system. There’s the software in your phone or laptop which is complex and exposed to online attack, so has to be patched regularly as vulnerabilities are discovered. It’s typically abandoned after a few years as patching too many versions of software costs too much. The other kind is the software in safety-critical machinery which has tended to be stable, simple and thoroughly tested, and not exposed to the big bad Internet. As these two worlds collide, there will be some rather large waves.

Regulators who only thought in terms of safety will have to start thinking of security too. Safety engineers will have to learn adversarial thinking. Security engineers will have to think much more about ease of safe use. Educators will have to start teaching these subjects together. (I just expanded my introductory course on software engineering into one on software and security engineering.) And the policy debate will change too; people might vote for the FBI to have a golden master key to unlock your iPhone and read your private messages, but they might be less likely to vote them a master key to take over your car or your pacemaker.

Researchers and software developers will have to think seriously about how we can keep on patching the software in durable goods such as vehicles for thirty or forty years. It’s not acceptable to recycle cars after seven years, as greedy carmakers might hope; the embedded carbon cost of a car is about equal to its lifetime fuel burn, and reducing average mileage from 200,000 to 70,000 would treble the car industry’s CO2 emissions. So we’re going to have to learn how to make software sustainable. How do we do that?

Our paper is here; there’s a short video here and a longer video here. The full report is available from the EU here.

RIP smart meters

The Telegraph has just run an op-ed they asked me to write over the weekend, after I pointed out here on Friday that the Conservative manifesto had quietly downgraded the smart meter programme to a voluntary one.

Regular readers of Light Blue Touchpaper will have followed the smart meter story for almost a decade, back through the dishonest impact assessment to the fact that they pose a threat to critical infrastructure.

Manifestos and tech

The papers went to town yesterday on the Conservative manifesto but missed some interesting bits.

First, no-one seems to have noticed that the smart meter programme is being quietly put to death. We read on page 60 that everyone will be offered a smart meter by 2020. So a mandatory national programme has become voluntary, just like that. Regular readers of this blog will recall that the programme was sold in 2008 by Ed Milliband using a dishonest impact assessment, yet all the parties backed it after 2010, leaving no-one to point out that it was going to cost us all a fortune and never save any carbon. May says she wants to reduce energy costs; this was surely a no-brainer.

That was the good news for England. The good news for friends in rural Scotland is high-speed broadband for all by 2020. But there are some rather weird things in there too.

What on earth is “the right of businesses to insist on a digital signature”? Digital signatures are very 1998, and we already have the electronic signature directive. From whom will businesses be able to insist on a signature, and if I’m one of the legislated victims, how much do I have to pay to buy the apparatus?

All digital businesses will have “to support new digital proofs of identification”. That presumably means forcing firms to use Verify, a dysfunctional online authentication service whose roots lie in Blair’s obsession with identity. If a newspaper currently identifies its subscribers via a proprietary logon, will they have to offer Verify as an option? Will it have to be the only option, displacing Facebook and Twitter? The manifesto also says that local government will have to use Verify; and elsewhere that councils must publish planning applications and bus routes “without the hassle and delay that currently exists.” OK, so some councils could so with more competent webmasters, but don’t worry: “hundreds of leaders from the world of tech can come into government to help deliver better public services.”

The Land Registry, the Ordnance Survey and other quangos that do geography (our leader’s degree subject) will all band together to create the largest open repository of land data in the world. So where will the Ordnance Survey get its money from then? That small question killed the same idea in 2010 after Tim Berners-Lee sold it to Cameron.

There will be a levy on social media companies, like on gambling companies, to support awareness and preventive activity. And they must not direct users, even unintentionally, to hate speech. So will Facebook be fined whenever they let users like a xenophobic article in the Daily Mail?

No doubt in view of the delicacy of such regulatory decisions, Leveson II is killed; there will be a Data Use and Ethics Commission instead. It will advise regulators and develop the principles and rules that will give people confidence their data are being handled properly. Wow. We now have the Charter of Fundamental Rights to give us principles, the GDPR to give us rules, and the ECJ to hammer out the case law. Now the People don’t have confidence in such experts we’re going to let the Prime Minister of the day appoint a different lot.

The next government will further strengthen cyber security standards for government and public services, so presumably all such services will have to use expensive networks such as the NHS-wide network from BT which will expect them to manage their own firewalls without telling them how to.

But don’t worry. It will become “as difficult to commit a crime digitally as it is physically”. There is text about working “with international law enforcement agencies to ensure perpetrators are brought to justice” but our local police force isn’t allowed to do anything effective about online accommodation fraud committed by a gang in Germany. They have to work through the NCA – who don’t care. The manifesto signals more of the same: the NCA will get to eat the SFO, which does crimes over £100m, leaving them even less interested in online crooks who steal a thousand pounds of deposit from dozens of students a year.

In fact there is no signal anywhere in the manifesto that May understands the impact of volume cybercrime, even though it’s now most of the property crime in the UK. She rather prefers to boast of the falling crime over the past seven years, as if it were her achievement as Home Secretary. The simple fact is that crime has been going online like everything else, and until 2015 the online part of it wasn’t recorded properly. This was not the doing of Theresa May, but of Margaret Hodge.

The manifesto rather seems to have been drafted in a geek-free room. And let’s not spoil the party by mentioning the impact that tight immigration targets will have on the IT industry, or for that matter on higher education. Perhaps they want us to hope that they don’t really mean that part of it, but perhaps we’d better make a plan to open a campus in India or Canada, just in case.

Bad malware, worse reporting

The Wannacry malware that has infected some UK hospital computers should interest not just security researchers but also people interested in what drives fake news.

Some made errors of fact: the Daily Mail inititally reported the ransom demand as 300 bitcoin, or £415,000, rather than $300 in bitcoin. Others made errors of logic: the Indy, for example, reported that “Up to 90 percent of NHS computers still run XP, released in 2001”, citing as its source a BMJ article which stated that 90% of trusts run this version of Windows. And some made errors of concurrency. After dinner I found inquiries from journalists about my fight with the Prime Minister. My what? Eventually I found that the Guardian had followed something Mrs May’s spokesman had said (“not aware of any evidence that patient data has been compromised”) with something I’d said a couple of hours earlier (“The NHS are saying that patient privacy hasn’t been compromised, but if significant numbers of hospitals have been negligently running unpatched computers for two months after the patch came out, how do they know?”). The Home Secretary later helpfully glossed the PM’s stonewall as “No patient data has been accessed or transferred in any way” but leaving the get-out-of-jail card “that’s the information we’ve been given.”

Many papers caught the international political aspect: that the vulnerability was discovered by the NSA, kept secret rather than fixed (contrary to the advice of Obama’s NSA review group), then stolen from the CIA by the Russians and published via wikileaks. Scary stuff, eh? And we read of some surprising overreactions, such as the GP who switched off his networking as a precaution and found he couldn’t access any of his patients’ records.

As luck would have it, yesterday was the day that I gave my talk on entomology – the classification of software bugs and other security vulnerabilities – to my first-year security and software engineering class. So let’s try to look at it calmly as I’d expect of a student writing an assignment.

The first point is that there’s not a really lot of this malware. The NHS has over 200 hospitals, and the typical IT director is a senior clinician supported by technicians. Yet despite having their IT run by well-meaning amateurs, only 16 NHS organisations have been hit, according to the Register and Kaspersky – including several hospitals.

So the second point is that when the Indy says that “The NHS is a perfect combination of sensitive data and insecure storage. And there’s very little they can do about it” the answer is simple: in well over 90% of NHS organisations, the well-meaning amateurs managed perfectly well. What they did was to keep their systems patched up-to-date; simple hygiene, like washing your hands after going to the toilet.

The third takeaway is that it’s worth looking at the actual code. A UK researcher did so and discovered a kill switch.

Now I am just listening on the BBC morning news to a former deputy director of GCHQ who first cautions against alarmist headlines and argues that everyone develops malware; that a patch had been issued by Microsoft halfway through March; that you can deal with ransomware by keeping decent backups; and that paying ransom will embolden the bad guys. However he claims that it’s clearly an organised criminal attack. (when it could be one guy in his bedroom somewhere) and says that the NCSC should look at whether there is some countermeasure that everyone should have taken (for answer see above).

So our fourth takeaway is that although the details matter, so do the economics of security. When something unexpected happens, you should not just get your head down and look at the code, but look up and observe people’s agendas. Politicians duck and weave; NHS managers blame the system rather than step up to the plate; the NHS as a whole turns every incident into a plea for more money; the spooks want to avoid responsibility for the abuse of their stolen cyberweaponz, but still big up the threat and get more influence for a part of their agency that’s presented as solely defensive. And we academics? Hey, we just want the students to pay attention to what we’re teaching them.

Hope this helps!

Video on Edge

John Brockman of Edge interviewed me in London in March. The video of the interview, and a transcript, are now available on the Edge website. Edge runs big interviews with several dozen scientists a year, with particular interest in people who do cross-disciplinary work. For me, the interaction of economics, psychology and engineering is one of the things that makes security so fascinating, as well as the creativity driven by adversarial behaviour.

The topics covered include the last thirty years of progress (of lack of it) in information security, from the early beginnings, through the crypto wars and crime moving online, to the economics of security. We talked about how cryptography can help less developed countries; about managing complexity in big projects; about how network effects lead firms to design insecure products; about whether big data can undermine democracy by empowering elites; and about how in a future world of intelligent things, security may become more about safety than anything else. Finally I talk about our current big project, the Cambridge Cybercrime Centre.

John runs a literary agency, and he’s worked on books by many of the scientists who feature on his site. This makes me wonder: on what topic should I write my next book?

Pico in the Wild: Replacing Passwords, One Site at a Time

The Pico team have just returned from Paris, where Kat Krol presented at both EuroS&P and the affiliated EuroUSEC workshop on usable security.

Pico is an ERC-funded project, led by Frank Stajano, to liberate humanity from passwords. It lets you log into devices and websites without having to remember any secrets. It relies on “something you have”: in the current prototype, that’s your smartphone, potentially coupled with other wearables, though high-security niche applications could use a dedicated token instead.

Our latest paper presents a new study performed in collaboration with the Gyazo.com website, where we invited users to test out the Pico authentication app for logging in to the site. A QR code was displayed on the Gyazo login page for the duration of the trial, allowing users to access their images simply by scanning the QR code and avoiding the need to enter a username or password.

Participants used Pico for two weeks, during which time we collected feedback using telemetry data, questionnaires and phone interviews. Our aim was to conduct a trial with high ecological validity, avoiding the usual lab-based studies which can run the risk of collecting intentions rather than actual behaviour.

Some of the key results from the paper are that participants liked the idea of Pico and generally found it to be secure and less cognitively demanding than passwords. However, some disliked the need to scan QR codes and suggested replacing them with another modality of interaction. There was also a general consensus that participants wanted to see Pico extended for use with more sites. The pain of password entry on any particular site isn’t so great, but when you scale it up to the plurality of sites we all routinely have to deal with, it becomes a much more serious burden.

The study attracted participants from all over the world, including Brazil, Greece, Japan, Latvia, Spain and the United States. However, it also highlighted some of the challenges of performing experimental studies ‘in the wild’. From an initial pool of seven million potential participants – the number of active users of the Gyazo photo sharing site – after reducing down to those users who entered passwords more regularly on the site and who were willing to participate in the study, we eventually recruited twelve participants to test out Pico. Not as many as we’d hoped for.

In the paper we discuss some of the reasons for this, including the fact that popular websites attempt to minimise the annoyance of password entry through the use of mechanisms such as long-lived cookies and dedicated apps.

While the purpose of the paper is to explore usable security and end-user reactions, it also allowed us to test out the Pico nginx reverse-proxy lens. Using this we could deploy Pico to the Gyazo website as in-page Javascript, demonstrating seamless deployment (zero changes to the backend Gyazo code) and removing the need for the user to install a browser plugin. The tech worked like a charm throughout the trial.

The paper is available from the Internet Society and the abstract for Kat’s short talk, covering future Pico evaluation studies, is available from the EuroS&P website.

1000 days of UDP amplification DDoS attacks

 

We presented “1000 days of UDP amplification DDoS attacks” at APWG’s eCrime 2017 conference last week in Scottsdale Arizona. The paper is here, and the slides from Daniel Thomas’s talk are here.

Distributed Denial of Service (DDoS) attacks employing reflected UDP amplification are regularly used to disrupt networks and systems. The amplification allows one rented server to generate significant volumes of data, while the reflection hides the identity of the attacker. Consequently this is an attractive, low risk, strategy for criminals bent on vandalism and extortion. Despite this, many of these criminals have been arrested.

These reflected UDP amplification attacks work by spoofing the source IP address on UDP packets sent from networks that negligently fail to implement BCP38/SAVE. Since UDP (unlike TCP) does not validate the source address, the much larger responses go to the attacker’s intended victim as they spoof the victim’s address on the packets they send out. There are many protocols that can be exploited in this way including DNS and NTP.

To measure the use of this strategy we analysed the results of running a network of honeypot UDP reflectors from July 2014 onwards. We explored the life cycle of attacks that use our honeypots, from the scanning phase used to detect our honeypot machines, through to their use in attacks. We see a median of 1450 malicious scanners per day across all UDP protocols, and have recorded details of 5.18 million subsequent attacks involving in excess of 3.31 trillion packets. We investigated the length of attacks and found that most are very short, but some last for days.

To estimate the total number of attacks that occurred, including those our honeypots did not observe, we used a capture-recapture statistical technique. From this we estimated that our honeypots can see between 85.1% and 96.6% of UDP reflection attacks over our measurement period.

We observe wide variation in the number of attacks per day over the course of the measurement period as attacks using different protocols went in and out of fashion.

This work is ongoing and data from our honeypot network is available to researchers through the Cambridge Cybercrime Centre.

Also, if you want to help stop these attacks being possible you could help CAIDA by
running their spoofer prober software that checks which ISPs are negligently failing to implement BCP38/SAVE.