Today Which? released their survey of online banking security. The results are summarized in their press release and the full article is in the September edition of “Which? Computing”.

The article found that there was substantial variation in what authentication measures UK banks used. Some used normal password fields, some used drop-down boxes, and some used a CAP smart card reader. All of these are vulnerable to attack by a sophisticated criminal (see for example our paper on CAP), but the article argued that it is better to force attackers to work harder to break into a customer’s account. Whether this approach would actually decrease fraud is an interesting question. Intuitively it makes sense, but it might just succeed in putting the manufacturers of unsophisticated malware out of business, and the criminals actually performing the fraud would just buy a smarter kit.

However, what I found most interesting were the responses from the banks whose sites were surveyed.

Barclays (which came top due to their use of CAP) were pleased:

“We believe our customers have the best security packages of all online banks to protect them and their money.”

In contrast, Halifax (who came bottom) didn’t like the survey saying:

“Any meaningful assessment of a bank’s fraud prevention tools needs to fully examine all systems whether they can be seen directly by customers or not and we would never release details of these systems to any third party.”

I suppose it is unsurprising that the banks which came top were happier with the results than those which came bottom, but to a certain extent I sympathize with Halifax. They are correct in saying that back-end controls (e.g. spotting suspicious transactions and reversing fraudulent ones) are very important tools at preventing fraud. I think the article is clear on this point, always saying that they are comparing “customer-facing” or “visible” security measures and including a section describing the limitations of the study.

However, I think this complaint indicates a deeper problem with consumer banking: customers have no way to tell which bank will better protect their money. About the only figure the banks offered was HSBC saying they were better than average. Fraud figures for individual banks do exist (APACS collects them), and they are shared between the banks, but they are withheld from customers and shareholders. So I don’t think it is surprising that consumer groups are comparing the only thing they can.

I can understand the reluctance in publishing fraud figures — it makes customers think their money is not safe, and no bank wants to be at the bottom. However, I do think it would be in the long-term best interests of everyone if there could be meaningful comparison of banks in terms of security. Customers can compare their safety while driving and while in hospital, but why not when they bank online?

So while I admit there are problems with the Which? report, I do think it is a step in the right direction. They are joining a growing group of security professionals who are calling for better data on security breaches. Which? were also behind the survey which found that 20% of fraud victims don’t get their money back, and a campaign to get better statistics on complaints against banks. I wish them luck in their efforts.