This is the third part in a series on password implementations at real websites, based on my paper at WEIS 2010 with Joseph Bonneau.

In our analysis of 150 password deployments online, we observed a surprising diversity of implementation choices. Whilst sites can be ranked by the overall security of their password scheme, there is a vast middle group in which sites make seemingly incongruous security decisions. We also found almost no evidence of commonality in implementations. Examining the details of Web forms (variable names, etc.) and the format of automated emails, we found little evidence that sites are re-using a common code base. This lack of consistency in technical choices suggests that standards and guidelines could improve security.

Numerous RFCs concern themselves with one-time passwords and other relatively sophisticated authentication protocols. Yet, traditional password-based authentication remains the most prevalent authentication protocol on the Internet, as the International Telecommunication Union–itself a United Nations specialized agency to standardise telecommunications on a worldwide basis–observes in their ITU-T Recommendation X.1151, “Guideline on secure password-based, authentication protocol with key exchange.” Client PKI has not seen wide-spread adoption and tokens or smart-cards are prohibitively cost-inefficient or inconvenient for most websites. While passwords have many shortcomings, it is essential deploy them as carefully and securely as possible. Formal standards and guidelines of best practices are essential to help developers.

Where Web standards come from

Traditionally, standards for secure IT systems have been developed by national standardisation agencies, such as DIN, and federal IT agencies, such as the BSI or NIST whose recommendations and guidelines first applied to government IT and whose scope was then extended to commercial installations. With the harmonisation of standardisation efforts, national standards then became European Norms (EN) or international standards (of the ISO). In some cases, technical specifications made it into codified law.

Second, new cross-national standardisation organisations have emerged, specifically targeted at the Internet and the Web. Most relevant are the Internet Engineering Task Force (IETF), with its committee the Internet Architecture Board (IAB), and the World Wide Web Consortium (W3C). Both are characterised by a fairly quick standards track and an impetus of “making things work.” More importantly, their standards are universally accessible for free. Other Internet standards bodies, such as the Internet Assigned Numbers Authority (IANA) would not include passwords on the Web into their scope of work.

Third, there are industry consortia such as Organization for the Advancement of Structured Information Standards (OASIS) and professional associations such as the Institute of Electrical and Electronics Engineers (IEEE) that shape Internet developments on an applications and a technical level, but their work does not include password advice.

Fourth, commercial entities that certify the security of websites and issue seals for a fee have (presumably) developed their own checklists. However, these lists are not publicly available and their quality cannot be the scrutinised. All the same, blatant security breaches at certified websites have cast doubt on the quality of these certification procedures (e.g. TÜV SÜD s@fer-shopping and Libri.de case, ControlScan), as has academic research into their effectiveness. Moreover, it is unclear if an evaluation checklist can also serve as a guideline during development.

Finally, large software vendors, providers of software development environments, website toolkits and frameworks can effectively shape the technical landscape on the Web. Defaults in such solutions are the most powerful way to establish a de-facto standard and potentially exercise positive paternalism.

Standards on password technologies

Government standards

Most government-track standards focus on systems security, but the recommendations on password security they contain could be analogously applied  to the Web. In the United States, NIST published FIPS 112 on “Password Usage” in 1985, which gave very specific advice on password implementations. The standard includes encrypting all passwords during transmission and storage, forcing users to change passwords every year, and forcing users to pick a new password after forgetting the previous one. The guidelines also motivate a password length of 5 to 8 characters as a compromise between memorability, security, and ease of typing, and supporting several potential character sets.

More recently, NIST published SP 800-63 “Electronic Authentication Guideline” in 2008. This document contains fewer specifics about password authentication, instead defining four assurance levels for remote authentication, for which only the lowest two levels can be based solely on passwords. It provides general requirements for password security, such as encryption during transmission and storage, but is not intended for website developers and provides no guidance on the interaction of password authentication with Web protocols. The standard also avoids recommending any specific password requirements, but does mandate upper limits on password guessability and provides a detailed algorithm for approximating the required entropy with length and character classes.

Amongst international standards, ISO/IEC 27001:2005 mandates that a secure system requires strong passwords, but provides no technical details; buyers of the accompanying implementation guide ISO/IEC 27002:2005 get some additional details on “access control,” including password considerations. The German BSI (Federal Office for Information Security) published technical standards for satisfying ISO 27001 which do contain specific recommendations. The IT-Grundschutz (IT Baseline Protection) Catalogues are organised by modules and provide details on threats and security safeguards; they can be downloaded for free. Now in its eleventh edition, it has been translated into several languages, and has grown to more than 4000 pages. On the subject of passwords, the compromise between security and usability is acknowledged again. Alphanumeric passwords should be made up of at least eight characters, and numbers-only passwords should have a minimum length of six. Further, error messages should be unspecific with no indication whether username and/or password were wrong and account locking should happen silently.

None of these standards deals with password recovery, assuming that an administrative help desk and a secure channel are available. Also they abstract away many implementation issues on the Web, such as the choice between HTTP authentication and form-based submission of login credentials.

HTTP and HTML standards

In HTTP authentication, the server detects that the requested resource requires authentication and sends an HTTP 401 status code along with a “WWW-Authenticate” header. According to RFC 2617, the browser then displays a (modal) dialog that prompts for username and password before any content is displayed. In the “Basic” variant, the credentials are submitted unencrypted; in “Digest” mode, introduced with HTTP/1.1 to “avoid the most serious flaws of Basic authentication”, salted MD5 hashes with incremental nonces can be used. Regardless of security considerations, HTTP authentication suffers from usability drawbacks and the inability to style the password prompt. As a result, we did not see a single instance of HTTP authentication in our survey.

In form-based authentication, the implementation is site-specific. The HTML (XHTML, WML, …) page includes a form with input fields for username and password. The data the user enters is submitted just as any other HTTP request, either as part of the request (GET) or in the body (POST). Consequently, transmission is unprotected by default unless the connection is encrypted via TLS (HTTPS) or hashing is done with a client-side script (we observed three websites doing so). The relevant standards are the HTML specifications that define the markup for Web forms and input fields; these elements were already part of HTML 2.0 in 1995. They came with “security considerations,” noting that the “widely deployed methods for submitting forms requests — HTTP and SMTP — provide little assurance of confidentiality. Information providers who request sensitive information via forms — especially by way of the `PASSWORD’ type input field — should be aware and make their users aware of the lack of confidentiality.” This advice was dropped in HTML 3.2, where it was instead said to obfuscate user input “using a character like * to hide the text from prying eyes when entering passwords” (suggesting a shift in the threat model). Incidentally, the example provided in the specification included a password field of “size=12” with no “maxlength” restriction. HTML 4.0 saw the re-introduction of a note that a plaintext password “affords only light security protection” as it is “transmitted to the server in clear text”. Similar but even more explicit caveats are also included in XForms. Security notes have disappeared again from HTML5, with the exception of a vague and potentially misleading hint that a “user agent should obscure the value so that people other than the user cannot see it.”

Developers are thus unable to glean concrete help from the HTML specifications on how to implement a good password prompt. Given the strong effort to revamp forms in HTML5, it seems a lost opportunity to add better security functionality to the password input field through client-side scripting, as it was done for many other of the input fields. As early as 1999, a W3C Note on User Agent Authentication Forms proposed “new HTML capability to aid in the development of authenticated Web user interfaces”, that came down to combining the security advantages of HTTP Digest authentication with the superior user experience of Web forms (the interplay between HTTP and form-based authentication has been revisited recently from a usability perspective, resulting in a rather obscure cookie-based scheme).

IETF standards

Meanwhile, the IETF published requests for comments on stronger authentication protocols and has spent more time on password replacements than actual passwords – there are many RFCs on Internet Key Exchange (RFCs 2409, 3526, 3664, 3706, …, 5903) but not a single one on  “how to do traditional passwords.” Anecdotally, the most relevant RFC on password technology is RFC 972 (from 1986). It describes a “Password Generator Protocol” that is a service to generate and suggest to the user six strong but memorable 8-character passwords from which the user may then choose one.

More recently, the prevalence of plaintext passwords on the Web has been acknowledged. The Survey of Authentication Mechanisms by the Internet Architecture Board, currently in its 7th draft, provides a valuable critical review of authentication technologies, including (plain) passwords and their alternatives. The section on passwords lists numerous problems, with countermeasures for each, such as dictionary attacks and phishing. Resistance against brute force attacks by guessing common passwords is also discussed with countermeasure alternatives such as incorporating a limited try capability with lock-down or an increasing delay for each additional guess. In a sense, this collection of problems and countermeasures it can be read as best-practices, but requires an attentive reader to transform suggested alternatives into real resolutions. Also, this review still fails to recognise that password reset and recovery have important implications for overall system security.

Following the “log-in metaphor,” authentication on the Web is often expected to last for a session rather than for a single request. This calls for preservation and replay of credentials. When using HTTP authentication, the user agent can include authentication information in the request headers. Otherwise, cookies are the universal technique for replaying client identification, as our survey has confirmed. However RFC 2964 states that “it is generally inappropriate to use the HTTP State Management protocol as an authentication mechanism,” since cookies are transmitted in an unencrypted manner. RFC 2965 states that “While it is common practice to use them this way, cookies are not designed or intended to be used to hold authentication information, such as account names and passwords. Unless such cookies are exchanged over an encrypted path, the account information they contain is highly vulnerable to perusal and theft.” This advice suggests a troubling divide between IETF standards and near-universal practice on the Web.

Industry libraries and practical guidelines

Good drop-in solutions exist for content management systems like Drupal and Web frameworks like Django, Ruby on Rails, or ASP.NET which cover most elements of the password life cycle. The Apache project has authentication modules, but only for the uncommon HTTP authentication mechanism and not form-based authentication. Because passwords interface with many areas of a website, it remains debated whether re-using an existing password module reduces development effort for a highly customised site. Certainly, many of the professional websites we observed in our survey made mistakes not found in the modules listed above (though there are certainly dubious password modules as well), indicating that large websites prefer developing their own password solution in contrast to other elements like TLS authentication.

For developers re-inventing the wheel, formal guidelines are harder to find, being largely spread across blogs from industry, open-source projects, and Web security experts. There does not appear to be a canonical set of guidelines online which emerges highly in search engine results. Academic research on traditional passwords has mostly focused on user studies; a literature review can be found in our paper. In search of scientific results, internal validity competes with external validity: the better controlled a laboratory scenario, the less similar it becomes to real-world deployments. Consequently, conclusions from academic studies often cannot be translated easily into actual code. Although very concrete dos and dont’s of password deployments sometimes arise in academia, the main body of knowledge in academic papers is generally not accessible to developers as it resides outside their normal fora (and may be behind pay walls).

Conclusions

The value of password standards lies in them being read by developers for help in deploying better password schemes on their websites. Whilst good standards appeal as is, they aren’t l’art pour l’art. Government standards for system security tend to ignore new Web-specific pitfalls and do not cover the entire lifecycle of passwords, especially reset and recovery with no availability of inherently secure channels. Advice in existing standards may also be too general to be turned into implementations easily. Standards bodies dedicated to the Web have not embraced the pragmatic questions of password security, devoting more effort to password replacements.

Password implementation is a very common task implemented by thousands of websites, but standards are doing little to improve the process. Until good standards become available, their role is partially fulfilled by industry de-facto standards or community-driven guidelines which remain scattered on the Web amidst much noise.