Study: Browser Warnings Don't Work

The lock-and-key icon was broken. The site-authentication image was not there. A security message popped up, warning that the site was not properly certified.

And still, more than half of them entered a password and tried to log in.

That's the bottom-line finding of a new study from researchers at Harvard University and MIT, who conducted a live test of banking users to measure the effectiveness of browser-based authentication and anti-phishing features earlier this year. The research is scheduled to be presented at the IEEE Symposium on Security and Privacy next month.

In the study, 67 customers of a single bank were asked to perform common online banking tasks. As they logged in, they were presented with increasingly conspicuous visual clues that suggested they might be about to enter a phishing or other fraudulent site.

In the first test, the researchers "broke" the HTTPS security key. The lock-and-key icon at the bottom of the screen clearly was not in one piece, and the URL showed "http" rather than "https." After seeing these cues, all (100%) of the participants proceeded to log in anyway.

In the second test, the researchers removed the site authentication image from the users' browser screens. These images, typified by Bank of America's Sitekey, are supposed to authenticate the site for the user by presenting a pre-selected image that the user can recognize. The researchers did not reveal which site authentication image technology was involved in the test.

When both the HTTPS security key and the site authentication image were displayed in an unsecured state, only 3 percent of the participants stopped the logon process before typing in their passwords. The rest of the users -- 97 percent -- went ahead and logged on.

In the third test, the researchers presented the participants with a browser "warning page" stating that there was a problem with the target site's security certificate. Users were then given the option of closing the page or continuing to the Website.

In the presence of the broken HTTP key, a non-secure URL, an absent site authentication image, and a strongly-worded pop-up warning, 53 percent of the participants chose to continue to the banking site. Only 47 percent chose to abandon the logon before they had typed their passwords.

"We confirm prior findings that users ignore HTTPS indicators," the researchers say in the study. "No participants withheld their passwords when these indicators were removed. We also present the first empirical investigation of site authentication images, and we find them to be ineffective."

The tests were done on Microsoft's IE6 browser and, therefore, did not evaluate the effectiveness of the new anti-phishing features in IE7, where color-coded URLs and pop-up warning screens are a new feature. "Very few of the participants had seen the warning pages before," the researchers conceded. "Now that IE7 is widely available, users may see warning pages often enough to become complacent about heeding them."

But the study findings support some experts' skepticism that anti-phishing warnings, such as the new Extended Validation SSL, will have much impact on users' behavior. A study conducted by Microsoft and Stanford University in February has already suggested that EV SSL doesn't work. (See EV SSL: Dead on Arrival?)

"Prior studies have reported that few users notice the presence of HTTPS indicators such as the browser lock icon," the study notes. "Our results corroborate these findings and extend them by showing that even participants whose passwords are at risk fail to react as recommended when HTTPS indicators are absent."

— Tim Wilson, Site Editor, Dark Reading