NSSLabs just recently published a study on browser security with regards to Phising and Malware protection, which we comissioned. To take it upfront: The whole methodology is transperent and therefore rather than challenging the results, let’s learn from them how we can improve.
As I do not want to take the joy away for you to read the study, I just want to show you two pieces of information from the report:
Let’s look at the Phising study first:
They looked at how long a user has to wait until a Phishing URL is blocked by the browser:
|Browser||Avg. Add Time (hrs)|
|Internet Explorer 8||4.96|
|Opera 10 Beta||6.19|
Scary to me is that Safari by far increases the mean of the group. Even though Chrome 2 is behind the other three, I guess that Internet Explorer, Firefox and Opera are comparable here (even though we are more than 20% faster).
So, speed is one thing, accuracy and completeness another one. Let me quote from the report: The average phishing URL catch rate for browsers over the entire 14 day test period ranged from 2% for Safari 4 to 83% for Windows Internet Explorer 8. Internet Explorer 8 and Firefox 3 were the most consistent in the high level of protection they offered. Statistically, Internet Explorer 8 and Firefox 3 had a two-way tie for first, given the margin of error of 3.96%. Opera 10 beta came in third due to inconsistent protection during the test. Chrome 2 was consistent, albeit at a much lower rate of protection, and Safari offered minimal overall protection.
Or in graphical terms:
Then they did a similar test with regards to socially engineered Malware protection:
Again, looking at the response time, I guess we can improve when it comes to the comparison with other browsers:
|Browser||Avg. Add Time (hrs)|
|Opera 10 Beta||5.5|
|Internet Explorer 8||9.2|
But again, there is a huge gap between the best and the worst (and they are very bad). When it comes then to the block rate, the game changes:
Internet Explorer 8 caught 81% of the live threats, an exceptional score which surpassed the next best browser (Firefox 3) by a 54% margin. Windows Internet Explorer 8 improved 12% between Q1 and Q2 tests, evidence of concerted efforts Microsoft is making in the SmartScreen technology.
Firefox 3 caught 27% of live threats, far fewer than Internet Explorer 8. It was, however, the best among products utilizing the Google SafeBrowsing API. (Note: Firefox 3.5 was not stable enough to be tested during the course of this test. A patch has subsequently become available to address the stability issue. We were able to manually verify that the protection was identical between versions 3.0.11 and 3.5).
Safari 4 caught 21% of live threats. Overall protection varied greatly, with two short periods of severe dips. Chrome 2 caught just 7% of live threats an 8% drop from the previous test.
Opera 10 Beta caught a mere 1% of live threats, providing virtually no protection against socially engineered malware. In our test bed validation, we verified there was effectively no difference between Opera 9 and Opera 10 Beta.
So, this is definitely interesting material for your next browser discussion