Vulnerable Web Browser Study – Full of Fail

So came across an interesting report today from various security folks (including Gunter Ollmann from ISS):

I can appreciate what they are trying to do - and I believe they were probably trying to be as un-biased and scientific as they possibly could given the nebulous goal of the study but it was, unfortunately, full of fail (at least with respect to the IE results).  What they seem to have done is combed the Google logs looking at the user-agent strings over a 1.5 year period to gather major + minor version information for the browsers they studied. The only problem?  IE doesn't send minor version information, so there's no way to determine IE patch levels from the user-agent string.  Oops.

So to compensate for that they:

  1. Threw out all IE 5.x and 6.x major version info for some reason - they say it's because IE7 is the most secure version.  While that is true - it is quite possible to be running fully patched IE 5.x or IE 6.x and be just as protected as a user running fully patched IE 7.x.  Why?  Because we will patch and support IE 5.x for as long as Windows 2000 is supported and IE 6.x for as long as XP is supported.  This makes the major version of IE much less interesting than say for Mozilla FireFox which as near as I can tell only supports the previous major version for 'up to 6 months' after the current major version is released.  I can imagine if we only supported IE 5.x and IE 6.x for 6 months after IE 7.x was released you'd see a lot more uptake on IE7 than we have - but alas - most businesses won't deploy new major versions unless they *have* to and with IE - they don't *have* to.

  2. They looked at a *completely different data set* for IE minor version info!!!  So for everything but Internet Explorer - they examined the Google logs, but for IE they relied on voluntary installs of the Secunia software inspector thing which is (I believe) a client-side app that will scan your machine and figure out the patch levels for various things and upload the results to Secunia.  Secunia claims about half a million installs so it's not insignificant - but it's also not comparable to combing the Google logs either (IMHO - but I'm not a statistician and wouldn't even try to play one on TV) and since it's not even the same set of data - I can't fathom why they felt it was scientifically valid to include along side the other browser results!

For these simple facts - I really don't think it was wise to add IE to the mix . . . they should have (in my opinion) stuck to examining the Google logs - and stuck to examining the user-agent strings for browsers that report minor version information.  Apples to Oranges comparisons aren't very good.

EDIT:  Meh - someone asked me why IE doesn't have the minor version info in the user-agent string and I had to admit I wasn't sure.  Just never really thought about it I guess.  And so it's with a bit of embarassment that I have to admit I didn't even think about the information disclosure risk that this would represent and how it could allow attackers to know exactly which exploit to throw at your browser.  Dave thought of that though. 🙂  Good job Dave. 🙂  I will admit - the browser and web app sec is not my forte . . . is there an easy way to ID the exact version of the browser purely from Javascript without using an AX?  That's left as an exercise to the reader and I don't have time to dig right now. 🙂

Comments (2)
  1. Anonymous says:

    Robert Hensing posted some criticism of a study that purported to analyze how many users are at risk

  2. Anonymous says:

    Pojawił się dokument Understanding the Web browser threat: Examination of vulnerable online Web browser populations and the "insecurity iceberg". Mała zajawka: For the last 18 month we analyzed the daily USER-AGENT data collected by Google’s Web s

Comments are closed.

Skip to main content