Windows Server 2003 spanks Red Hat’s monkey?

Interesting information from RSA, it’s nice to see someone other than me notice the pure creamy goodness of WS2003 for once (I’ve noticed it from the incident response side of things by noting a marked absence of WS2003 hacking cases over the last 2 years as compared to Windows 2000).

Why am I not surprised by their findings? 
Simple – I’ve been slowly biding my time as the WS2003 OS matures watching the bug counts in our OS and those of our competitors month by month using an independant site like Secunia – anyone who’s been doing this already knows that there are dramatically more bugs discovered and fixed by our open source competitor which in my mind does not seem to indiciate any superior secure coding kung-fu being employed on their part (or that the ‘many eyes’ approach is indeed contributing to provably more secure code).

Here are the stats from Secunia – an organization not affiliated with Microsoft b.t.w. 🙂
Windows Server 2003 Enterprise Edition
44 advisories since June of 2003, 11% un-patched right now, 0% extremely critical, 45% highly critical, 59% exploitable ‘from remote’.
Looking at the two un-patched ones, one is an HTML help vuln from 2003 – so I am betting that is a mistake and another is a minor information disclosure bug – obviously we have work to do still and we are doing it.

Red Hat Enterprise Linux ES 3.0
136 advisories, since NOVEMBER of 2004, 0% un-patched right now (I wonder if that includes all the latest slew of Linux kernel vulnerabilities reported in the last day or so) 1% extremely critical, 24% highly critical, 66% exploitable ‘from remote’.

We are by no means perfect, and we still have a loooooooong way to go, but the journey has at least started and as the first batch of products to go through our secure development lifecycle start to withstand the test of time – it is really no surprise to me to see that our focus on building secure products first and foremost is starting to pay off in terms of better quality software with fewer and less damaging vulnerabilities.

Comments (31)

  1. Anonymous says:

    Ephedrine faq ephedrine fatloss.

  2. Anonymous says:

    Does buspar work. Buspar experience. Buspar anxiety. Buspar. Buspar and weight gain.

  3. Anonymous says:

    Robert Hensing’s Secure Windows Initiative Blog : Windows Server 2003 spanks Red Hat’s monkey? Some interesting comment about the study comparing Redhat and Windows Server 2003 over at Robert Hensings blog. As people’ll know if they’ve been following slashdot, it…

  4. Anonymous Coward says:

    Wonder if the test used a default install of each OS…on the MSFT page, they list a bug in NNTP and on the Linux side a bug in kdelib. I wouldn’t imagine a production Windows server running NNTP if it was unnecessary, and the same goes for kdelib.

    What would be interesting would be an anonymous survey of what people were actually running in the field, and what vulnerabilities exist there.

    Of course, there’s always OpenBSD, which claims to have one remote hole in the default install in more than eight years…I don’t think Windows or any Linux distro can touch that.

  5. Robert Hensing says:

    Great points and well written – here are some counter points.

    1. It’s generally un-intersting comparing one vendors OS CD to another (as you point out). What’s interesting are comparisons of real world servers and roles, especially ones that are web-facing. To do that you need to add some things to the stack like IIS6, ASP.NET and SQL2000 (on WS2003) and Apache, MySQL and PHP on Linux. Then what you’ve got is what most people actually use these operating systems for on the Internet – a web application. But now you have to include all of THOSE application vulnerabilities as well. I assure you this only makes it worse for Linux – not better (for example, check out the IIS6 vs. Apache bug counts using the same Secunia web site). I leave this as an exercise to the reader. 🙂

    2. OpenBSD – that’s an interesting distro, it certainly does have an impressive security record but they slip in what anyone else woudl call ‘security updates’ all the time that they don’t label as ‘security updates’.

    They fix things that lead to DoS but call them ‘reliability fixes’ or something like that – whereas from Microsoft anything that can remotely DoS Windows is rated at Important at least and we release a security bulletin.

  6. Anonymous Coward says:

    1. I would argue the numbers on the RHES3 page actually include PostgreSQL and Apache, as I see at least one vulnerability for each of those listed on that page. The default ES3 install includs both of those. I could be misreading the numbers…but there’s a whole bunch of stuff included there that would never be running in the real world.

    This is partly because of RH’s decision to turn on the kitchen sink in the default server install so their product can look extremely feature rich…it hurts them in situations like this.

    Agree w/respect to OpenBSD…plus, if nothing is really enabled in the default install, it’s going to look a lot nicer in this kind of comparison.

  7. Peter da Silva says:

    To be fair, you would need to compare the buglist of Windows, Microsoft Office, and maybe a hundred third-party packages… because that’s all stuff that ships on the Red Hat CD. That would make things look a lot worse for Windows.

    Similarly for Apache, a huge number of Apache bugs are actually in third-party add-ons that are shipped (even if not used) with it.

    The big problem I have with Windows security is that it’s a lot harder (and in some cases impossible) to turn things off and know that they’re off.

    If I could get Windows without the HTML control, for example, all the "cross zone" attacks would go away. But if you did that, Windows Update, the Control panel, and no doubt lots of other things that I haven’t thought of yet would break.

    In UNIX, I can bind services to one particular port, and run them chrooted or in a jail so that even if they’re exploited they can’t get out again. In Windows, the only way to do the equivalent of binding many services to specific ports is by playing with firewall rules… and there’s no chroot or jail.

    IIS and the HTML control both have had a bad problem with reparsing strings, in Apache you run into that with some applications that run under it… but it does a pretty good job of not screwing up encodings before they get passed to CGIs or applets. How about a "secure IIS" that never reparsed a URL submitted to it?

    Finally, Red Hat is hardly a good example of UNIX/Linux. The only Linux I would trust less is that Linspire thing.

  8. Robert Hensing says:

    Very good feedback – I disagree we’d need to do things like include Office in the bugcount as I’m pretty sure RHEL is not shipping an Office suite like OpenOffice (I could be wrong – I’m not like a Linux expert). One could also easily review the 136 advisories to see if its in a component that is shared between the two OS’s and eliminate ones that aren’t. But if it’s in a component that’s not shared – what’s that tell you? That Linux has a BIGGER potential attack surface than Windows due to the inclusion of everything but a mod_kitchensink in the distro?

    You do have a very valid point about Linux and being able to strip it down to bare appliance-like functionality – this is presently something that can be done more easily on Linux than on Windows.

    That said – you CAN run most Windows services (most of them, not all) as whatever user account you like (i.e. low privilege network service or local service limitted user account) to mitigate the damage that can be done by exploiting it (this is like your chroot jail as these accounts don’t have write permissions anywhere interesting and aren’t root / admin accounts).

  9. Drew says:

    Let me guess . . .

    You’re worried about job security because you haven’t seen many instrusions on Server 2003. And the best way to advertise that you’re skilled and available for a new position is to try to write something that will get your blog on /. again.

    Am I close?

  10. Robert Hensing says:

    ROTFL!! Okay THAT was a great blog post man – I appreciate the laugh. 🙂

    I actually could care less about being /.’d, I’m definitely not in this for the fame – if I were I’d just write a book and try to get rich – I do this for fun and to help educate customers. 🙂

    Don’t get me wrong – /. is a great community and all, and I frequent the site from time to time, but I was a little amused that they only managed to pick up my blog post on pass-phrases about 6 months after it went live (check the date on when I posted that thing). 🙂

    Perhaps that says something about the /. community?

    I say that only because my blog post was picked up by Win2k News AND PC Magazine (not to mention full-disclosure, bugtraq, etc.) loooooong before /. ran it. 🙂 I had actually sort of assumed it may have already been submitted and somehow I missed it. 🙂

  11. Michel says:

    This kind of stats does only work for MS since MS distributes their patches monthly.

    The switch to this model was a marketing issue, to allow such favorable comparisons.

    Before that time it was not unusual to have 2-3 patches per month only for IE.

    By doing 1 cumulative patch per month per product the MS stats are not comparable !

  12. Robert Hensing says:

    That is NOT the reason we went to monthly security updates and ignoring that you’re still wrong. Secunia isn’t tracking security bulletins – they are tracking vulnerabilities and for any given bulletin there will be one or more vulnerabilities that are resolved by the security update. So if we released one Windows bulletin that fixed 5 vulnerabiltiies, this doesn’t count as ‘1’ this counts as ‘5’. Furthermore, since we switched to the monthly update process we have released IE out of cycle updates twice as needed to protect customers.

    Releasing bulletins on monthly schedules is a win win for us and for customers. It allows us to plan which month we’re going to release our updates in and then test the snot out of them before we release. Customers benefit as well becuase they can plan their resources and staffing and outages accordingly. We’ve had overwhelmingly positive feedback from this change and other vendors have even taken steps to do the same. Imitations is the sincerest form of flattery I guess.

  13. John says:

    Just my two penneth, but at the end of the day these are just stats, we could play with them all day and not get any real answers. (I had have! see below).

    I think that it’s great the MS is now taking security much more seriously and they are making some really good changes. But at the end of the day it’s more about how you approach the whole of your security. If you are just going to place a default install on the web without any changes then quite frankly you deserve to get attacked! I would be splendid if you could but just a little amount of planning would tell you that this is not currently the case. You don’t keep your stock in an open barn so why do it with your data?

    On the subject of chroot jails the idea is that even if the intruder does manage to gain elevated privilages in some way then all they can see, even as the superuser will be a small copy of the parts of the system that are required to run that one process. It goes one step further than just running a process as a special user. But even these are not perfect and have lead to things like SELinux and RBACS on Solaris.

    Oh, and by the way at least one of the advisories for RHEL is for OpenOffice. There are more for things like squirrelmail, gaim (IM client), cvs,(Version control), ethereal(Network monitoring) and more than one Database. You have to compare like with like. Just because Redhat ships a full product does not mean that you have to install it. You don’t put exchange or MS office on your Web server. And if you want to look at bug counts then skip over to the Debian entry for some big numbers! But then again they are shipping about 8000 packages on 10 archetcures so 400 bugs is not that bad!

  14. Harlan says:


    "…if I were I’d just write a book and try to get rich…"

    Emphasis on "try", dude. It doesn’t happen. Oh, wait…are you talking about writing romance novels with images of a shirtless Fabio on the cover? Now *that* kind of book you can get rich from…but writing in our field? No way!

    Regarding /….stuff only appears there when someone posts it. Someone wrote a review of my book and it didn’t appear on the site for quite a while…evidently, it was written in such a way that the moderator didn’t know whether to try and fix it, or just trash it.

    With regards to your post of 2/17, at 8:33pm…interesting what some people post, isn’t it? Never let the facts get in the way of a good rant!

    Carry on, my friend!

  15. Steve loughran says:

    I think a key message here is not "which one is best over a certain interval" but "look, both are vulnerable, there are no silver bullets".

    You cannot move to RHEL and expect your system not to be 0wned within a week, nor could you bring up Win2K03 and expect not to have to invest time locking it down. One thing Server 2003 does do is lock everything down by default (no exported printers over IPP here :), and tightened up a lot of other stuff, low level stuff. Its so tight that some apps dont work in untrusted user mode (we test our apps in non-power use mode, see). Which is inconvenient, but ultimately a good thing.

    What irritates me is this: regardless of the OS you use, you have to patch and reboot monthly. That is the harsh reality of the situation, and it means that when i go on a four week vacation I have to turn off my work sever *and* my home server, as there is no way I can keep them up to date while I am away.

    It also means that any VMWare image I have of either OS is a security risk as it ages. Those monthly DVDs of my WinXP images may be perfect backups of machine state, but they are a chain of differently vulnerable virtual systems.

    We, that is the software development community, have to do better. We have to stop thinking that because patches are possible, we can be less than thorough. We have to stop putting features in ‘because they may be useful’, unless you know that the value outweighs the possible insecurities.

  16. Robert Hensing says:

    Man – I have to admit, I used the dramatic title to try and get the Linux followers who may be subscribed to my blog ‘out of curiosity’ whipped into a frenzy to see if I could elicit some really passionate ‘you suck’ type replies but you all have managed to disappoint me!

    These are all for the most part GREAT replies, very well through out and very well written. It’s nice to see my blog attracting a higher caliber IT person – true professionals! 🙂

    I pretty much agree with your post and John’s before yours etc. (and Harlan’s about not getting rich off of books. 🙂

    P.S. Steve – when you go on vacation – why not just enable automatic udpates at 3am on your Windows boxes so that they install the patches themselves while you’re away? 🙂 The new AU client works amazingly well . . . I never patch my XP machines at home and here at Microsoft if you forget to patch, Corpsec carpet bombs the network with patches and you get them whether you like it or not. 🙂

  17. Alex Harden says:

    Another metric to add to the comparison might be the number of patches cited that require a complete OS restart to be activated. My day job is as a Windows Server admin, but I’m a Linux hobbyist. Most of the patches I’ve ever installed on Linux required at most a restart of the affected service to be activated. Most of the critical patches released for Windows Server 2003 have required complete OS restarts. That might mean that while there are less vulnerabilities on WS2003, it requires more drastic measures and additional downtime to patch.

  18. Robert Hensing says:

    Another very good point – on WS2003 most patches should NOT require a restart – the ones that do usually affect the kernel and I believe that patches on Linux that affect the kernel require you to recompile and restart – so that’s pretty much the same.

    The problem with Windows is that most people don’t understand WHY restarts are required or how to avoid them. Right now if a file that needs updating is in use – the update installer may or may not try to stop the process hosting that file. If it doesn’t or can’t stop the process hosting the file – then it will copy the file anyways putting it in the PendingFileRenameOperations registry queue and ask you to restart. To avoid reboots for non-kernel security updates it’s usually as simple as figuring out what files are being updated (using the file manifest in the bulletin) and then using something like Process Explorer to see what processes they are loaded in and thus what services need to be stopped before the update is installed and then re-started afterwards.

    That said – even doing all of this I think we might still be a bit behind Linux here – but we’re working on that . . . WS2003 SP1 is going to allow us to do ‘hotpatching’ in some cases – eliminating the need for a reboot even if the DLL or driver is in use. 🙂

  19. Alex Harden says:

    Thanks for the response, Robert. My team at work administers hundreds of W2K and W2K3 servers and our current patching solution is SUS/AU, soon to be WUS when it comes out of beta. We defer to MS’s recommended best practices when patching our fleet, so watching the usage of a particular DLL on a particular server to determine if/when we can restart a service to activate a particular patch vs. the default behavior of restarting the server when prompted would be very time consuming. Note also that the AU client stops requesting packages from SUS after it’s installed hotfixes that indicated that a server restart was required.

    I’m currently beta-testing WUS (and, as a result, the new MSI3-powered AU clients), and I don’t believe they have any of the functionality you’re describing from a reboot-avoidance perspective. Perhaps later hotfix packages will check for dependencies and allow the admin to determine if they’d like to temporarily take a service down in lieu of a restart. I’ve seen a few IIS-related hotfixes do that. However, that certainly isn’t the norm and I’ve seen nothing in my testing to indicate that that is going to change. Granted, I haven’t pointed any WS2003 servers running SP1 RC2 at WUS, yet.

    Your point regarding recompiling the Linux kernel to address security vulnerabilities is contrary to what I’m used to seeing. Perhaps the reloading of a patched kernel module might be required in some cases, but I thought the commercial Linux distributions don’t require their users to recompile the kernel. Most of my Linux experience is at home, behind a firewall, and I haven’t ever been "forced" to patch for security reasons. So I don’t really have much experience to draw on there. 😉

  20. Robert Hensing says:

    Okay lots of things to comment on.

    1. Update.exe (our patch installation product) is just that – it’s a product. It has its own product group inside Microsoft. What they do is ‘release’ this product internally and other product groups consume that product and use it to distribute patches. update.exe is highly configurable and it uses an .INF to read what it’s supposed to do for a given patch installation. One of the sections in the .INF file for any given update is [ServicesToRestart]. It’s up to the product group shipping the security update to fill this in. For example, say we’re shipping an IIS update. The IIS product group would be responsible for taking the latest update.exe package and creating an update. If they were to leave the [ServciesToRestart] section blank – well guess what? If you installed that update on an IIS server – it’d say you needed to reboot (because it would never stop the service and the files would get copied via the MoveFileEx() API with the "delay until reboot" flag passed in to it). I assure you, our update installer is quite capable of stopping affected services and then restarting them if it is configured to do so.

    2. This is completely different from hotpatching.

    3. This does not require MSI 3.0 – MSI is actually a different update installer – we have standardized formally on two patch installers and they are ‘update.exe – for the OS’ and ‘MSI 3.0 – for everything else’. We’ve tried to make them as similar as possible, but I assure you the OS installer can stop services and restart them if it’s configured to do so via the INF file.

    4. Finally – your comment about Linux isn’t quite apples to apples. To be fair like Windows, Linux has a kernel and the concept of device drivers (which they call ‘loadable kernel modules’). What you are talking about doing is updating the equivalent of a device driver and you can certainly load and unload Linux loadable kernel modules without requiring a recompile or a restart. To be fair on Windows you can also unload and reload some device drivers without requiring a restart (some, not all). What I was referring too was patching the Linux kernel itself – not an LKM. I’m far from a Linux afficionado but I am told that patching the Linux kernel itself requires you to download the latest kernel, compile it and then update the one on your box – which DOES require a restart. The amusing thing (to me) is that lots of people I know who run Linux don’t keep up with the Linux kernel security updates (probably because there are so many Linux sites and advisories are released by so many people its a lot harder). In fact just this week there was a slew of Linux kernel security fixes – I don’t know if they require a restart. I’m sure you can also get them in pre-compiled form so all you have to do is download them and not recompile . . .

  21. Alex Harden says:

    Great conversation. I’ll concede the Linux points.

    Hopefully the product groups you mention are reading this and will choose to build the functionality you’ve described into future packages. Right now, for the majority of hotfix packages, they’re not anywhere close.


  22. Robert Hensing says:

    Thanks – right now, most of the kernel updates we release, unfortunately really do require reboots (even if the update is primarily to a device driver like mrxsmb.sys – which can’t be unloaded / reloaded AFAIK). We’ll eliminate hopefully upwards of 30% of these reboots using hotpatching on WS2003 SP1 (yet another reason to ugprade!).

    You should definitely be suspicious of any update to an application like Windows Messenger, MSN Messenger, Media Player, Office or any other app like that – if it says it requires a restart. 🙂 It most likely does NOT and you can simply shut down the application, apply the update and then restart. 🙂

  23. Toggler says:

    As a Microsoft insider can you confirm that the study by Dr. Ford and Dr. Thompson was not sponsored by Microsoft and was totally independent.

    The VNUNet article says that the article was by Linux enthusiasts thus implying that they are impartial and hate coming to this conclusion.

    But both these guys work for organizations that have either ongoing sponsorship from Microsoft or have large contracts to do research/testing for Microsoft. (Go to their web sites)

    This doesn’t mean that the results are wrong. But we shouldn’t think of them as Linux enthusiasts who have seen the light at last and had a Religious conversion.

  24. Robert Hensing says:

    I actually was just as surprised by this announcement as the rest of the world – I had no idea this was going down at RSA – I don’t think it was sponsored by anyone (I thought the articles called that out).

    I’m pretty sure all they did was look at public data and present it at a conferrene under the guise of a friendly ‘bet’ between the two parties – as such it should be pretty easy to prove – one great source of public data is secunia which I linked too in the blog post.

  25. Bruce McLeod's Weblog says:

    Trackback Ping

  26. Norman Diamond says:

    > I have to admit, I used the dramatic title

    > to try and get the Linux followers who may

    > be subscribed to my blog ‘out of curiosity’

    > whipped into a frenzy

    OK, you asked for it, you got it, though I’ve posted the same opinion in various other places including recently in the blog of one of your colleagues.

    There are two essential differences between Linux and Windows:

    (1) With Linux, you DO get what you paid for (except if you paid for it).

    (2) With Linux, if something is broken, then if you’re a programmer you DO have a snowball’s chance in hell of getting it fixed.

    One time I temporarily added a third, that Linux never destroyed the entire contents of a hard disk partition other than when I told it to. But someone replied saying that Linux did it to him, so it seems that Linux and Windows do not differ this way.

  27. greg says:

    So, (1) would imply that with Linux you get nothing – as that’s what you paid. This, of course is not true.

    And as for (2), hoestly, how many sysadmins can or want to write code? And if they do, will they write it securely and be able to test it well? Not likely.

    And one for Robert. Your note about the "update product" is insightful. however, MS Mgmt should demand that product groups use other update effectively and to the customer’s best interest.


    Windows & Slackware user

  28. anonymous cowhard says:

    You can exec kernel with kexec after update, without restart

  29. joe smoe says:

    funny… microsoft has 11% un-patched

    redhat has 0% un-patched…

  30. jim says:

    no, the funny thing was supposed to be the vast difference in the overall numbers, RH vs MSFT.

    It’s easy to SAY 0% un-patched when you have a whole team of uhh…experts?…spread the world over releasing patches whenever they *think* they’ve fixed it.