Network Managing, Engineering and Administration with an eye toward security.
Best practices meets the real world!
Wednesday, January 26, 2005
Linux -vs- Windows, an age old argument I've been doing some investigation lately into the security aspects of Linux, BSD, Solaris, and Windows. Before I go too much further let me just say that all of them have their merits, and to a large degree any of them can be configured securely.
One of the things I've been looking at is the number of vulnerabilities that a product has and the severity of the vulnerabilities. What exactly have I been looking into, you ask? I've seen a number of folks discuss the merits of Windows -vs- Linux and seen little data to back it up. Usually it's WIndows -vs- Red Hat, or some such nonsense. It is especially difficult to come up with clear numbers on vulnerabilities on these products as they are "bundled'" differently. IE is built into Windows, so do IE vulnerabilites get included, or are they a seperate item? Most Linux distros have a large number (some can install upwards of 500) of applications that can be installed with them, do they get included, or are they seperate items?
For a while now I've had to keep up with the security patches for our servers. Since we really don't run Linux, and there are so many different versions of Linux to track I thought it's really difficult to trace the differences in security among the versions. Then I thought about Secunia and their database. Seemed to me that this was a good place to start. I pulled together some information from their database, querying for several different products. The search included Windows 2000 (Pro), XP (Home), 2000 Server, 2003 Server, RedHat 9, Debian 3, SuSE 9, Gentoo 1, RedHat ES 3, SuSE ES 8, Solaris 9, Open BSD 3, and Free BSD 5. I was thinking this was a good representation, and pulled the data which is cataloged quite nicely. Here's a chart of some of what I've found:
A couple of things kind of struck me first off when going over this information. First was the number of vunlnerabilities that the Linux/BSD/Solaris systems had. Then I started digging into them, and as we discussed earlier these include hundreds of applications that Secunia included (which is fine) are counted here. The second thing I noticed was the patched percentage. The BSDs had a 100% patched, and many of the Linux variants had 100%, with some just missing at 99% (which may be how Secunia classified them - which again is ok). Now here is a chart showing the Linux/BSD/Solaris -vs- Windows numbers in breakdown:
Notice here the discrepancies between the Unpatched column and the Patched column. Now, this appears that Windows has around 20% more unpatched issues than Linux/BSD/Solaris average. Given that there were twice as many vulnerabilities with the Linux/BSD average that's a significant number of Windows advisories that have not been patched. I then checked out the criticality of the advisories, and here is what I found:
When I started looking into the browsers that's when I knew that the numbers like this were just useless. I compared IE6 with Firefox 1, Opera 7, and Safari 1. Here are the numbers that I got:
After seeing this I thought, wow one of the alternative browsers is really bad, and I found it in a suprising place: Firefox. According to Secunia, Firefox has 6 advisories and 83% of them are unpatched. Then I started digging into why, because I've been a Firefox fan for a while now and haven't found anything that I personally didn't like (e.g. it was my favorite browser) and it always seemed to be much more secure than IE. That's when I found that several of the 6 vulnerabilities (that do need to be fixed) are systematic of issues in the standards, and are also listed under the other browsers. I also found that several of the unpatched items are of a low security risk (in fact, Firefox had no issues of a High security risk in the Secunia advisories). This of course skews the results (for all of the products), but also gives the appearance that Firefox is less secure than it really is.
That's when I started realizing that it's really impossible to do this kind of testing. The only kind of testing that could be done like this is comparing the Linux distros to each other, or comparing the BSDs to each other, or comparing Windows to each other. It's too easy to skew the numbers one way or the other depending on how you want the results to play out, and there are too many caveats for those that don't have an agenda (the addendum would be as large as the research paper).
The numbers are interesting in and of themselves, but they prove nothing. I still find them fascinating, and may continue to see if there is a way to level the playing field (hmm, adding advisories from CERT?) but I doubt there is a way as each database of advisories is different in how they categorize issues and some don't include issues that are included in others.
Just thought you might want to see some of this.
The Reigster - Security Report: Windows vs Linux
Secunia - Vendor Advisories
NewsForge - Linux and Windows security compared
posted by David | 1/26/2005 08:01:00 PM