Microsoft Forefront Protection 2010 for Exchange Server Virus Bulletin test results

I have been wanting to tell you about Forefront product testing by independent evaluators for a long time but have not had a chance to do so due to being very busy working on other projects (Oh boy, this really sounds like a lame excuse…). 

Forefront has already received many Gold/Platinum Awards and Certifications from leading independent testing organizations like West Coast Labs (Premium Anti-Spam Certification).  Now the moment has finally arrived for me to write this blog, and the trigger is that Forefront Protection 2010 for Exchange Server received its 4th successive award from Virus Bulletin.  

In the last round of testing for the March publication of Virus Bulletin, Forefront Protection 2010 for Exchange Server clearly exhibited outstanding performance and produced stellar results. Let me quote from the bulletin:

“One of the top performers in the previous test, Microsoft’s Forefront Protection 2010 for Exchange Server saw its performance improve even further and the product outperformed its competitors in all spam categories. Thanks to just four false positives, Forefront was the only product to achieve a final score of over 99%”. 

This is great to see and results like this add credibility to the antispam protection provided in our newest flagship product, which was released in November 2009, and enhance our trustworthiness among customers.   The antispam vendors’ world is a very tough and competitive place with several companies delivering very strong products capable of producing impressive results. 

So what’s behind the Forefront numbers and how we were able to outperform every competitor?  Looking behind the numbers, it is clear that all of the participating antispam products performed well, but the biggest differentiator was the fact that Forefront did extremely well on catching spam without generating many False Positives (FPs).  There were only four FPs over the course of a rigorous three week test by Virus Bulletin! While missed spam is simply unacceptable by the end users, incorrectly classified good mail could mean lost business opportunities and is much less tolerable in the messaging community.  Having only four FPs over three weeks of testing says a lot about the efforts our engineering team put in place to drastically improve in this area and outperform competition. 

We saw similar results for missed spam or False Negatives.  The overall median for False Negatives for the products tested (without the Forefront) was 3867 missed spam messages. In contrast, Forefront missed only 602 spam messages.  This is more than six times better than the average!  The nearest competitor’s product missed 1717 spam messages which is almost three times more than Forefront!  Forefront’s Catch Rate (CR) was 99.86% on live spam and the rest of the tested products were only around low 99% or below 99%.  These days anything less than 99% is just not enough to meet the enterprise-grade quality requirements, and during the 4 consecutive rounds of testing Forefront never got below 99% in spam CR.  Here is the historical chart for the Forefront’s catch rate performance:

Spam Catch Rate Chart

You might ask what’s the difference between the Overall spam CR and Live spam CR?  In a nutshell, live spam arrives from the Project Honey Pot in real time and it represents the latest trends and spam attack vectors in the real world in real time.  Scoring 99.86% in this category is simply outstanding, but to add a little more to this – Forefront also scored 99.86% CR on images and, as we all know, images as a spam distribution vehicle are highly popular in the recent spam attacks. For more information about the testing methodology, please read the Virus Bulletin article that describes the methodology used for their comparative antispam testing.    The chart below shows comparative results for Forefront and other products tested:

VB Results Chart

So why is Forefront such as strong performer and what makes it a leader in the antispam area?  It all starts with vigorous internal product development and testing.  Forefront Protection 2010 for Exchange Server is unique because it is built-in not bolted-on to Exchange 2010 server; it’s an integral part of the end-to-end messaging pipeline and processing where the bits of information are mutually available and shared between the Exchange server and Forefront. 

Being a layered messaging hygiene solution, Forefront exposes multiple technologies to rid the messaging stream of malware and spam.  Starting with the beginning of a mail transaction, Forefront acts on an incoming connection request from a remote party to decide whether to accept the connection request or deny it.  This layer of protection encompasses multiple Safe, Block, and Reputation Lists and Forefront exclusively delivers a technology not available in any other product – a combined real-time block list. 

Forefront aggregates multiple feeds from various external and internal block list providers, combines them into a single database, and hosts that data in its own datacenter for use by all Forefront consumers free of charge.  This technology itself is capable of contributing around 90% of spam rejections. One of the biggest benefits of having this feature in Forefront is that it helps to greatly reduce the carbon footprint of spam. 

We often think of various messaging hygiene commodities as simply means to defend our production environments from threats but not always realize that the rejection place is extremely important.  The sooner you reject junk the better, because there is no real need to push all the unnecessary garbage payloads through the messaging and network layers and clog up communication channels, which increases the overall cost of implementation and reduces message throughput.  Again, the sooner you reject spam the better and Forefront is right there on the front line of early spam identification. 

The next layer in line is SMTP filtering.  While most of the modern antispam solutions have various SMTP filtering capabilities, Forefront is uniquely positioned to take advantage of the end user data (data which is aggregated from the end users mailboxes, such as Safe Senders/Safe Recipients Lists).  Forefront, being deeply integrated into the messaging infrastructure of Exchange, gets this data in real time and makes correct decisions regarding message acceptance based on an individual recipient.  In this regard, Forefront provides not only data-centric protection (what content to protect from/accept) but enforces user-centric processing (for whom to accept the data and for whom to reject it) of the messaging stream.  Having this amalgamation of sometimes mutually exclusive configurations allows Forefront to make decisions based on individual recipient settings during the real-time scanning.  In addition, technologies like SenderID and Backscatter filtering greatly reduce spoofing attacks and eliminate backscatter penetration. 

Content filtering is another layer where Forefront shows exceptional quality in its spam categorizations.  While the infrastructure around the content filter and its adapter layers were created by the core Forefront product development team, at the heart of content filtering lies the Cloudmark Authority Engine.  Highly efficient and accurate, Forefront Content Filter removes the last remaining pieces of spam without generating a large number of FPs.

One of the most appealing Forefront qualities is its ease of setup and administration.  By default, after installation Forefront comes on-line with all of its antispam shields up (except the Backscatter filter) and has smart defaults to allow for protection to start immediately after the setup without the need of manual intervention from the administrator.  It’s easy to manage and most importantly to monitor.  All the major statistics for both malware and spam are available right at the administrators fingertips on the Forefront dashboard. 

All of the great features and innovative antispam technologies in Forefront translate into the following VBSpam Quadrant (available here):

VB Quadrant chart

Forefront Protection 2010 for Exchange Server is clearly on the top of the game, so give it a try if you haven’t done so yet.  Will it solve all of your messaging hygiene problems and headaches, is it a new panacea or silver bullet against spam?  Of course not.  But in the ever-changing and fierce battle spammers unleash against your messaging environment, you want to enlist the most effective and reliable defense and partner.  As Virus Bulletin outlines in its latest release, Forefront outperformed its competitors in all spam categories.  There is a whole development and testing team behind this success and the team constantly monitors spam attacks/latest trends and develops the best in class protection as threats emerge.  We are here to help you, and I’m sure together we can be successful and win against spammers. 

I naturally do not trust words and all marketing buzz, but I do trust data.  The data from VB speaks better than any words and shows the new kid on the antispam block has arrived!  Say hello to the emerging antispam leader, Forefront Protection 2010 for Exchange server.

Alex Nikolayev

Program Manager  - Forefront Server Protection

Comments (3)

  1. RichardW says:

    Very satisfied about the product, but missing reporting capabilites. When does Forefront Protection Manager become available? Is the a beta version available for testing?

  2. thomas says:

    Very good product…Its catching most of SPAM mails..Now we are protected 99% with this this software..Dash board and SPAM summary is good..Like Ricard told..Reporting feature is missing with this prodcut..Hope microsoft will realse son enable reporting feature..

  3. Kirill says:

    Sorry for question, where the Trend Micro Scan Mail for Exchange in this graphs?

Skip to main content