Testing FCS antimalware detection with your own library
During evaluation of the Forefront Client Security antimalware protection many customers will review the information provided by independent antimalware testers such as
(When reading these sites, note that Microsoft’s Forefront Client Security and OneCare products use the same malware protection engine and definitions)
Other customers may want to test FCS detection capabilities against their own private malware library. Doing this with a large library can present somewhat of a challenge because the user interface can becoming quite filled and slow to respond when filled with thousands of entries. Alternatively, you might try to examine the event log for 3004 events(real-time protection detected threats) and/or 1006 events (scan detected threats).
The problem with both the user interface and the event log query is that FCS will consolidate threats of the same type into a single infection report. Therefore, if you have 5,000 sample files but only 4,500 are unique (e.g. you have multiple EICARs or multiple RBOTs.gens) then you will get a maximum of 4,000 detection events or items in the UI. Why does FCS do this aggregation? It is done because in a typical “real-world environment” administrators will act on malware as a whole rather than have different actions for different instances, components, or files of a malware threat. It also ensures removal is done properly and threats are not only partially addressed.
To illustrate this behavior, I used a single copy of EICAR and nine of the different SpyCar samples in a test with FCS. So although I have 10 total files, FCS shows the following:
Notice that FCS says that only 2 items were detected. To a casual reviewer unfamiliar with the FCS behavior described above this would leave the impression that FCS only detected 20% of the files. While it is easy to understand what is happening in this trivial example, it is more difficult in a larger library.
By investigating the detection further, and scrolling down, we can see that the files for each threat are listed as resources:
This is also true of the event log entries:
Log Name: System
Source: FCSAM
Date: 4/15/2009 3:36:21 PM
Event ID: 1006
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: computer.microsoft.com
Description:Microsoft Forefront Client Security scan has detected spyware or other potentially unwanted software.
For more information please see the following:
https://go.microsoft.com/fwlink/?linkid=37020\&name=Tool:Win32/SpycarTestSuite\&threatid=17424
Scan ID: {FF8CDDEA-1CE8-4644-97FB-55C5C36329C9}
Scan Type: AntiMalware
Scan Parameters: Custom Scan
User: DOMAIN\craigw
Name: Tool:Win32/SpycarTestSuite
ID: 17424
Severity: Severe
Category: Tool
Path Found: file:C:\fcstest\IE-SetSearchPage.exe;file:C:\fcstest\IE-SetHomePage.exe;file:C:\fcstest\IE-KillAdvancedTab.exe;file:C:\fcstest\IE-HomePageLock.exe;file:C:\fcstest\HKLM_RunOnceEx.exe;file:C:\fcstest\HKLM_RunOnce.exe; file:C:\fcstest\HKLM_Run.exe;file:C:\fcstest\HKCU_Run.exe;file:C:\fcstest\AlterHostsFile.exe
Detection Type: Concrete
So to properly determine the number of files (not different threats) you would need to find each detection and count the number of file based resources associated with it. This is clearly a challenging undertaking for a large malware library. It can be helped somewhat by leveraging the MpLog-xxx-xxx.log file found in the %ALLUSERSPROFILE%\Application Data\microsoft\Microsoft Forefront\Client Security\Client\Antimalware\Support directory. It will list the total number of threats found, and then the count of resources for each threat:
Product Version: 1.5.1958.0
Engine Version: 1.1.4502.0
AS Signature Version: 1.55.1678.0
AV Signature Version: 1.55.1678.0
************************************************************
Begin Resource Scan
Scan ID:{FF8CDDEA-1CE8-4644-97FB-55C5C36329C9}
Scan Source:1
Start Time:Wed Apr 15 2009 15:35:30
End Time:Wed Apr 15 2009 15:36:21
Explicit resource to scan
Resource Schema:folder
Resource Path:@S-1-5-21-124525095-708259637-1543119021-1851\C:\fcstest\
Threat Count:2Threat Name:Tool:Win32/SpycarTestSuite
ID:17424\
Severity:5
Number of Resources:9
Resource Schema:file
Resource Path:C:\fcstest\IE-SetSearchPage.exe
SigSeq:37829992017238
Resource Schema:file
Resource Path:C:\fcstest\IE-SetHomePage.exe
SigSeq:37827904169718
Resource Schema:file
Resource Path:C:\fcstest\IE-KillAdvancedTab.exe
SigSeq:37829556567249
Resource Schema:file
Resource Path:C:\fcstest\IE-HomePageLock.exe
SigSeq:37827406791016
Resource Schema:file
Resource Path:C:\fcstest\HKLM_RunOnceEx.exe
SigSeq:37828071703283
Resource Schema:file
Resource Path:C:\fcstest\HKLM_RunOnce.exe
SigSeq:37829705680185
Resource Schema:file
Resource Path:C:\fcstest\HKLM_Run.exe
SigSeq:37827006632665
Resource Schema:file
Resource Path:C:\fcstest\HKCU_Run.exe
SigSeq:37825853825063
Resource Schema:file
Resource Path:C:\fcstest\AlterHostsFile.exe
SigSeq:37826058080799Threat Name:Virus:DOS/EICAR_Test_File
ID:2147519003
Severity:5
Number of Resources:1
Resource Schema:file
Resource Path:C:\fcstest\eicar.com
SigSeq:5866324352432
End Scan
************************************************************
When we sum up the detected resources we see that FCS did indeed detect all 10 files.
Another testing approach would be to set all of the default actions to remove via the FCS client Tools > Options UI:
Then stop the FCS service(net stop FCSAM), copy the malware library to the machine, restart the FCS service(net start FCSAM), run a custom scan of the folder, and then choose to Apply Actions in the detection dialog. This should remove all the threats and resources detected. If any threats remain in the folder after the action has completed that would indicate that either there was no detection or there was a failure to remove them. Failures would be reflected in both the UI and the event log.
If you find malware that FCS does not detect or detects but has problems removing, please submit it through the Microsoft Malware Protection Center portal.
Thanks,
Craig Wiand
Microsoft Forefront Escalation Engineer