Enterprise Mobility and Security Blog

RSS

Intro

ATA analyzes and learns user and entity behavior by aggregating data from various data sources, such as deep packet inspection of domain-controller traffic, windows events, and data provided by SIEM systems. After ATA begins gathering information about Active Directory traffic and correlating that information with AD components, it will scan for abnormal behavior and suspicious activities. ATA alerts on three different categories of detection: security issues and risks, malicious attacks and abnormal behavior.

While the deterministic attacks such as account enumeration and PtT can be surfaced immediately as they occur, the abnormal detection engine has some requirements to build the model. ATA continuously learns from the organizational entity behavior and adjusts itself to reflect the changes in the enterprise. Information such as resources users access, where they are accessed from, and date and time of access is analyzed. The anomaly detection engine is based on a combination of association rule mining and decision trees. Based on this analysis ATA builds an organizational graph and starts detecting security issues, advanced attacks and abnormal entity behavior. A common question raised by customers is, how do they confirm the abnormal detection engine is running and validate it’s working properly?

 

Requirements

ATA behavioral analytics uses machine learning to detect suspicious activities in the organization. The abnormal behavior detection engine requires a minimum of 3 active weeks (an active week has to have at least 4 days of authentication activity) to build a behavioral profile for an entity and it requires a minimum of 50 entity profiles to build the organizational model in order to be able to detect any abnormal behavior in the organization. This can include 50 active “human” user profiles and service accounts.

 

Validation

ATA provides a variety of logs to provide insight into the different detections which it monitors. On the ATA Center server there is a detection log file named, Microsoft.Tri.Center-Detection.log which by default is located in C:\Program Files\Microsoft Advanced Threat Analytics\Center\Logs folder. This log contains details on detection progress and debug information. For more information about ATA log files see, https://technet.microsoft.com/en-us/library/mt637889.aspx.

 

Once ATA is able to validate the abnormal detection engine requirements outlined above, the detection log will show an entry for “[AbnormalBehaviorDetector]Building a Model.” ATA captures the information on the number of users whose behavioral profiles have been completed in this same log. This can take some time depending on the size of the customer and number of accounts. Portion of the log with this entry is shown below:

2016-06-28 15:43:54.4510 3564 21 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] Building a model [NumberOfAccounts=81 iteration=0

accountIdsToExclude=]

2016-06-28 15:45:02.9540 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] Decision Tree:

SourceComputer_New <= 1.5

Resource_New <= 5.69166666666667 [Label=0 Confidence=0.5

ProbabilityOfAbnormal=0]

Resource_New > 5.69166666666667 [Label=1 Confidence=0.166666666666667

ProbabilityOfAbnormal=1]

SourceComputer_New > 1.5 [Label=1 Confidence=0.333333333333333

ProbabilityOfAbnormal=1]

 

After the model is created, it is validated multiple times which can take some time to complete. Validation entries are shown in the portion of the log below.

2016-06-28 15:45:02.9540 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] Validating tree using simple validation with the training data

[StartTime=06/14/2016 00:00:00 EndTime=06/21/2016 00:00:00]

2016-06-28 15:45:02.9600 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] [Results=TP:3402 FP:0, FN:0 TN:3402 Recall=1 FalsePositiveRate=0]

2016-06-28 15:45:02.9945 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] Decision Tree after pruning with training data:

SourceComputer_New <= 1.5

Resource_New <= 5.69166666666667 [Label=0 Confidence=0.5

ProbabilityOfAbnormal=0]

Resource_New > 5.69166666666667 [Label=1 Confidence=0.166666666666667

ProbabilityOfAbnormal=1]

SourceComputer_New > 1.5 [Label=1 Confidence=0.333333333333333

ProbabilityOfAbnormal=1]

 

2016-06-28 15:45:02.9945 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] Validating tree using cross validation with the training data

[StartTime=06/14/2016 00:00:00 EndTime=06/21/2016 00:00:00]

2016-06-28 15:45:03.4065 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] [Results=TP:681 FP:0, FN:0 TN:680 Recall=1 FalsePositiveRate=0]

2016-06-28 15:45:03.7880 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] [Results=TP:680 FP:0, FN:0 TN:680 Recall=1 FalsePositiveRate=0]

2016-06-28 15:45:04.1605 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] [Results=TP:680 FP:0, FN:0 TN:681 Recall=1 FalsePositiveRate=0]

2016-06-28 15:45:04.5470 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] [Results=TP:681 FP:0, FN:0 TN:680 Recall=1 FalsePositiveRate=0]

2016-06-28 15:45:04.9540 3564 10 8b11a05d-c15f-4285-bf7c-b214264bd8ba Debug

[AbnormalBehaviorDetector] [Results=TP:680 FP:0, FN:0 TN:681 Recall=1 FalsePositiveRate=0]

 

Once the validation process is completed we will see another entry in the detection log called “[AbnormalBehaviorDetector]Validation process ended successfully.” Note that these entries may not be shown consecutively in the log. At this point the Abnormal Behavior Detection is running using the model created. Everyday a new model is created and validated. Accounts might be added or removed from the model depending on their activity

2016-06-28 15:51:29.6666 3160 20 aa095d1b-7fa5-4c67-a63a-4c7cdea187f8 Debug

[AbnormalBehaviorDetector] Validating tree using simple validation with the validation data

[StartTime=06/21/2016 00:00:00 EndTime=06/28/2016 00:00:00]

2016-06-28 15:51:29.6676 3160 20 aa095d1b-7fa5-4c67-a63a-4c7cdea187f8 Debug

[AbnormalBehaviorDetector] [Results=TP:3402 FP:0, FN:0 TN:3402 Recall=1 FalsePositiveRate=0]

2016-06-28 15:51:29.6676 3160 20 aa095d1b-7fa5-4c67-a63a-4c7cdea187f8 Debug

[AbnormalBehaviorDetector] Validation process ended successfully

2016-06-28 15:51:29.6706 3160 20 aa095d1b-7fa5-4c67-a63a-4c7cdea187f8 Debug

[AbnormalBehaviorDetector] Decision Tree after refining:

SourceComputer_New <= 2

Resource_New <= 5.69166666666667 [Label=0 Confidence=0.5

ProbabilityOfAbnormal=0]

Resource_New > 5.69166666666667 [Label=1 Confidence=0.166666666666667

ProbabilityOfAbnormal=1]

SourceComputer_New > 2 [Label=1 Confidence=0.333333333333333

ProbabilityOfAbnormal=1]

 

Finally, the log shows an entry when a suspicious alert is raised and the evidence used to raise the alert.

2016-06-28 16:17:22.3136 2392 6   5ee3c263-436a-4cb7-be56-0e104d886e0f Debug

[AbnormalBehaviorDetector] Running behavioral detection for 81 human user accounts

2016-06-28 16:17:22.3251 2392 6   5ee3c263-436a-4cb7-be56-0e104d886e0f Info

[AbnormalBehaviorDetector] A model already exists [AccountType=HumanUser

CreationTime=06/28/2016 15:51:29]

2016-06-28 16:17:26.3053 2392 10 962d9219-80d7-425e-b0c2-c48e11f822b0 Debug

[AbnormalBehaviorDetector] Found abnormal row [Key=SourceAccountId=d8192f8d-3819-

4161-a82d-11a086b26fc4 Date=06/28/2016 00:00:00 AttackSimulationType=None Label=0,

FeatureVector=11, 1, 11, 0.920502092050209, 10]

2016-06-28 16:17:35.8527 2392 23 5f3742e4-bb7d-4a69-a2e4-9608c09e603c Info

[AbnormalBehaviorDetector] Wendel Robertson (Software Engineer) exhibited abnormal

behavior when performing activities that were not seen over the last month and are also not in

accordance with the activities of other accounts in the organization. The abnormal behavior is

based on the following activities:

Performed interactive login from 11 abnormal workstations.

Requested access to 11 abnormal resources.

Exceeded the normal amount of working hours.

 

FAQ

1. How can I validate that a single user is included in the behavior analysis model?

ATA behavior analysis model will include users who have consistent network activity (4 days a week during the last 3 weeks). To confirm the user is part of the model, we will retrieve the associated userID and validate that profile exists for him with the following steps:

  • From a command prompt on the ATA center, go to c:\Program files\Microsoft Advanced Threat Analytics\Center\MongoDB\bin
  • Output the list of entities to a file to retrieve the userID in question
    mongo ATA –eval “printjson(db.UniqueEntities.find().toArray())” >> ATAentities.json
  • Open the ATAentities.json file from the .\bin folder in notepad
  • Use notepad Find feature to search for string: “Name” : “<UserName>”
  • Copy the value of the “_id” field to your clipboard

Blog Behavior Blog Pic 1

  • Output the list of profiles to a file and search for userID to confirm user has a profile.
    mongo ATA –eval “printjson(db.Profiles.find().toArray())” >> ATAprofiles.json
  • Use the Find feature in Notepad to search for string – “UniqueEntityId” : “<paste from clipboard>”

Example: “UniqueEntityId” : “80aafeef-c589-4ad4-832e-b1a1d4853784”

Blog Behavior Blog Pic 2

In this case the administrator account is indeed part of the behavior model since his unique entity ID is included in the profiles built.

For questions or feedback contact me directly:
Shalini Pasupneti
shpasupn@microsoft.com
Senior Program Manager
C+E Security CxP Team