Federated to Microsoft Cloud and Account Lockouts


Bad People Do Bad Things

Sometimes brute force style bad password submissions against organizations which use Microsoft Cloud services occur. These malicious routines can take place against companies which use federated single sign on (via Windows Server Active Directory Federated Services a.k.a. ADFS or other federated services).

Malicious entities can target highly visible users like executives or managers. This kind of malicious attempt is most effective if your organization uses Active Directory account lockout policies to lock user accounts on premise if a certain number of bad passwords are submitted. If you have strong passwords enforced, the issue quickly becomes less about the possibility of someone brute forcing the password and more about the account being locked out. This turns into a denial of service to the users as their accounts are locked out and the users are not able to authenticate on-premises or to their cloud services.

How is this possible?

When users are federated (meaning that their user principal (UPN) name matches one which has been configured in Azure AD/Office 365 for federated single sign on) the Microsoft Cloud forwards the authentication requests to the on-premises federated servers to verify the user’s credentials. In the case of Windows ADFS, ADFS will immediately attempt a credential check against the user. If there is a bad password submitted for an identity that bad password will count against any configured Active Directory domain bad password attempt. Other federated services typically work in the same manner and increment the bad password count as well. If an Active Directory domain has a very low account lockout threshold then the user may be locked out in short order.

Malicious bad password attempts combined with an account lockout threshold will result in account lockouts and effective (and perhaps intentional) denial of service to users where they will not be able to access on-premises resources nor Microsoft Cloud services due to their account being locked out.

How do you protect against it?

There are several things to do in different stages of the incident.

Identify Bad Guys and Identities Targeted

The first thing is to identify where the brute force attempts are coming from and what identities they are targeting.

Turn on ADFS Auditing

ADFS can be configured for security auditing and service verbose events. These events will show information about the accounts which are being targeted and the IP address of the malicious client doing it.

  1. The TechNet article on how to manually configure ADFS servers for auditing is here. If you have a load balancer for your ADFS farm, as most people do, you will need to enable this auditing on each ADFS server in the farm. This does not need to be configured on the Web Application Proxy servers.
    1. A PowerShell script is also available on TechNet here which can be used to automate the audit configuration being enabled.
  2. If you have Windows Server 2012 R2 ADFS, you can search All ADFS Servers’ Security event logs for Event ID 411 Source ADFS Auditing events.
    1. You can download the PowerShell script to search your ADFS servers for events 411 at this link. The script will provide a CSV file which contains the UserPrincipalName, IP address of submitter, and time of all bad credential submissions to your ADFS farm.
    2. You can open the CSV in Excel and quickly filter by username, or IP or times.
    3. More information on the 411 events themselves:
      1. These events will contain the user principal name (UPN) of the targeted user.
      2. These events will also contain a message “token validation failed” and will say if it was a bad password attempt or the account is locked out.
      3. There will be one per brute force attempt-there may be a lot.
    4. If your server has 411 events showing up but the IP address field isn’t in the event make sure you have the latest ADFS hotfix on your servers.
      1. More information can be found in KB3134222.
  3. If you have Windows Server 2008 R2 or Windows Server 2012 ADFS you will not have the needed Event 411 details. Instead, download and run the PowerShell script below to correlate security events Security Event 4625 (bad password attempts) and 501 (ADFS audit details) together to find the details for the affected users.
    1. You can download the PowerShell script to search your ADFS servers for events at this link. The script will provide a CSV file which contains the UserPrincipalName, IP address of submitter, and time of all bad credential submissions to your ADFS farm.
    2. You can also use this method if you are planning on discovery of what connections are taking place successfully for the users in the 411 events you can search the ADFS events 501 for more detail.
    3. When running the PowerShell script to search your events, just pass the UPN of the user identified in the 411 event(s) or by account lockout reports to your helpdesk.
    4. The IP address of the malicious submitters will appear in one of two different fields in the 501 events.
      1. For web based and most application authentication scenarios the malicious IP will be in the x-ms-client-ip field.

Note

For non-Modern Authentication Outlook clients, the IP address of the malicious submitter will be in the x-ms-forwarded-client-ip and Microsoft Exchange Online server IPs will be in the x-ms-client-ip value.

This is a result of “legacy” or Basic authentication having the Exchange Online servers in the cloud proxying the authentication verification on behalf of the Outlook client. Mail clients which support Modern Authentication (aka ADAL) will not proxy the auth this way.

More information on Modern Authentication and Basic Authentication Office 365 scenarios can be found online here.

Mitigation: Block the Bad Guys

The best way to address this scenario is the same way you would address a denial of service to a public web site: block the IP address(es) of the submitters at the network level (firewall). This approach is basically the same one you would take if the scenario was your website being targeted for brute force or denial of service attacks.

For federated single sign on, the best practice is to have the proxy servers for ADFS (or any federated service) in a DMZ. Since DMZs have network traffic rules it would make sense to add a blocking rule at the DMZ to prevent the traffic from the suspect IPs from ever reaching the Web Application Proxy (WAP) servers. However, some environments make this an easier approach to do in the load balancer, or even at the internet service provider (ISP). The net result is to prevent the brute force traffic from ever reaching the ADFS servers in the first place.

The only fly in the ointment in the traffic blocking scenario is that “legacy” Exchange Online authentication mentioned above which will actually proxy the authentication of the thick “Basic Authentication” Outlook clients to the on premise ADFS servers. This scenario sounds complicated but really only means that Outlook connects to the Exchange Online servers and then the Exchange Online servers authenticate to your on premise ADFS servers on the users behalf. So there is a connection from the Exchange servers IP address as the client.

In that instance blocking the malicious IP address at the DMZ perimeter won’t work since the IP address is actually a Microsoft Exchange Online one. Instead you would need to either examine your ADFS server’s events for the IP address in the x-ms-forwarded-client-ip value or do SSL termination on your network before the Web Application Proxy servers and review the data on a network device. Again, this requires SSL termination in a network device prior to the WAP servers.

If your environment is seeing Exchange Online Basic Authentication being used to pass along brute force attempts then you should also consider disabling POP and IMAP protocols for the targeted users.

In addition to the account lockouts that you were experiencing due to constant Exchange Active Sync Authentication requests we also identified that there are similar types of requests on the IMAP and POP protocol currently occurring.

Our recommendation is to disable these protocols if they are not needed for the users.  This will block the requests at the service side preventing it from being forwarded to your ADFS servers which will reduce the available attack service.

Unless you have an account that specifically relies on it you can disable IMAP and the POP protocols across all your mailboxes by running the following command:
get-mailbox | Set-CasMailbox -PopEnabled $False -ImapEnabled $False

This action should also help to mitigate the issues related around these protocols that also rely on basic authentication and may not be needed in your environment.

The GUI method in the Office 365 Portal is as follows: Disable IMAP & POP
Mitigation: Refine Sign In Security

Review AD Account Lockout Threshold, Implement ADFS Extranet Lockout feature, Enable MFA

Password security is important, so this should start with a review your organization’s password complexity requirement to make sure passwords are sufficiently complex and cannot be quickly guessed via brute force attempts.

Once that is done review and consider raising the current Active Directory on-premises account lockout threshold to determine whether the current setting is providing sufficient security to prevent password guessing while at the same time preventing intentional or unintentional account lockouts from bad password attempts which will result in denial of services. This topic is discussed on TechNet here.

If your organization is seeing targeted account lockouts to a specific subset of users you may be very granular in addressing the concern by using Fine Grained Password Policy settings. This feature essentially allows for assignment of a different password complexity and account lockout configuration for a security group or specific users.

In my experience, organizations discover that the on-premises Active Directory account lockout threshold is set too low when there is a rash of maliciously based account lockouts or unintentional account lockouts from an application. In this scenario, I encourage reviewing the balance between preventing password guessing via brute force and allowing denial of services for users since their account had too many bad password attempts.

When using ADFS, it is also important to implement the ADFS soft extranet lockout feature. This feature will help mitigate large numbers of attempts from locking out accounts since it is a lower threshold than what you have defined in AD and will cause ADFS to stop forwarding the bad password attempts to AD. It is not a resolution in itself since each user accounts bad password count is never removed or decremented till an unlock event or until the “reset account lockout counter” setting kicks in (if you have configured it). Over time the account may still be locked out but the extranet lockout will delay the lockout. ADFS Extranet Lockout is documented here on TechNet.

Additionally, it’s important to require Multi Factor Authentication (MFA) sign in for users. This can’t be stressed enough as being a useful security item to implement. That additional factor being required for sign in will not prevent the bad password increment-the lockout may still occur. However, the MFA requirement will mitigate the risk if a password is guessed since MFA will still be required to complete the authentication and a malicious user will lack the required additional factor of a phone call, a text message or other method.

Office 365 MFA or Azure AD Multi Factor Authentication (MFA) may be used. If you have your own MFA solution then that works too. In any case, MFA can be very quickly implemented via your cloud services. For Azure AD MFA the setup would require:

  • Assigning an Azure AD Premium license to the user(s)
  • Deciding on MFA settings for complexity, duration of MFA authentication, and a review of the user’s device(s) and applications to ensure their device apps support MFA.

For organizations which need to decide which MFA solution to use, information on the differences between Office 365 and Azure AD MFA can be found online at this link and general pricing information for Azure AD MFA is documented here. Some additional MFA information can be found in Channel 9’s MFA OVerview, in Azure online documentation here, and finally in the MFA Deployment Guide.

If your ADFS farm is Server 2012 R2 then you do not have to do additional steps to simply use Azure AD MFA. However, ADFS allows for on premise control of MFA via claims rules if you would like to implement them. For example, if a user is a member of a security group and they are signing in from extranet you can require MFA auth from them. More information on MFA related claims rules can be found in the blog posts below:

Review, Report, Monitor

It’s important to review your online reports for Office 365 and Azure AD to see what impact (if any) the incident has had to online services and resources. This entails a review Azure AD Security Reports (for alerts on activity deemed to be malicious), Azure AD Audit Reports (for details on changes to anything in Azure AD), and Sign In Activity (which will show when denies attempted sign in, to what and the end result). A review of these reports for activity for and by the affected VIP users to gauge impact of the event is vital to address the concern.

More information on Azure AD reports is online here: https://docs.microsoft.com/en-us/azure/active-directory/active-directory-view-access-usage-reports

Outside of the immediate review it’s also important to implement a consistent routine of automatic download of Azure AD Security Reports, Audit Reports, and Sign In Activity reports.  This can be done by setting a scheduled task on a Windows computer to run PowerShell scripts to pull the reports down periodically for reference.

Just as important is to configure Azure AD to notify global administrators if anomalous sign ins are seen in the future.  This is a switch setting in Azure AD (screenshot below) in the Azure AD web portal:

Long Term Recommendation

Consider moving to a Windows Server 2016 ADFS farm in order to mitigate bad password submission attempts by using the Multi Factor Authentication feature in 2016. This will allow Azure MFA as the primary authentication method.  This prevents the scenario of a low account lockout threshold and malicious bad password attempts via brute force from being a concern.  This is discussed in more detail on TechNet here.

Note that a server 2016 ADFS farm could be in place as a parallel switch-over upgrade or an in place farm upgrade.

To sum things up, scenarios where malicious entities can submit bad password attempts can be a challenge at first. However, like any challenge you may face in life you can build strength and capability in how to address the problem and overcome it. Once you’ve overcome the challenge any next one won’t be near as impactful or difficult.

Comments (0)

Skip to main content