Finally Remove Insecure LDAP and Protect your Credentials with Project VAST

What if I told you that there exists a set of predictable, repeatable vulnerabilities which is shared by most every customer that I work with? What if I told you that, unless you’ve taken careful, specific steps to mitigate these vulnerabilities*, your organization is also very much at risk.

Generally speaking, these vulnerabilities fall under the rubric of credential theft and reuse. And today I’d like to dive deeper into one such widespread vulnerability: Lightweight Directory Access Protocol (LDAP) security misconfiguration. We need to talk about it, and you need to do something about it. And Project VAST can help.

Hey there, it’s Jon again, with this month’s installment about Project VAST (the Visual Auditing Security Tool).

Quick Review: What is LDAP

LDAP (an X.500 derivative) is a protocol for data exchange between clients and data stores like Active Directory (AD). Clients bind (or authenticate) against a Domain Controller (DC), run a query to ask for a certain data set, and then receive the data set back from the DC. An example of this exchange might be something like this: Client1 binds against DC1 and asks for the multivalued attribute value member for an object with the displayName of zookeeper. In lay terms, the application, person, or script running the query just asked for the name of all members of an organization’s group zookeepers in the directory. (AD uses LDAP for all sorts of internal calls as well.)

Simple enough, right? Well, not so fast.

What’s the problem?

The problem is with how the client asks for the data. Specifically, in how it binds to the DC. Unless you’ve configured the DC to require signing, many clients are returning unsigned traffic, which is susceptible to replay or attacker-in-the-middle attacks. This may result in nefarious activity, such as modified packets, in which a server or even a person makes decisions based on forged data.

Let’s have a look at a simple network cap below. See anything? Yup, we’ve exposed potentially sensitive data to an attacker, or anyone else with a sniffer.

Let’s look a bit deeper and examine the binds themselves. They’re similarly not secure. Turning back to our simple network cap, we can see the credentials being asserted in the simple LDAP bind. (For clarity, this is not an AD problem. It’s a problem of LDAP and specifically of LDAP implementations.) Even more concerning, many organizations are exposing their highly-privileged authentication principals' credentials – such as the ones for those that belong to the Domain Admins group. With a simple network cap, an attacker can capture these credentials and achieve an escalation path or even a domain dominance type scenario.

Don’t despair – yes, you are probably vulnerable and yes, you have work to do. But with some help from Project VAST, you can mitigate this scenario.

What to do about insecure LDAP?

Well, the first order of business is to verify that you indeed do have this vulnerability. We’ll do this by checking first for the existence of Event 2887 in the Directory Service log. If you find this event (and you probably will), it means you have had, in the last 24 hours, a client attempt to bind with either a SASL (non-signed) bind or a simple bind over a clear-text connection. At the same time, the existence of 2887 tells us that the directory service is not currently configured to reject such binds. In other words, we have verified the vulnerability exists in the directory.

An example of an Event ID 2887

At this point, we know we have an insecure LDAP configuration and we have clients leveraging it insecurely. But we haven’t yet discovered the source – we don’t know where to take action on the client side. (We could simply start requiring LDAP signing on the DCs, but that would cause the DCs to reject these binds. That would probably not end well.  😊 ) For this, we’ll need to configure some auditing in order to gain a bit more information in the form of Event ID 2889

In order to capture this event, we need to make a registry change on each of our DCs. We’ll target HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics /v "16 LDAP Interface Events" with a DWORD value of “2.”

If you have more than a handful of DCs, you’ll probably want to use Group Policy Preferences to set this registry value on each of your DCs. (This can be a very verbose log. You may need to increase the size limit of your log, and certainly turn it off when you’re finished.)

For our project, 2889 is a high-value event, in that it tells us some pretty granular information about the sources of insecure LDAP: The following client performed a SASL (Negotiate/Kerberos/NTLM/Digest) LDAP bind without requesting signing (integrity verification), or performed a simple bind over a cleartext (non-SSL/TLS-encrypted) LDAP connection.

Event 2889

By closely examining this example event 2889, we can see that account svc4 has actually performed an unsigned SASL or clear-text bind from IP address 192.168.2.67. Also, pay attention to the value called Binding Type. The value of 1 in the example tells us that svc4 performed a simple bind; a value of 0 would indicate an unsigned SASL bind.

The Challenge

Do you see the challenge here? In order to enforce signing on our DCs, we need to first remediate insecure binds (or else we’ll break the applications that rely on them). Look closely at the picture above. This log file represents one bind out of 6,771 during a 13-day period from one DC in my lab.

You could use Windows Event Forwarding or a SIEM to aggregate the log files. If you are able to accomplish mitigation of insecure LDAP using these technologies, then you should use them. (You should also share how you were able to accomplish this.) In my experience, aggregation simply isn’t enough. Assuming for a second that the DC above is representative and that my organization has 10 DCs, I have something in the neighborhood of 60,000 logs to sift through. That’s simply not realistic on its own. What we have here is a problem of big data.

Enter Project VAST

This is one of the charters for Project VAST: to make big data sets in the form of audit logs actionable and realistically usable for security mitigation activities. As I discussed in a previous blog entry, we’ll use the relatively new technologies of Azure Log Analytics and Power BI to get a handle on our data here. Using the Microsoft Monitoring Agent (MMA) and some Azure configuration, we’ll pump all of our 2889 events into Azure Log Analytics. Once the data has arrived (typically a delay of a few minutes), we can view and begin controlling the output with a Kusto query in Azure.

That’s better, but it’s not really truly actionable yet. After all, this data set of seven days and two domain controllers contains 9,258 event records.

The real goal here is to leverage the rich visualizations that we’ve come to know and love in Power BI in order to represent the data in a truly actionable format. This allows us to make logical decisions about how we present the data. More importantly, it allows data consumers to make rational, data-centric decisions and to act upon their data.

A Closer Look

Now that we’ve covered the vulnerability and the problem in taking action, let’s have a closer look at Project VAST’s Insecure LDAP tab.

The first thing to notice here is the general look and feel. In the coming months and blog entries, I hope that this interface will become familiar to you. Let’s start in the upper left-hand corner and move through the tab clockwise. (As in the past, a few details have been obfuscated to protect the innocent. 😊)

In the upper left, you’ll see the data flow. This visualizer is mapping the flow of client to DC data. Here, we see a large amount of insecure LDAP traffic coming from two clients and against two DCs. (Recall that the data source here is 2889; in a normal production environment, the data flow may be more complex, so we’re limiting the view to the Top 5 clients.) Right off the bat, we’ve got a few IPs with which to begin our detective work.

To the right of the data flow are some slicers: source IP and destination DC. These will be useful once we’ve decided upon the specific LDAP workflows that we need to research. To the right of Destination DC is a critical filter called isAdmin. This is just what it sounds like. Because VAST does a limited set of queries against AD, we can determine what accounts constitute Administrative accounts; we can (and should) pay special attention to these high-value, high-risk accounts.

This brings us to Insecure LDAP by Account. This is actually where we tend to start our research with customers. The first thing you’ll notice is that some data bars are red and others are yellow. This is because red bars delineate that these are Administrative accounts. (By Administrative accounts, we mean that these accounts have an adminCount value of 1; they are or have been members of a group protected by AdminSDHolder.) By clicking on one (let’s say svc1 in the example above), we can filter all other traffic out and focus solely on this account’s insecure LDAP traffic. By doing this, we’re taking a problem that might seem overwhelming and tackling it in bite-size chunks. This allows us to really dig down into the data to understand where it’s coming from – if necessary, we can look to a raw(er) view of the data (lower right-hand corner) and also delve into the date and time stamps from the 2889s with our final slicer in the lower left-hand corner.

Once we have discovered the source of our insecure LDAP traffic, we’ll work with application owners, developers, and vendors to move to signed and/or encrypted LDAP. Ideally, we’ll also work with decision-makers in our environment to set a date for insecure LDAP to be turned off. We’ll do this so that once the project is complete and we see no more LDAP binds (e.g. we’ve established a KPI of zero insecure LDAP binds), we can disable insecure LDAP on the DCs. That way, new LDAP calls that appear in the environment won’t be able to use insecure or unsigned implementations.

A small point of clarity

After my last blog post, a few folks asked some questions around Project VAST. Specifically, folks wanted to know how they could implement Project VAST in their environment. At present, the only way to implement Project VAST is through Microsoft Premier services, as Project VAST is a Premier Field Engineer (PFE)-designed service. If you’re interested, please contact your TAM or Account Team. (If your organization is not yet a Microsoft Premier customer, please click here.) In the future, the project may become open source or may even integrate into one of our Azure Security products. Stay tuned!

Until then, happy auditing.

*For a broader discussion of credential theft mitigation, please browse the Securing Privileged Access (SPA) roadmap at https://aka.ms/privsec.