Authentication is entering every facet of our lives nowadays. It is common to have multiple passwords: passwords for work, home email, and Internet websites to name a few. It’s easy to have a lot of different passwords, and equally easy to use only one and risk a widespread identity breach. Passwords are one way of guaranteeing the identity of an individual in a communication but that’s just a start. Two factor authentication is becoming more prevalent in the corporate world, and may someday soon be a part of your daily routine in your home life.
For now, two factor authentication is commonly used as a smartcard plus a user specific password used in an Active Directory domain authentication context. Let’s take a high level refresher of what a smartcard is and how its usually used in an Active Directory domain. Smartcards are essentially a small device which contains a certificate issued to user. The user accesses that certificate by placing the card into a card reader and then supplying a password (or personal identification number) to gain access to that certificate prior to requesting authentication. This certificate, in lieu of the traditional password string of text, is used in communication with the domain for user logon and authentication.
Smartcard logon in part works by having a Domain Controller template based certificate in the authenticating domains local computer certificate stores. In the more straightforward scenario of an Enterprise Certificate Authority, where information regarding the installed CA is stored in the forest AD, the domain controller certificate is auto enrolled to the domain controller as a matter of course. That can make for a nice starting place for configuring smartcard logon to work in your environment.
What if you are a company that maintains a separate certificate authority (CA) from some or all forests and would like to use that CA as an issuer for your smartcard certificates? There are some clear benefits to doing things that way. Foremost would be the ability to use one CA to allow smartcard logon for the users in different forests. This can be useful when you have one unifying corporate structure which has very distinct and separately managed child companies and would also allow for some central control over authentication standards.
Better written and more technical guidance on smartcard logon for domains and how to do it is in the book Windows Server® 2008 PKI and Certificate Security, and also in the KB article Guidelines for enabling smart card logon with third-party certification authorities.
The point of this post is not to discuss the value of configuring smartcard logon and how to do it, but rather to talk about what to do when a specific problem involving smartcard logon occurs.
This is a fairly lengthy premise for a specific problem that you could see: smartcard logon failing while ‘traditional’ credential logon of username plus password succeeds.
There are a few different causes that can make this sort of thing happen but the things you want to look at in order to diagnose what is happening are all approximately the same.
First, when did it happen? This can also be a useful piece of information since it can infer what the cause was. Did the problem start after a reboot of the domain controllers? Did some or all of them seem to fail within a short span of time without a reboot?
In today’s scenario we’ll put forward that the issue occurred following a reboot of the domain controllers, and that we see some interesting events (below) in the System event log of the domain controllers which seem to relate to the problem.
Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 7
The Security Account Manager failed a KDC request in an unexpected way. The error
is in the data field. The account name was host$ and lookup type
Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 19
Computer: <Computer Name>
Description: This event indicates an attempt was made to use smartcard logon, but
the KDC is unable to use the PKINIT protocol because it is missing a suitable
Besides the event logs and the events above one of the most useful tools for this type of issue is Certutil.exe. Certutil.exe is the tool to use in situations where you need to look into the “health” of the certificates in a store.
For this situation you would want to run the command
In today’s post scenario here’s we do that and see that the private key for the Domain Controller certificate doesn’t appear to be there.
================ Certificate 0 ================
Serial Number: <snip>
Issuer: CN=MS CertSrv Test Group CA, OU=Windows NT, O=Microsoft, L=Las Colinas, S=TX, C=US
Subject: CN=DC5.child2.domain1.com, CN=MS CertSrv Test Group CA, OU=Windows NT, O=Microsoft, L=Las Colinas, S=TX, C=US
Certificate Template Name: Domain Controller
Template: Domain Controller
Cert Hash: <snip>
No key provider information
Missing stored keyset <—MISSING
Verified Issuance Policies:<snip>
Verified Application Policies:
188.8.131.52.184.108.40.206.2 Client Authentication
220.127.116.11.18.104.22.168.1 Server Authentication
Certificate is valid
CertUtil: -verifystore command completed successfully.
So the above appears to be the problem-the private key is missing. Oddly, when we run the verifystore on the other domain controllers of the domain we see the same subject reference of CN=DC5.child2.domain1.com. That in itself is a problem since every domain controller should have its own uniquely issued domain controller certificate.
For DC5 perhaps the association to the private key is the only thing missing. In other words, perhaps the private key and rest of the certificate are both there but just not “linked” to each other for some reason. What can we do to repair that if it can be repaired?
Certutil.exe to the rescue once more. We can use that tool to repair things with the command below, using the serial number value found in the verifystore command. For the domain controller, however, we would need to do that in the DC context which is something you should be able to achieve by using the AT command, either launching a command prompt interactively if allowed or by putting the command below into a batch file and running it that way.
certutil -repairstore my “serial number”
Viola!, we reboot DC5 and suddenly it can service smartcard logon requests. The other domain controllers are another matter though. In a situation such as this the other domain controllers must go through the entire request process for their own Domain Controller certificates.
So what can we hypothesize happened in this scenario? Domain Controller template based certificates are issued to specific hosts and cannot be used on a computer other than the one they are issued to. In light of seeing Domain Controller type certificate Subject field on all of the affected domain controllers all containing the host name of DC5 we can guess that someone exported the certificate from DC5 and imported it to all of the domain controllers in the domain. For the DCs which were not DC5 of course that certificate would never work, and barring the little host uniqueness aspect there the default Domain Controller certificate template does not allow for private key export anyway (which is a good thing). That may also explain why the DC5 certificate had a problem associated with it’s own private key-the certificate was exported without it, and then that exported certificate was imported in, thus breaking the association.
More often you’ll see similar behavior if you are using a third party or non-Enterprise certificate authority and the domain controller certificates expire. That’s a problem that folks with Microsoft Enterprise CAs are not likely to see since the domain controllers will auto enroll in those certificates.
The scenario discussed here is by no means a common one. I’m passing it along, though, to lend some insight into some AD and authentication specific behavior and some troubleshooting that can be applied to a variety of similar issues. Hopefully you won’t see problems, but if you do then let’s hope this info helps you out.