Howdy folks. Keith here with my first blog in quite a while. I am no longer supporting ISA/TMG/UAG but moved over to a new role supporting Microsoft's implementation of Public Key Infrastructure (PKI) and along with that Secure Channel (Schannel). I am excited about the challenge and I get to see a lot of interesting cases.
I recently had a case that was perplexing to say the least.
After installing KB3042058 on a Windows 2008 R2 machine that was running IIS the clients were no longer able to connect to the SSL site on that machine. A network trace showed the setup of the TCP three way handshake, a Client Hello, and then a RST by the IIS Server.
Schannel errors with Event ID 36888 were also showing up at those times. The sub codes were 80, 1250 and 80, 1051.
80 is a Fatal Error
It took a while for us to realize what was going on and I would like to share with you a way to determine if you are having this exact issue.
At an administrative command prompt, run the following command
certutil -v -verifystore My * >out.txt
Now open the corresponding out.txt file that was created and locate the certificate that your website is using. In my lab I had issued myself one for www.wtt.com (WingTipToys).
Continue scrolling down until you find the lines that tell you the Provider and the KeySpec value. More than likely the Provider will be a CSP. In my case it was "Microsoft RSA Schannel Cryptographic Provider." The Keyspec was also 0 which is NOT valid for a certificate issued by a CSP.
Keep looking through the output and look for a Unique Container Name. Make a note of that value.
Now open %programdata%\Microsoft\Crypto\Keys and see if that same container is listed there. My bet is that it does exist there.
Now go look at the folder %programdata%\Microsoft\Crypto\RSA\MachineKeys and check the permissions of that folder. If you see anything other than the defaults (Everyone and Administrators) chances are that is our problem. In my case System had been added with Full Permissions.
Default permissions for Everyone is listed below.
Default permissions for Administrators is listed below.
Remove any groups that should not be there and makes sure the default permissions are set back to what I have listed above.
Now go delete the private key that you located earlier under %programdata%\Microsoft\Crypto\Keys
Also, in the Certificate Store delete the corresponding public key
Now you should be able to re-import the original .pfx file (assuming you still have it) and if you did the steps above correctly the private key will be written to the correct location (%programdata%\Microsoft\Crypto\RSA\MachineKeys) and KeySpec value will be something other than 0.
You can verify this again by running the certutil -v -verifystore My * >out.txt and finding the pieces of information you checked earlier.
Note: An alternative method to fixing this issue is to have your PKI administrators issue a new certificate for you that is from a Key Storage Provider.
I created a Web Server template in my lab and made sure to change the Provider to be a KSP, requested a certificate, imported it into the Certificate Store on my IIS Server, bound it to IIS and then tested. Worked like a charm.
In this post I showed you how to solve a pretty perplexing issue that ultimately resulted in the private key for a certificate being written to the wrong location on the file system.