The Case of the Mysterious Code Signing Failures

I digitally sign code on a regular basis in the course of preparing Sysinternals executables for upload to the site. When you digitally sign a file, you encrypt the hash of the file with the private key of a public/private key pair. Someone can verify that you’ve signed the file by decrypting the encrypted hash with your public key and comparing the result with the hash of the file they calculate themselves. The signing process is made simple with Signtool.exe, a utility that comes with the Platform SDK and the .NET Framework. You pass it your signing certificate, private key file, and target file as command-line arguments and it does the rest, appending the signed hash in the file as a final step.

The other day I went to sign an updated Sysinternals tool and ran into this error message:

It had been a week or so since the last time I had tried signing anything, but I couldn’t think of any changes I had made to the system that would have lead to this failure. However, anyone that’s used computers for any length of time knows that they’re not really deterministic and that system configuration is often subject to spontaneous corruption. I resigned myself to never knowing the root cause and set out to resolve the problem.

The first thing I did was search for capicom.dll with the built-in Where utility, which looks for the file you specify in each of the directories listed in the PATH environment variable. The PATH environment variable is used for DLL searches, so I expected this step to confirm that I was missing Capicom.dll:

The output appeared to confirm it, but then I realized that because I was running on a 64-bit system and Signtool is a 32-bit executable, Where.exe wouldn’t look in the %SystemRoot%\Syswow64 directory, which is the directory in which 32-bit system DLLs are stored. When I manually looked in that directory I was surprised to find a copy of Capicom.dll:

Signtool must therefore not be looking for Capicom.dll in the directories listed in the PATH environment variable, so the question before me was, where was Signtool looking? I knew Process Monitor was the perfect tool to answer a question like that, so ran it, configured an Include filter for any Path ending in “capicom.dll” and then repeated the Nmake command that triggered the error:

The trace shows that, for some reason, Signtool only looks for Capicom.dll in two directories: the Microsoft Shared sub-directory of the system’s Common Files directory, and the \Bin directory, which was where Signtool is located on my system.

To fix the problem I simply copied the Capicom.dll file from the \Windows\Syswow64 to the \Bin directory. I reran the make command and, as I expected, it succeeded. Process Monitor to the rescue!