The Case of the Mysterious Code Signing Failures

I digitally sign code on a regular basis in the course of preparing Sysinternals executables for upload to the site. When you digitally sign a file, you encrypt the hash of the file with the private key of a public/private key pair. Someone can verify that you’ve signed the file by decrypting the encrypted hash with your public key and comparing the result with the hash of the file they calculate themselves.  The signing process is made simple with Signtool.exe, a utility that comes with the Platform SDK and the .NET Framework. You pass it your signing certificate, private key file, and target file as command-line arguments and it does the rest, appending the signed hash in the file as a final step.

The other day I went to sign an updated Sysinternals tool and ran into this error message:

It had been a week or so since the last time I had tried signing anything, but I couldn’t think of any changes I had made to the system that would have lead to this failure.  However, anyone that’s used computers for any length of time knows that they’re not really deterministic and that system configuration is often subject to spontaneous corruption. I resigned myself to never knowing the root cause and set out to resolve the problem.

The first thing I did was search for capicom.dll with the built-in Where utility, which looks for the file you specify in each of the directories listed in the PATH environment variable. The PATH environment variable is used for DLL searches, so I expected this step to confirm that I was missing Capicom.dll:

The output appeared to confirm it, but then I realized that because I was running on a 64-bit system and Signtool is a 32-bit executable, Where.exe wouldn’t look in the %SystemRoot%\Syswow64 directory, which is the directory in which 32-bit system DLLs are stored. When I manually looked in that directory I was surprised to find a copy of Capicom.dll:

Signtool must therefore not be looking for Capicom.dll in the directories listed in the PATH environment variable, so the question before me was, where was Signtool looking? I knew Process Monitor was the perfect tool to answer a question like that, so ran it, configured an Include filter for any Path ending in “capicom.dll” and then repeated the Nmake command that triggered the error:

The trace shows that, for some reason, Signtool only looks for Capicom.dll in two directories:  the Microsoft Shared sub-directory of the system’s Common Files directory, and the \Bin directory, which was where Signtool is located on my system.

To fix the problem I simply copied the Capicom.dll file from the \Windows\Syswow64 to the \Bin directory. I reran the make command and, as I expected, it succeeded. Process Monitor to the rescue!

Comments (28)

  1. Anonymous says:

    Je viens de tomber sur cet article chez Mark Russinovich , qui décrit un problème possible avec SignTool,

  2. Anonymous says:

    Toma: Signtool actually registers capicom if it’s in the same directory and not yet regsvr32’d. It’s nonstandard, but true in this case.

  3. Where.exe is included in Server 2003 SP1 (it might be in Server 2003 SP0, but I don’t have a copy to check), Windows XP 64-bit for x64, and Windows Vista.

  4. Anonymous says:

    Sorry, Mark. I know I tested it in a 32 bit cmd.exe, but IIRC (it’s been a few moons since I left) we considered invoking directly from 64-bit-land out of scope. Oops.

    I’m curious. What’s the %path% on your box for a 32 bit EXE launched from 64-bit-land? I could have sworn that signtool looked anywhere in the path and tried to install capicom if it could be found.

    I’ll check this out myself when I’m in front of my test boxes again tomorrow.

    Drew the ex-‘softie, previous test owner of signtool.exe

  5. Anonymous says:

    I know this may seem like a silly solution, and I understand that Windows does things Windows’ way, but common sense would tell me to look in my /BIN folder for a file that signtool.exe is telling me should be in my /BIN folder.  Upon noticing its absence, initiating a search for the missing file would allow me to find it, place it in the /BIN folder and see if this corrected the problem.  I understand that exploring the Ins and Outs of Windows is fun, and explaining it in a useful way is very helpful. [you are exceptionally good at it, and for this we are all indebted]  But wouldn’t it sometimes be easier to say that sometimes, rarely, when you least expect it, Windows behaves the way it used to/should?

  6. The path search order is indeed COM at work: it’s first looking in the location where the COM registration indicates the DLL resides and if that fails it falls back and looks for it in the current directory. A full Process Monitor trace also shows the Registry accesses where OLE reads the COM DLL path.

  7. Mike Dunn says:

    The app must be explicitly looking in those two dirs. LoadLibrary doesn’t search other dirs if the caller includes a dir name in the first parameter.

  8. Dave says:

    Wow, that’s good to know and several people have reported similar problems with Vista and CAPICOM. However, Microsoft’s official position is that CAPICOM isn’t supported in Vista.

    You are supposed to use the shiny new Certificate Enrollment API, which has finally been documented in the past week:

    I wonder if Microsoft will have a new signtool.exe for Vista, or whether they will use the unsupported CAPICOM interface as well. I hope it’s the latter, because that I sure don’t want to create a Vista-specific code path when CAPICOM works perfectly well on other versions of Windows.

  9. Ross Presser says:

    Um, where.exe certainly is not built-in, at least not as far as XP SP2 or Windows 2003 Server. Nor is it part of the downloadable resource kit utilities.

    It is certainly an essential functionality for debugging things, but it’s not builtin as far as I know. I use cygwin’s which.exe.

  10. Ross Presser says:

    OK, guess I was wrong about 2003. It definitely isn’t supplied with XP 32-bit though.

  11. Matt says:

    For those running XP, you can find a "where" substitute here:

  12. Chris says:

    FWIW, where.exe is also included with visual studio (at least in VS 2003, it’s in the "Common7/Tools/bin" subdirectory).

    Problem is that which.exe (from cygwin) stops at the first hit that matches the searched executable, whereas where.exe lists all occurances.

  13. Toma Bussarov says:

    Signtool.exe is NOT searching directly for capicom.dll. Capicom.dll contains a number of COM objects (obviously 🙂 ). The signtool attempts to create a COM object located in capicom.dll and the COM subsystem looks for it (check MSDN for the exact order).

    I had a similar problem with one of my apps that was using capicom objects. Since Process Monitor exists only from 2 months, I used File Monitor to find the cause of this strange problem.

    <b>Mark</b>, try to set a filter only for signtool.exe without path filtering and you will find it! I suspect Visual Studio installer in messing thing up.

  14. Toma Bussarov says:

    Mark, yes it is normal that the full trace shows the registry access where OLE reads the COM DLL path, but …

    On my machine the problem was that in the registry the full path was written in quotes, which by some reason COM does not understand. Actually quotes are not needed because the WHOLE value is supposed to contain only path, not path and parameters.

    I was not able to find which app registered capicom.dll in this way. I suspected Visual Studio, but can’t prove it yet.

  15. ML49448 says:

    Where.exe is also available in VS2005 at C:Program FilesMicrosoft Visual Studio 8Common7ToolsBinWhere.Exe

  16. Shimon Amit says:

    You must have uninstalled a program between the last time you signed your exe and the present. That is why capicom.dll was missing.

    I’ve often found that during an uninstall, I get a window telling me the uninstaller wants to remove a "shared file" but in all likelihood, no other program is using it.

    Usually I’m cautious and do not allow it to remove these shared files but a while ago I took a chance and allowed it. Soon after, every time booted or tried to open an MS Office file (doc, xls, etc.) the Office installer popped up and asked for its installation CD.

    Windows Installer has a built in feature which fixes broken installations. If you run a program installed using Windows Installer and later, when the program requests a resource (dll in our case) that is corrupt or missing, it fetches it from the installer.

    Apparently, when I allowed the uninstaller to remove the "shared file", it actually removed a file Office was using.

    In your case, you also experienced two conflicting installers.

  17. phs3 says:

    "However, anyone that’s used computers for any length of time knows that they’re not really deterministic and that system configuration is often subject to spontaneous corruption."

    Um, no…that may be true of Windows, but not of mature systems, such as z/VM, z/OS, z/VSE, and perhaps even Linux.

    Hate to be a pismire — and I’m generally NOT a Microsoft-basher — but this assertion is a bit scary from someone as senior as you.


  18. James Davis says:

    Actually phs3, I’ve had plenty of system configuration corruption with Fedora Linux, especially when updating, as the built in updater likes to replace my NVidia supplied graphics-card LKM with an ‘open source’ alternative that cannot handle dual screens.  Resulting in the system refusing to boot properly.

    (In particular, an X server cannot be started.)

  19. Smit-Tay says:

    "However, anyone that’s used computers for any length of time knows that they’re not really deterministic and that system configuration is often subject to spontaneous corruption."

    In fact computers are very deterministic, it’s the creators of crappy operating systems and applications that introduce this non-determinism.  

    You need only review the trace output and notice how many times the same path is searched for the dll to see the type of poor quality I am talking about. Once the dll is not found there, why search again ?  This is especially true for Windows where registry corruption of one form or another is a daily occurance and subsystems like .NET and COM and others are haphazardly bolted on to each other producing millions of duplicated instructions and yet are still unable to provide the requested service – in this case digitally signing your app.

  20. John M. Dlugosz says:

    The 4NT shell from JP Software has an intrinsic "which" command.

    It sounds like Where doesn’t work properly with the 64/32 bit path differences… does it perhaps have a flag to look in the 32-bit side?

  21. Roger Chaplin says:

    Cygwin which has the -a switch that causes it to output all instances, not just the first one it finds.

  22. micaman says:

    Regardless of what operating system you use, it is still a piece of software. And ALL software can be hacked – period. This is as old as the day is long!

    Windows is one of the most secure software programs out today – it just has a larger user group.

    If you think your other operating system is unhackable, just provide the challenge, and a hacker will be glad to show you some flaws! I have seen published hacks for just about every piece of major software on the market (including the operating systems Linux and Apple), and the programs I haven’t seen hacks for – well, I simply didn’t have the time to read about them, but, they do exist!

    2007 is going to be one of the worse years in malware history for all software programs connected to internet. Just as Firefox didn’t have issues for a "short" while, as their numbers grew – so did the hacks & flaws. Once software grows in numbers, IT IS ATTACKED, period.

  23. tom says:

    Put this into where.bat, somewhere in your PATH

    for %%i in (%1) do echo %%~dp$PATH:i

    Works in Win2K and XP.

  24. jim says:

    If you’re using VS2005, launch the Visual Studio 2005 command prompt and use where.exe.

    or add it to your path:

    n:Program FilesMicrosoft Visual Studio 8Common7ToolsBin;

    or have the VS cmd prompt init at startup…

  25. friedrich says:

    Well the most interesting thing is that you do not have any chance finding out which file they are lookign for without something like procmon.

    I run into a similiar problem once which just resulted in a 0x000003 or so error code meaning "file not found" no hint which file was missing.

    I guess it’s not important why something is broken, just that it is….



  26. says:

    These comments have been invaluable to me as is this whole site. I thank you for your comment.

  27. cellotin says:

    However, anyone that’s used computers for any length of time knows that they’re not really deterministic and that system configuration is often subject to spontaneous corruption."

    Um, no…that may be true of Windows, but not of mature systems, such as z/VM, z/OS, z/VSE, and perhaps even Linux.

    Hate to be a pismire — and I’m generally NOT a Microsoft-basher — but this assertion is a bit scary from someone as senior as you.


  28. julian says:

    I stumbled across this blog after experiencing the same error message as Mark. Difference is, I’m just using a plain old 32-bit version of Vista. I downloaded the latest capicom.dll from the Microsoft site, but it didn’t make the slightest difference where I installed it, I got the same error.

    Fortunately I prefer working in XP. I was just testing a little code signing tool I made to check that it worked under Vista. The old signcode.exe still works, anyway.