Resolving WinRM errors and Exchange 2010 Management tools startup failures


EDIT 12/10/2010: Added a permissions section.

As was discussed in the previous (related) blog post “Troubleshooting Exchange 2010 Management Tools startup issues“, in Exchange 2010 the Management tools are dependent on IIS. As was discussed in that blog, we have seen situations where the management tool connection to the target Exchange server can fail, and the error that is returned can be difficult to troubleshoot. This generally (but not always) happens when Exchange 2010 is installed on an IIS server that is already in service, or when changes are made to the IIS server settings post Exchange Install. We have seen that these changes are usually done when the IIS administrator is attempting to “tighten up” IIS security by editing the Default Web Site or PowerShell vdir settings.

The situation is further complicated by the fact that some of the errors presented have similar wording; most seem to originate with WinRM (Windows Remote Management), and in some cases different root problems can produce the exact same error message. In other words, depending on how knowledgeable you are with these errors, troubleshooting them is all around… not much fun.

I was approached by a good friend of mine and he asked what I thought we could do to make these errors a little easier to troubleshoot. I was studying PowerShell and PowerShell programming at the time (I just happened to be reading Windows PowerShell for Exchange Server 2007 SP1), and I thought that this would be a perfect opportunity to try and apply what I was learning.

This is the result.

Introducing the Exchange Management Troubleshooter (or EMTshooter for short).

What it does:

The EMTshooter runs on the local (target) Exchange server and attempts to identify potential problems with management tools connection to it.

The troubleshooter runs in 2 stages. First, it will look at the IIS Default Web Site, the PowerShell vdir, and other critical areas, to identify known causes of connection problems. If it identifies a problem with one of the pre-checks it will make a recommendation for resolving the problem. If the pre-checks pass, the troubleshooter will go ahead and try to connect to the server in the exact same way that the management tools would. If that connection attempt still results in a WinRM-style error, the troubleshooter will attempt to compare that error to a list of stored strings that we have taken from the related support cases that we have seen. If a match is found, the troubleshooter will display the known causes of that error in the CMD window. Here is an example of how this might look like:

When I was designing the troubleshooter, I could have just written a little error lookup tool that handed over the appropriate content for the error you were getting, but I felt that was not as robust of a solution as I was aiming for (and not much of a learning experience for me). So the tool runs active pre-checks before moving on to the error look-up. The amount of pre-checks it can run depends on the flavor of OS you are running on and the options you have installed on it, such as WMI Compatibility.

Basically, I have taken all of the documentation that has been created on these errors to date, and created a tool that will make the information available to you based on the error or problem it detects. Hopefully this will cut down on the amount of time it takes to resolve those problems.

Event reporting:

When you run the EMTshooter it will log events in the event log. All results that are displayed in the CMD window are also logged in the event log for record keeping.

Events are logged to the Microsoft-Exchange-Troubleshooters/Operational event log and are pretty self-explanatory.

Things to remember:

Depending on your current settings, you may need to adjust the execution policy on your computer to run the troubleshooter, using:

Set-ExecutionPolicy RemoteSigned

Or

Set-ExecutionPolicy Unrestricted

Remember to set it back to your normal settings after running the troubleshooter.

This version of the troubleshooter needs to run on the Exchange Server that the management tools are failing to connect to. While our final goal is that the troubleshooter will be able to run anywhere the Exchange Management tools are installed, the tool isn’t quite there yet.

We have seen instances where corruption in the PowerShell vdir or in IIS itself has resulted in errors that seemed to be caused by something else. For instance, we worked on a server that had an error that indicated a problem with the PowerShell vdir network path. But the path was correct. Then we noticed that the PowerShell vdir was missing all its modules, and quite a few other things. Somehow the PowerShell vdir on that Exchange Server had gotten severely… um… modified beyond repair. WinRM was returning the best error it could, and the troubleshooter took that error and listed the causes. None of which solved the problem. So be aware that there are scenarios that even this troubleshooter cannot help at this time.

The troubleshooter is still a bit rough around the edges, and we plan to improve and expand its capabilities in the future. We also hope to be able to dig a little deeper into the PowerShell vdir settings as time goes on. Also note that the troubleshooter will NOT make any modification to your IIS configuration without explicitly asking you first.

Permissions required:

In order to run the troubleshooter, the user will have to have the rights to log on locally to the Exchange server (due to local nature of the troubleshooter at this time) and will need permissions to run Windows PowerShell.

Installing the troubleshooter:

First, you will need to download the troubleshooter ZIP file, which you can find here.

Installing the EMTshooter is pretty easy. Just drop the 4 files from the ZIP file into 1 folder, rename them to .ps1 and run EMTshooter.ps1 from a normal (and local) PowerShell window. I personally just created a shortcut for it on my desktop with the following properties:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noexit -command “. ‘C:\EMTshooter\EMTshooter.ps1′”

However, as most users probably won’t run this more than a few times you might not need or want an icon. Just remember that EMTshooter.ps1 is the main script to run.

Providing feedback:

As I mentioned before, the troubleshooter is still a work in progress. If you wish to provide feedback on it, please post a comment to this blog post. I will be monitoring it and replying as time allows, and also making updates to the troubleshooter if needed. If you run into errors that are not covered by the troubleshooter, please run the troubleshooter, reproduce the error through it and send me the transcript.txt file (you will find it in the folder where the 4 scripts have been placed), along with what you did to resolve the error (if the problem has been resolved). My email is sbryant AT Microsoft DOT com.

Errors currently covered:

  • Connecting to remote server failed with the following error message : The WinRM client cannot process the request. It cannot determine the content type of the HTTP response from the destination computer. The content type is absent or invalid.
  • Connecting to remote server failed with the following error message: The connection to the specified remote host was refused. Verify that the WS-Management service is running on the remote host and configured to listen for requests on the correct port and HTTP URL.
  • Connecting to remote server failed with the following error message: The WinRM client received an HTTP server error status (500), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic. It was running the command ‘Discover-ExchangeServer -UseWIA $true -SuppressError $true’.
  • Connecting to remote server failed with the following error message : The WinRM client received an HTTP status code of 403 from the remote WS-Management service.
  • Connecting to the remote server failed with the following error message: The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol.
  • Connecting to remote server failed with the following error message : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service:
  • Connecting to remote server failed with the following error message : The WS-Management service does not support the request.
  • Connecting to remote server failed with the following error message : The WinRM client cannot process the request. The WinRM client tried to use Kerberos authentication mechanism, but the destination computer

Steve Bryant

Comments (27)
  1. itgirl says:

    I cannot run these files, when I open powershell it tells me that the hash doesn’t match what it expects to see.

  2. Hornet says:

    Can you provide any information how to troubleshoot issues with the EMC being very slow when you try view information under the Organization and Server containers?  For example, if you want to look over the OWA Virtual Directory settings for a CAS Server, it takes forever to load the data so you can view the settings?   I noticed it the responsiveness of the EMC has slowed to a crawl after adding more Exchange 2010 Servers in other AD sites.

  3. ninob says:

    @ itgirl: does your organization have a PowerShell execution policy defined? We mentioned that in the blog post as something that might need to be tweaked…

    @ Hornet: is this with Exchange 2010 SP1? There were some fixes that went into EMC in SP1 that should make this a much better story.

  4. bday says:

    @Hornet, also make sure your Exchange Trusted Subsystem is in the Local Admins group of any 2007 CAS servers still in the org.

  5. Erwin says:

    Hi, I could not start the Exchange Management Shell/GUI. I got the  error :Connecting to remote server failed with the following error message : The WinRM client cannot process the request. It cannot determine the content type of the HTTP response from the destination computer. The content type is absent or invalid.

    After trying every solution found on the net (took me 2 days) with no luck, I logged on to the Exchangeserver using a new created account, everything worked fine.

    So finally my admin accounts kerberos token size was too big !

  6. get987him says:

    Hi everybody…

  7. Steve Bryant says:

    @ Erwin: Nice job figuring that out.  Not sure how I would look for that in the tool as a pre-check, but I’ll see if I can repro and at least add the error to the list of ones I watch for.  Thx

    @itgirl: I haven’t seen that particular error before, and other users have been able to download and run the script without a problem.  If defining an execution policy doesn’t work, send me an email with a screen shot of the error.  

  8. Elad says:

    Steve,

    i am experiencing the same issue as itgirl

    the error msg i am getting is:

    File C:EmtshooterEMTshooter.ps1 cannot be loaded. The contents of file c:EMTshooterEmtshooter.ps1 may have been tampered because the hash of the file does not match the hash stored in the digital signature. the script will not execute on the system. Please see "get-help about_signing" for more details…

    at line:1 char:2

    FYI i do have 4 files in the EMTshooter folder :

    EMTConnectFuntions

    EMTErrorHandler

    EMTShooter

    EMTshooter.strings

  9. armani jeans says:

    cheap christmas gift UGG,GUCCI,LV,NIKE,JORDAN,ARMANI,POLO,ED HARDY on http://www.bizsuper.com

    <a href="http://www.bizsuper.com/">Armani Jeans</a>,

    <a href="http://www.bizsuper.com/">Christian Audigier Jeans</a>,

    <a href="http://www.bizsuper.com/">Versace Handbags</a>,

  10. Sam Bradford says:

    Receiving the following error when running.

    C:EMTshooterEMTshooter.ps1?
    Drinks Do not run  [R] Run once  Sleep Suspend  [?] Help (default is "D"): R

    Exception calling "FindOne" with "0" argument(s): "The specified domain either
    does not exist or could not be contacted.
    "
    At C:EMTshooterEMTshooter.ps1:125 char:25
    + $result = $query.findOne <<<< ()
       + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
       + FullyQualifiedErrorId : DotNetMethodException

  11. eladrez says:

    So after the issues i hat with the MT, i installed the MT on another machine and run the ExBPA, this morning the MT on the the original machine is seems to be working just fine.

  12. Steven Beard says:

    @ itgirl To resolve these issues create new text files, open the downloaded files then copy and paste all the code into the new text files.

    Rename the old files (.old) then rename the newly created text files to match the downloaded filenames.

    Check your execution policy is set to unrestricted (get-executionpolicy)

    The scripts should now run fine :)

  13. Marco says:

    Great stuff!  I ran into the same problem as itgirl and Elad, but then just ran "Set-ExecutionPolicy Unrestricted" and tried again.  It worked then.

    I was hoping that it would help me out with my "The WinRM client cannot complete the operation within the time specified", but not yet.  Still, I appreciated the tool.

  14. Just FYI – I tested this by changing the HTTP port to 81, then running EMTShooter. It detected the issue and added a HTTP binding for port 80 in.  I think this created problems with multiple CAS services, breaking EWS and Autodiscover.  I modified the bindings for the Default Website so that there was only one named "http" and it resolved the problem for me.

  15. Karsten says:

    @Steve

    I did today somthing that maybe helps to reproduce the Errormessage of Erwin: I now get the same error as him (Ex2010 SP1).

    The only thing I did was renewing the VMWARE Network Adapter:

    – I put "exchange sa service" to "manual", unistalled vmware tools and shut down the server

    – then I delteted the old adapter, and put in the new

    – then I booted the server

    – then I reainstalled the vmware tools=Driver,  put in the same IP to the nic as the old one had had and put exchange sa to auto again

    – after an other reboot I now get the same error as him – but using a different user doesn’t help

    Your tool says:

    Failed to connect to any Exchange Server in the current site.

    Problem found:

    Looking for error…

    Unknown Error

  16. Karsten says:

    Intersting – I just tried to reproduce this in my testenvironment: The missing NIC doesn’t seem to be the Problem: In my first test I omittet the step "setting sa to manual". The Error doesnt appear.

    After that I did now nothing but:

    – setting sa to manual,

    – rebootet in that state

    – set sa to auto again

    – rebooted agein

    – the problem occours

  17. Karsten says:

    Sorry for posting again – however that testsystem works now again – while the production system has still the error. I Don’t know why the testsystem temporarily seemed to show the error.

  18. Klaus says:

    Hi !

    Got the tool finaly runing, but it could not tell me what is wrong :-(

    Have an error of this kind "Connecting to the remote server failed with the following error message: The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol."

    Only information I receive from the tool is "Failed to connect to any exchange server in the current site" after the tool reports that EMT successfully comleted the connection to the server …

    And the last output is "unknown error" :-(

    Can send a screenshot if helpful.

    Thanks for everybodys support !

  19. William says:

    <quote>Then we noticed that the PowerShell vdir was missing all its modules, and quite a few other things. Somehow the PowerShell vdir on that Exchange Server had gotten severely… um… modified beyond repair</quote>

    I appear to have the same problem as described here – is there a way to do a repair installation or do I have to remove and re-install?

  20. Klaus says:

    I want to add two informations to my problem description:

    A) I found error warnings in the eventlog for Windows Remote Management, first event id 129, describing an HTTP error 404 and immediately after this event id 142 describing an WSMan error "CreateShell" with an error code 2150859027; all other commands performed OK in the log

    B) no listener is configured for WinRM, but it seems it never was before, and the Exchange GUI worked well till now.

  21. Klaus says:

    Case solved !

    I added new IP addresses to the network adapter for new websites on the IIS 7 – it seems this has started the problems. I just found a hint on that in this article:

    http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/5f6950b0-7215-479d-9e5e-fb93c76699ce#5f6950b0-7215-479d-9e5e-fb93c76699ce

    After changing the bindung for the Default website in IIS from a specific address (the one which was present at installation of Exchange) to "All unassigned" the Exchange Mangement GUI reacts normal now !

    Hope this is useful for others with such problem.

  22. Raghu says:

    Really stupid, but one solution that worked for me is to get in to the windows firewall properties and enable ICMP traffic.

  23. What do you suggest for:

    The WinRM client tried to use Kerberos authentication mechanism, but the destination computer (SERVER.domain.net:80) returned an ‘access denied’ error. Change the configuration to allow Kerberos authentication me

    chanism to be used or specify one of the authentication mechanisms supported by the server. To use Kerberos, specify th

    e local computer name as the remote destination. Also verify that the client computer and the destination computer are

    joined to a domain. To use Basic, specify the local computer name as the remote destination, specify Basic authenticati

    on and provide user name and password. Possible authentication mechanisms reported by server

  24. ChrisCompton says:

    I have an interesting issue.  We installed a secondary NIC on a server running Windows 2008 R2 and Exchange 2010.  After installing the second NIC, there are WinRM issues and EMC and Powershell will not connect up.  EMC gets an Initialization Failed: The Client Could not connect to the destination specified in the request.  

    Upon running this EMTshooter tool, I keep getting prompted that Port 80 is not bound.  Of course, when I check the bindings for the Default Web Site, it is.  If I change the port to something like 81, run EMTshooter, get the error and choose yes to update the bindings, it does add the Port 80 entry back in.  So, it does exist.  The problem I have is this Second NIC seems to have messed everything up.  I found another forum that discusses it.  Their only solution seems to be to format and start over.  Surely, there has to be another option.  We’ve tried removing the second NIC and reverting back to the way things were, but the problems still persist.  Here is a link to the other forum:  http://social.technet.microsoft.com/Forums/en-US/exchangesvradmin/thread/89b34787-321d-4657-9ffb-80889d65b7cd/

    Any suggestions?

  25. ChrisCompton says:

    Well, I just fixed it.  I came across some other forums.  While I thought it definitely had to do with the second NIC being installed, the issue ended up being the virus software solution (AVG 2011 in this case) that was causing the problem.  I would have never thought this in a million years, as we installed AVG 2011 a couple months ago and we have had access to the EMC just fine.  In fact, we had access to EMC moments before we installed the secondary NIC card and powered the server down to install it, as we were in there creating a new user account.  However, uninstalling AVG and rebooting the server seemed to fix the WinRM issues which seemed to be causing the EMC issues.  What a pain, as I have spent nearly all day chasing my tail.  I kept coming across the virus software as a possible issue all day, but never thought to try uninstalling it since it had been working just fine.  Lesson learned.  Hopefully this feedback might assist others with these problems.  

  26. Josh says:

    EMTShooter reports fine, but I am still getting:

    WARNING: Can’t generate Export-Module for the current session using Import-PSSession.

    Import-PSSession : Executing Get-Command command in remote session reported the following error: Starting a command on

    remote server failed with the following error message : The WinRM client cannot process the request. It cannot determin

    e the content type of the HTTP response from the destination computer. The content type is absent or invalid. For more

    information, see the about_Remote_Troubleshooting Help topic..

    At C:Program FilesMicrosoftExchange ServerV14binCommonConnectFunctions.ps1:116 char:48

    +             $global:importResults=Import-PSSession <<<<  $global:remoteSession -WarningAction SilentlyContinue -Disab

    leNameChecking

       + CategoryInfo          : InvalidResult: (:) [Import-PSSession], RuntimeException

       + FullyQualifiedErrorId : ErrorFromRemoteCommand,Microsoft.PowerShell.Commands.ImportPSSessionCommand

    Globals, kerb, all good – any ideas?

  27. josh says:

    Found it – web.config for the site had some absolute paths in it – there really shouldn’t be anything in the root web.config file unless you are using IIS redirects..

Comments are closed.