OpsMgr 2007 Event 21402

A colleague of mine Dirk van Coeverden has written an excellent article about troubleshooting OpsMgr 2007 Event 21402. With this article you can see how you can use the Workflow name from the Event details to trace which rule/discovery/monitor is causing the error.

Introduction

This article describes how to troubleshoot OpsMgr 2007 Event 21402, which indicates a script error or warning. However the article is intended to be an example for other events, since a lot of details are also in other events (like Workflow name).

         Event Type: Warning        Event Source: Health Service Modules        Event ID: 21402        Description:                Forced to terminate the following process started                 at 12:08:50 PM because it ran past the configured                 timeout 120 seconds.
  

With this article you will find

  • The discovery, monitor or rule which causes the event
  • Retrieve the script content in a SQL table (limited by SQL2005 Studio)
  • How it looks in XML and understand the workflow

The query to find the discovery, monitor or rule should work for every event where the workflow name is listed. The script content is more specific to 21402 but might give an impression on how to retrieve content from an XML columns (of NVARCHAR type).

The queries aren't tested for performance, so the best course is to import the MP of the customer in your test environment and run them on your own machine.

 

21402 details

An example of 21402 is

         Event Type: Warning        Event Source: Health Service Modules        Event Category: None        Event ID: 21402        Date: 7-4-2008        Time: 4:05:34        User: N/A        Computer: server01        Description:        Forced to terminate the following process started at         04:05:04 because it ran past the configured timeout 30         seconds.        Command executed: "C:\WINDOWS\system32        \cscript.exe" /nologo "C:\Program Files\System Center         Operations Manager 2007\Health Service State\Monitoring Host         Temporary Files 297\177        \CheckVirtualMachineNameMatchComputerName.vbs"        Working Directory: C:\Program Files\System Center Operations         Manager 2007\Health Service State\Monitoring Host Temporary         Files 297\177\         One or more workflows were affected by this.          Workflow name: Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineName_does_not_match_computer_name.rule         Instance name: servern01.contoso.com         Instance ID: {3B3FA6E2-BB6B-CD49-274A-8722250C3D0C}         Management group: OpsMgrdemo        For more information, see Help and Support Center at         https://go.microsoft.com/fwlink/events.asp.

Important Event Elements

Command Executed:

"C:\WINDOWS\system32cscript.exe" /nologo "C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 297\177\CheckVirtualMachineNameMatchComputerName.vbs"

Working Directory:

C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 297\177\

Script:

CheckVirtualMachineNameMatchComputerName.vbs

Workflow:

Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineName_does_not_match_computer_name.rule

Which discovery monitor or rule?

To find out which discovery monitor or rule the script belongs to run the following SQL script (based on the example event above) on the OperationsManager database.

  DECLARE @ObjectName NVARCHAR(256) SET @ObjectName = 'Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineName_does_not_match_computer_name.rule' IF EXISTS (SELECT 1 FROM DiscoveryView WITH (NOLOCK) WHERE Name = @ObjectName)         SELECT                 'Discovery' As 'Object Type',                d.DisplayName AS 'Displayname in Console',                d.Name AS 'Internal Monitor Name',                d.Id AS 'MonitorId',                p.Displayname AS 'ManagementPack',                p.Version AS 'ManagementPack Version',                p.Name AS 'Management Pack Library Name'        FROM DiscoveryView d WITH (NOLOCK)        INNER JOIN ManagementPackView p WITH (NOLOCK) ON d.ManagementPackId = p.Id        WHERE d.Name = @ObjectName ELSE IF EXISTS (SELECT 1 FROM MonitorView WITH (NOLOCK) WHERE Name = @ObjectName)         SELECT                 'Monitor' AS 'Object Type',                m.DisplayName AS 'Displayname in Console',                m.Name AS 'Internal Monitor Name',                m.Id AS 'MonitorId',                p.Displayname AS 'ManagementPack',                p.Version AS 'ManagementPack Version',                p.Name AS 'Management Pack Library Name'        FROM MonitorView m WITH (NOLOCK)         INNER JOIN ManagementPackView p WITH (NOLOCK) ON m.ManagementPackId = p.Id        WHERE m.Name = @ObjectName ELSE IF EXISTS (SELECT 1 FROM RuleView WITH (NOLOCK) WHERE Name = @ObjectName)         SELECT                 'Rule' AS 'Object Type',                r.DisplayName AS 'Displayname in Console',                r.Name AS 'Internal Rule Name',                r.Id AS 'RuleId',                p.Displayname AS 'ManagementPack',                p.Version AS 'ManagementPack Version',                p.Name AS 'Management Pack Library Name'        FROM RuleView r WITH (NOLOCK)         INNER JOIN ManagementPackView p WITH (NOLOCK) ON r.ManagementPackId = p.Id        WHERE r.Name = @ObjectName

This will give the following results

Object Type Displayname_in_Console Internal Rule Name RuleId ManagementPack ManagementPack Version Management Pack Library Name
Rule Virtual Machine: Virtual machine name does not match computer name Microsoft.Virtualization.VirtualServer.2005R2. VirtualMachineName.rule E465679D-BC55-C4D3-FEF0-C108DF37DFD9 Microsoft Virtual Server 2005 R2 1.0.2627.0 Microsoft.Virtualization.VirtualServer.2005R2

In the OpsMgr Console / Authoring you can find the Discovery, Monitor or Rule (depending on Object Type) in Management Pack Objects.

Retrieve the script content

To retrieve the script you can run the following SQL query on the OperationsManager database.

  SELECT          ManagementPackId,         ScriptName,         ScriptFile FROM (                 SELECT                  ManagementPackId,                 Script.Col.value('(Name/text())[1]', 'NVARCHAR(128)') AS ScriptName,                 Script.Col.value('(Contents/text())[1]', 'NVARCHAR(MAX)') AS ScriptFile         FROM (SELECT ManagementPackId, CONVERT(XML, MPXML) AS MPXMLFormat, MPName FROM ManagementPack) p          CROSS APPLY p.MPXMLFormat.nodes('//File') Script(Col)         WHERE p.MPName LIKE '%2005R2%') s WHERE s.ScriptName = 'CheckVirtualMachineNameMatchComputerName.vbs'

Take note that SQL 2005 Studio has a limit of (about) 8000 char for its return results. Also since the MPXML field of the ManagementPack table is not of type XML but NVARCHAR, the formatting isn't really fancy. However it might give you a quick impression of what the script is about.

If people find this a useful query, I can work out one which gives a nice formatted output of the script content.

In XML

If you want to see how it is build up and configured in XML than

  • Export MP (Powershell Export-ManagementPack)
  • In .xml search for <Rule ID="Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineName_does_not_match_computer_name.rule"

This looks like

        <Rule ID="Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineName_does_not_match_computer_name.rule" Enabled="true" Target="Windows! Microsoft.Windows.Computer" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100">         <Category>AvailabilityHealth</Category>         <DataSources>           <DataSource ID="DS" TypeID="Microsoft.Virtualization.VirtualServer.2005R2.CheckVirtualMachineName">            <IntervalInSeconds>60</IntervalInSeconds>             <Expression>               <And>                 <Expression>                   <SimpleExpression>                     <ValueExpression>                       <XPathQuery Type="String">Property[@Name='IsVirtualMachine']</XPathQuery>                     </ValueExpression>                     <Operator>Equal</Operator>                     <ValueExpression>                       <Value Type="String">True</Value>                     </ValueExpression>                   </SimpleExpression>                 </Expression>         ...

Here you find all the properties of the (in this case) the rule. Actually you can read the workflow of this object here. One part of the workflow is the Databasource ModuleType

  <DataSource ID="DS" TypeID="Microsoft.Virtualization.VirtualServer.2005R2.CheckVirtualMachineName">

This points to the ModulesType part of the XML file

       <ModuleTypes>       <DataSourceModuleType ID="Microsoft.Virtualization.VirtualServer.2005R2.VirtualServerVirtualMachineDiscovery" Accessibility="Internal" Batching="false">       <DataSourceModuleType ID="Microsoft.Virtualization.VirtualServer.2005R2.RelVirtualMachineComputerDiscovery" Accessibility="Internal" Batching="false">       <DataSourceModuleType ID="Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineComputerDiscovery" Accessibility="Internal" Batching="false">       <DataSourceModuleType ID="Microsoft.Virtualization.VirtualServer.2005R2.VirtualMachineState" Accessibility="Internal" Batching="false">       <DataSourceModuleType ID="Microsoft.Virtualization.VirtualServer.2005R2.CheckVirtualMachineName" Accessibility="Internal" Batching="false">        <Configuration>          <IncludeSchemaTypes>            <SchemaType>System!System.ExpressionEvaluatorSchema</SchemaType>          </IncludeSchemaTypes>          <xsd:element name="IntervalInSeconds" type="xsd:integer" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />          <xsd:element name="Expression" type="ExpressionType" xmlns:xsd="https://www.w3.org/2001/XMLSchema" />        </Configuration>        <OverrideableParameters>          <OverrideableParameter ID="IntervalInSeconds" Selector="$Config/IntervalInSeconds$" ParameterType="int" />        </OverrideableParameters>       ....

Here you will find the Module itself which is executed

              <MemberModules>               <DataSource ID="DS" TypeID="System!System.CommandExecuterPropertyBagSource">                 <IntervalSeconds>$Config/IntervalInSeconds$</IntervalSeconds>                 <ApplicationName>%windir%\system32\cscript.exe</ApplicationName>                 <WorkingDirectory />                 <CommandLine>/nologo $file/CheckVirtualMachineNameMatchComputerName.vbs$</CommandLine>                 <TimeoutSeconds>30</TimeoutSeconds>                 <RequireOutput>true</RequireOutput>                 <Files>                   <File>                     <Name>CheckVirtualMachineNameMatchComputerName.vbs</Name>                     <Contents>         ' Copyright (c) Microsoft Corporation. All rights reserved. ' VBScript source code ' CheckVirtualMachineNameMatchComputerName.vbs ' Arg 0 : SourceID Option Explicit  Const StrVMMManagementGroupInstallationRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Server\Setup\InstallPath" Const StrVirtualServerRegKey = "HKLM\System\CurrentControlSet\Services\Virtual Server\Start" Const StrVMMServerInstallationRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Server\Setup\InstallPath" Const StrVMMServerVersionRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Server\Setup\ProductVersion" Const StrVMMSelfServiceServerInstallationRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Self-Service  Portal\Setup\InstallPath" Const StrVMMSSsiteEngineMachineRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Self-Service  Portal\Settings\VmmServerName" Const StrVMMDatabaseServerRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Server\Settings\Sql\OnRemoteServer" Const StrVMMDatabaseNameRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Server\Settings\Sql\DatabaseName" Const StrVMMDatabaseInstanceRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Server\Settings\Sql\InstanceName" Const StrVMMRemoteDatabaseMachineFQDNRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007  Server\Settings\Sql\MachineFQDN" Const StrVMMConsoleInstallationRegKey = "HKLM\SOFTWARE\Microsoft\Microsoft System Center Virtual Machine Manager 2007 Administrator Console\Setup\InstallPath" Const StrVMNameRegKey = "HKLM\SOFTWARE\Microsoft\Virtual Machine\Guest\Parameters\VirtualMachineName" Const StrWebsitDisplayName = "Microsoft System Center Virtual Machine Manager 2007 Self-Service Portal" Const StrSQLServerDefaultInstance = "MSSQLSERVER" Const StrEmpty = "" Const StrFolderSeparator = "\" Const StrLocationHost = "Host" Const StrLocationLibrary = "Library" '============= ' Method:       ReadRegistry ' Description:  This function read the regiestry, return true if the registry exit; otherwise return false '============= Function ReadRegistry(ByVal regString, ByRef regValue)     Dim bHasRegistry     Dim oReg     Set oReg = CreateObject("WScript.Shell")        On Error Resume Next         regValue = oReg.RegRead(regString)     If Err.number &lt;> 0 Then         bHasRegistry = False     Else         bHasRegistry = True     End If     On Error Goto 0      ReadRegistry = bHasRegistry End Function  Call Main() ...