As I work in different Orchestrator environments I find it very helpful to monitor for jobs that have been in a running state for a long period of time (excluding any Runbook that begins with a monitor activity). This ultimately help identify Runbook logic issues, infinite loops, and other environmental issues.
We can identify Runbooks that have been running for longer than X minutes using the following query.
Query: Value 1440 in minutes
Select POL.Path, JOB.Id, JOB.CreationTime
FROM [Orchestrator].[Microsoft.SystemCenter.Orchestrator].[Runbooks] POL
JOIN [Orchestrator].[Microsoft.SystemCenter.Orchestrator.Runtime].[Jobs] JOB on Pol.Id = JOB.RunbookId
Where POL.IsMonitor = 0 and Status = 'Running' and DATEDIFF(MI,JOB.CreationTime,GETUTCDATE()) > 1440
Generally when investigating an active Orchestrator environment I will start this process at one day and then work my way back to those that have been running for over just a few hours.
Once I’ve identified these long running jobs, I can then begin investigate the Job using the Runbook Designer (or DB data) looking for issues. In some cases it is expected to have a longer running Runbook and that is ok, however I find that when running through this exercise more often than not several actual issues are uncovered.
Good Weekend All.