PowerShell Workflows: Job Engine

Summary: Windows PowerShell MVP Richard Siddaway continues his workflow series by talking about the job engine. Microsoft Scripting Guy, Ed Wilson, is here. Today, we have the fourth in a series of guest blog posts written by Windows PowerShell MVP and Honorary Scripting Guy Richard Siddaway dealing with Windows PowerShell workflow.

Note   The first article, PowerShell Workflows: The Basics, introduced the basic concepts of Windows PowerShell workflow. The second article, PowerShell Workflows: Restrictions, discussed the restrictions encountered with working with Windows PowerShell workflows. The third article, PowerShell Workflows: Nesting, talks about nesting workflows. You should read those articles prior to reading today’s article. Richard has written a number of guest Hey, Scripting Guy! Blog posts, and he has also written two books on Windows PowerShell. His most recent book, PowerShell in Depth, is co-written with fellow MVPs Don Jones and Jeffrey Hicks. Now, take it away, Richard … Remember the first workflow you saw in this series—it was a simple “Hello World.”

workflow hello {

 “Hello World”

} You could run it like this:

PS> hello

Hello World

Running a workflow as a job

All workflows have the ability to run as a Windows PowerShell Job:

PS> hello -AsJob -JobName w1

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

9      w1              PSWorkflowJob   NotStarted    True            localhost You get the –AsJob parameter, which starts the workflow as a Job, and you also get the –JobName parameter, which allows you to give your own name to workflow’s Job. Contrast that with other situations where you only get the ability to run a cmdlet as a Job, for example, with the WMI cmdlets you don’t get the ability to give the Job a name. The other important point to note is the PSJobTypeName—it’s PSWorkflowJob. This is a new Job category introduced in Windows PowerShell 3.0, just for workflows. After you’ve started your workflow as a Job, it is handled as any other Job:

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

9      w1              PSWorkflowJob   Completed     True            localhost

 

PS> Receive-Job -Name w1

Hello World   By definition, workflows are long-running tasks for which you don’t expect to provide interactive input. They are ideally suited for running as Windows PowerShell Jobs. There’s a bit more to workflows and the way they work with the Windows PowerShell Job engine, which is just as well as otherwise I wouldn’t have enough to write about.

Stopping and starting workflows

Windows PowerShell workflows are built on top of the Windows PowerShell Job engine. One of the capabilities this gives you is the ability to suspend and resume the workflow. Let’s simulate a long-running workflow:

workflow test-wfsuspension {

 Start-Sleep -seconds 10

 Suspend-Workflow

 Get-ChildItem

}

  The workflow will sleep for 10 seconds and then suspend itself through the call to Suspend-Workflow. Once the workflow has suspended, you will see a display similar to this:

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

7      Job7            PSWorkflowJob   Suspended     True            localhost    Notice that the State is set to Suspended. The data and state of the workflow have been saved to disk. You can restart the Job by using the Resume-Job cmdlet:

PS> Resume-Job -Id 7

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

7      Job7            PSWorkflowJob   Suspended     True            localhost  

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

7      Job7            PSWorkflowJob   Completed     True            localhost  The data returned from the Resume-Job cmdlet may mislead you. It states that the job is Suspended! The cmdlet reports the state of the Job before it tells it to restart. After it is restarted, the Job completes and the data is available as usual. This is useful but what is more useful is the ability to suspend a workflow Job, and then restart it in a different Windows PowerShell session! If you remove the old jobs and rerun the test-wfsuspension workflow, you’ll get something like this:

PS> test-wfsuspension

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

9      Job9            PSWorkflowJob   Suspended     True            localhost  I’m running these demonstrations in ISE, but this still works if you are using the Windows PowerShell console. Shut down your PowerShell session and you don’t need to save any files. That’s right. Just click the little close button at the top-right corner of the Windows PowerShell or ISE window. This is safe to try at home—honest. Now open up a new session (you need to run it with elevated privileges). I’m going to use ISE again, but feel free to use the console if you want to. Now try Get-Job, and you’ll see … nothing! OK, don’t panic. Import the Windows PowerShell workflow module again, and you’ll see your job:

PS> Get-Job

PS> Import-Module PSWorkflow

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

8      Job9            PSWorkflowJob   Suspended     True            localhost  You can then resume the job and retrieve the data when appropriate:

PS> Get-Job

PS> Import-Module PSWorkflow

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

8      Job9            PSWorkflowJob   Suspended     True            localhost  Notice that the job ID may change between sessions although the job name will remain constant.

Suspending a workflow job

There is another way to suspend workflow jobs—that’s by using the Suspend-Job cmdlet. Start with a workflow:

workflow test-wfsr {

 Start-Sleep -seconds 30

 Checkpoint-Workflow

 Get-ChildItem 

} Start the workflow as a Job:

PS> test-wfsr -AsJob

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

11     Job11           PSWorkflowJob   Running       True            localhost

 

PS> Suspend-Job -Id 11

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

11     Job11           PSWorkflowJob   Suspending    True            localhost  You can then use Suspend-Job to suspend the job. The job will show a State of Suspending until the Job reaches the CheckPoint-Workflow statement. At this point, the Job enters a Suspended state:

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

11     Job11           PSWorkflowJob   Suspended     True            localhost  You can then resume the job when appropriate:

PS> Resume-Job -id 11

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

11     Job11           PSWorkflowJob   Suspended     True            localhost  

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

11     Job11           PSWorkflowJob   Completed     True            localhost   In this case, I’ve resumed the Job in the same session. A very important point to remember is that the Suspend-Job and Resume-Job cmdlets only work on workflow jobs. You will get an error if you try to suspend other job types! In the last example, you need to use the Checkpoint-Workflow activity. This writes a copy of the workflows data and state to disk so that the job can be resumed. In other words, it takes a snapshot. If you don’t take that snapshot, the workflow has nowhere to suspend and it will effectively ignore the suspension. Here is a summary of the two suspension techniques:

• If you want the suspension to be triggered from inside the workflow, use Suspend-Workflow.
• If the suspension is to be triggered from outside the workflow, use Checkpoint-Workflow and Suspend-Job.

Checkpoints

Checkpoints are a good technique for saving the current state of your workflow so that you can restart it if the machine or session is terminated. You can only restart from the last checkpoint taken—you can’t choose which checkpoint to use. Checkpoint data is saved in your user profile on the system you are using to run the workflow. You could add a checkpoint after every activity but there is a balance between the time needed to write the data and state to disk compared to the time needed to rerun the workflow. You have seen Checkpoint-Workflow being used—there are a number of other ways of adding checkpoints to your workflows:

• Checkpoint-Workflow – Can be used after any activity, but not inside an InlineScript block. It takes an immediate checkpoint.
• PSPersist workflow parameter – Adds checkpoints at the beginning and at end of the workflow and after each activity. Does not modify any explicit checkpoints in the workflow.
• PSPersist activity parameter – Takes a checkpoint after the activity completes. This is not valid on expressions or commands in an InlineScript block.
• $PSPersistPreference preference variable – When set to true, takes a checkpoint after every following activity until it’s reset to false. Only effective within workflows.

If your activity is in a pipeline and it’s checkpointed, the checkpoint doesn’t apply until the pipeline completes. Within parallel blocks, the checkpoint doesn’t apply until the parallel processing has been applied to all items compared to a sequence block when checkpoints are applied after each activity. These rules are best illustrated with some code. By using the Checkpoint-Workflow activity you’ve seen before, you can checkpoint after every activity like this:

workflow test-wfchkpnt {

 Get-WmiObject -Class Win32_ComputerSystem

 Checkpoint-Workflow

 Get-WmiObject -Class Win32_OperatingSystem

 Checkpoint-Workflow

 Get-WmiObject -Class Win32_LogicalDisk

 Checkpoint-Workflow

 }

  In this example, the same result of checkpointing after every activity can be achieved by using the –PSPersist workflow parameter.

workflow test-wfchkpnt {

 Get-WmiObject -Class Win32_ComputerSystem

 Get-WmiObject -Class Win32_OperatingSystem

 Get-WmiObject -Class Win32_LogicalDisk

 }

 test-wfchkpnt -PSPersist  This next example uses the –PSPersist activity parameter to perform a checkpoint after each individual activity:

workflow test-wfchkpnt {

 Get-WmiObject -Class Win32_ComputerSystem -PSPersist

 Get-WmiObject -Class Win32_OperatingSystem -PSPersist

 Get-WmiObject -Class Win32_LogicalDisk -PSPersist

 }  The same result is achievable by using the preference variable:

workflow test-wfchkpnt {

 $pspersistpreference = $true

 Get-WmiObject -Class Win32_ComputerSystem

 Get-WmiObject -Class Win32_OperatingSystem

 Get-WmiObject -Class Win32_LogicalDisk

 $pspersistpreference = $false

 }  My preference is to use Checkpoint-Workflow to perform explicit checkpoints at places of my choosing. Let’s see how checkpointing works. This simple workflow checkpoints itself after every iteration of the loop.

workflow test-wfchkpnt {

 $i = 0

 while ($true){

   $i++

   $i

   Checkpoint-Workflow

 } 

}  You need to start the workflow as a Job.

PS> test-wfchkpnt -AsJob

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

7      Job7            PSWorkflowJob   Running       True            localhost

 

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

7      Job7            PSWorkflowJob   Running       True            localhost    After the Job has been running for a few seconds, shut down Windows PowerShell. Open a new session (with elevated privileges), import the Windows PowerShell module, and then view the available jobs.

PS> Import-Module PSWorkflow

PS> Get-Job

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

8      Job7            PSWorkflowJob   Suspended     True            localhost

 

PS> Resume-Job -Id 8

Id     Name            PSJobTypeName   State         HasMoreData     Location 

—     —-            ————-   —–         ———–     ——– 

8      Job7            PSWorkflowJob   Suspended     True            localhost

 

PS> Stop-Job -Id 8

PS> Receive-Job -Id 8 -Keep | select -f 3

WARNING: The workflow job “Job7” was stopped. Receive-Job is only displaying par

tial results.

1

2

3   Resume the Job, and let it run for a few seconds. The workflow is an infinite loop, so stop the Job.  Use Receive-Job to pull the data back.

Summary

Windows PowerShell workflows work with the Windows PowerShell job engine. You can run workflows as jobs. Workflows provide the ability to suspend and restart their jobs. Checkpointing the workflow saves the state so your workflow can survive a session failure. Next time, you’ll see how workflows can survive a computer restart! ~Richard Thank you, Richard! Your series on Windows PowerShell workflows is both important and timely. Great job, and I cannot wait for next Wednesday’s article. Join me tomorrow when Charlotte Windows PowerShell User Group member Brian Wilhite talks using Windows PowerShell to work the mouse on his computer. I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.

Ed Wilson, Microsoft Scripting Guy