Workflow Parallell and Sequence….for the control freak


Following up on my last post on PowerShell WorkFlows , I thought I would take a moment to dive a little deeper into a topic that I personally could not find a lot of good details on, at least in a way that made sense to me:  How the Parallel and Sequence functions work within workflows.

Sequence

In the movie "As Good As It Gets", Jack Nicholson plays a character that battles with OCD.  One of his character quirks is that he compulsively sorts his candy by color into separate jars.  If you sympathize with that character or behavior, you're going to the love the control that the Sequence script block provides within a WorkFlow.

Sequence is useful for situations where you have a list of commands to run that have dependencies on the order in which they run (i.e. task #1 needs to run to completion before task #2).

The workflow script below basically is passed an array of machines, going through each server (in parallel, note the "-parallel" parameter in the ForEach line) to get the newest 2000 event 4264's (successful logons) from the security event log and returning the count of the event IDs found.  Next the event ID 4634's (logoff events) are queried and counted in the same manner.

WorkFlow ParallelSequenceTest
{
    param([Array[]]$Servers)

    ForEach -Parallel -ThrottleLimit 4 ($Server in $Servers)
    {
        Sequence
        {
            $EvntCnt = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4624'}).Count
            Write-Output "$Server $EvntCnt Event 4624's"
            $EvntCnt2 = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4634'}).Count
            Write-Output "$Server $EvntCnt2 Event 4634's"
        }
    }
}

You will notice that I have a "ForEach -Parallel" statement in the workflow with a "-ThrottleLimit 4" parameter. This limits the WorkFlow to 4 concurrent threads and sets the table for an explanation of how the Sequence statement within the WorkFlow works. Below is an illustration to help visualize how this works, based on a group of 9 servers.

As you can see in the script output below, the D4, DC3, DC2, & DC1 get all of the Event 4624's. Then they do the next query of event 4634's.  As noted above, the WorkFlow is limited to 4 threads, hence the 4 machine grouping.  Now the next group of machines (DC6, DC7, DC8, & DC5) repeat the same process of retrieving the 4624 Events, then moving on to the 4634 events.  Since I had 9 machines and limited the WorkFlow to 4 threads at a time, there is 1 lonely machine left to process, my Tools server.  True to form, the script processes the event 4624's then does the event 4634's.

Parallel

Since I have a movie theme going, I'm going to use my favorite Michael Keaton movie "Multiplicity" for comparison.  In Multiplicity, the lead character creates  a clone of himself so that he could do his demanding job while spending time with his family and getting all of his home improvement projects done. The movie gets interesting when the clones start making more clones...you get the picture. Doing a parallel statement within a parallel loop always makes me think of that movie.

The Parallel script block is useful for situations where you have a list of commands to run that have no dependencies on the order that they are run.  In my example code below. I am querying the Security Event Logs for event 4624 (successful logons) and Event 4634 (successful logouts).

With a slight modification, I have changed the processing behavior of the WorkFlow by changing the Sequence script block to a Parallel script block below:

WorkFlow ParallelParallelTest
{
    param([Array[]]$Servers)

    ForEach -Parallel -ThrottleLimit 4 ($Server in $Servers)
    {
        Parallel
        {
            $EvntCnt = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4624'}).Count
            Write-Output "$Server $EvntCnt Event 4624's"
            $EvntCnt2 = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4634'}).Count
            Write-Output "$Server $EvntCnt2 Event 4634's"
        }
    }
}

When this block of code runs. the behavior will be quite different.  To stay consistent for comparison's sake, I'll stick with 4 concurrent threads in the ThrottleLimit parameter and use the same 9 machines for processing.  Below is an overview of how the Parallel script block executes in this scenario.

As described above, each of the first 4 servers processes the event logs to return the event 4624 and the 4634's at the same time, then the next group of servers processes the events, finally leaving server #9 to process the events.

Sequencellel/Parallence?

Now it gets really weird...what would happen if we mix the Parallel and Sequence script blocks?  Let's find out.  I'm adding a new couple of Event ID's to the script, namely events 4769 (Kerberos service ticket requested) and 4770 (Kerberos ticket renewed).  The Sequence script block is the same as before, collecting the 4624/4634 events.  In addition, in a parallel script block, we will grab all of the 4679/4770 events.

If I am going to continue the movie theme for this script...."Primer" definitely comes to mind.  If you've never seen it, go watch it, you'll see what I mean.

WorkFlow ParallenceTest
{
    param([Array[]]$Servers)

    ForEach -Parallel -ThrottleLimit 4 ($Server in $Servers)
    {
        Sequence
        {
            $EvntCnt = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4624'}).Count
            Write-Output "$Server $EvntCnt Event 4624's"
            $EvntCnt2 = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4634'}).Count
            Write-Output "$Server $EvntCnt2 Event 4634's"
        }

        Parallel
        {
            $EvntCnt = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4769'}).Count
            Write-Output "$Server $EvntCnt Event 4769's"
            $EvntCnt2 = (Get-EventLog -LogName "Security" -Newest 2000 -ComputerName $Server| `
            Where-Object {$_.EventID -eq '4770'}).Count
            Write-Output "$Server $EvntCnt2 Event 4770's"
        }
    }
}

Below is the script output with Sequence and Parallel script blocks.  Notice that the first group of machines (DC4, DC3, DC2, & DC1) process the 4624's then the 4634's but for some reason DC1 takes longer than the others to return the 4634.   Before DC1 can return it's 4634 into the output, DC4 has moved on to the Parallel script block and returned it's 4769 & 4770 event counts.  The second group of threads (DC6, DC7, DC8, & DC5) now start processing.  DC 5 repeats the same issue seen with DC1 in the first group with DC6 moving on to the parallel script block before it can return it's 4634.  The Tools server, last but not least, finishes up the processing with the last thread.  As you can see, the output of this would be confusing to handle if you expected it to come back in a defined order or grouped together, no candy sorting here.

It's important to understand the processing behavior of these code blocks so you know the dependencies and how the output may be returned since it is different between the two script blocks, and definitely different with Sequence and Parallel mixed up.  Each one has it's place in the world and hopefully you are now armed with the knowledge to know when and how to use these to solve you toughest problems.

 

Comments (0)

Skip to main content