Parameter Sweeps are one of the most common types of jobs that get run on HPC clusters, and we've done some work in Windows HPC Server 2008 to make them easier (and faster) than ever. Today I'll dig in a little on what's different and how to take advantage of these new features.
For those familiar with other scheduling products, you may recognize these as very similar to "Job Arrays."
What is a Parameter Sweep?
Simply put, a parameter sweep is when you run a single command many times over a set of different input parameters. Problems which can be solved in this way are very common. They're also a great use of HPC clusters, since parameter sweeps are inherently embarrassingly parallel; namely they can be run in parallel with little or no effort and scale almost limitlessly.
In general, Parameter Sweeps take the form of a single command line which is run N times, with different input, output, and/or command line arguments for each of the N steps. For example, think about an application called FileZipper.exe that takes an input file and generates a compressed output file. To run it on 100 data files, you'd basically want to do something like:
FileZipper.exe <FileToZip1.dat >ZippedFile1.zip
FileZipper.exe <FileToZip2.dat >ZippedFile2.zip
FileZipper.exe <FileToZip3.dat >ZippedFile3.zip
FileZipper.exe <FileToZip100.dat >ZippedFile100.zip
These instances can already independently of one another, and you can run as many in parallel as you have processors to do the compression with.
Make sense? If it does, then you already understand pretty much everything there is to know about Parameter Sweeps.
What's different in Windows HPC Server 2008?
In the Compute Cluster Pack (our product from 2005), parameter sweeps could be generated pretty easily from the UI by inputting the Start Index (the number to start counting from), End Index (the last number to use use), and the Increment (the number to add for each step in the sweep). Using the UI to create such a sweep would then create N individual tasks in the job. This was useful, but had a number of downsides, namely:
- You're storing a lot of repeated information in the scheduler database
- If you wanted to change a part of your sweep, you had to change every task
- Large sweeps became very unwieldy to view in the UI or at the command line
In Windows HPC Server 2008, we add a new type of task called a Parametric Task:
Figure 1: Adding a Parametric Task to a Job
Now, these tasks are stored as a unit, which makes, storing, editing, managing, and monitoring them a snap! Let's go ahead and give it a try . . .
Creating a Parameter Sweep
To create your first parameter sweep, open up the HPC Job Manager. The simplest way to create a sweep is to click on the Parametric Sweep Job link in the right-hand Actions Pane.
Let's create a sweep in the form of the example up above; namely, a sweep that zips up 100 files:
1. Go ahead and provide a Name like "File Zipper".
2. Set the Start Value of 1 and an End Value of 2.
3. Leave the Increment Value to be 1 (since we'll be counting up by 1's).
4. Enter your Command Line, in this case "FileZipper.exe".
5. We'll see the Standard Input and Standard Output as above:
1. Stdin: FileToZip*.dat
2. Stdout: ZippedFile*.zip
6. Check the preview box at the bottom of the dialog to see what your sweep will look like, then go ahead and submit.
You should end up with something that looks like this:
Figure 2: Creating a Parameter Sweep Task
Tracking Your Sweep's Progress
If you check the job list, you should now see that you've submitted a job with a single task in it. But actually, you can easily track each task individually by checking the box labeled Expand parametric tasks. This allows you track your sweep as a unit, but dig in on failures or results for individual steps.
Doing that From the Command Line
Of course you can do the same thing from the Command Line:
C:\>job submit /parametric:100 /StdIn:"FileToZip*.dat" /StdOut:"ZippedFile*.zip" FileZipper.exe
Or from PowerShell:
PS> New-HpcJob | Add-HpcTask -Parametric -Start 1 -End 100 -Stdin "FileToZip*.dat" -Stdout "ZippedFile*.zip" -CommandLine "FileZipper.exe" | Submit-HpcJob
That's all for this time. Happy sweeping!