How to Submit a Job

I was asked today where there was a quick explanation of how to submit a job from the command line, and I realized that I didn't have a really good answer. So I figured I'd do a blog post (first in a while, I know!) on how to do this.

Starting Simple

Let's say your have an application called "Divide.exe" which takes two arguments, "-Numerator X" and "-Denominator Y". So for example, to divide 3/6 you would run the command:

divide.exe –Numerator 3 –Denominator 6

So now you want to run this thing on a single processor on some node in the cluster. It's really easy! You can just do:

job submit divide.exe –Numerator 3 –Denominator 6

No problem, right?

Getting Parallel

Of course if you're using a cluster, you don't just want to run one command line on one processor of one machine. You want to run in parallel! There are a few ways that you can create parallel jobs depending on the way that your application works.

The two most common ways to do this in HPC are using MPI or using Parameter Sweeps.

Submitting MPI Jobs

Submitting MPI jobs is just as easy as submitting any other job! You simply do a job submit followed by your mpi command line. The number of options offered by mpi is astounding (try mpiexec –help3 for more on that) and will probably be covered in a future post, but the basic command line to submit a 16 processor mpi job would be:

Job submit /numcores:16 mpixec MpiDivide.exe –Numerator 3 –Denominator 6

Submitting a Parameter Sweep

We use the term "parameter sweep" to refer to a job which encompasses running a single, serial application many, achieving parallelism by running many instances of it at the same time. In HPC Server (and in the Compute Cluster Pack), this is accomplished by creating a job with many tasks. So for example, you could create a job that divded 3 by 6, 9, and 12 by doing the following (where X is the job ID returned by the first step):

Job new /jobname:"My Parameter Sweep Job"

Job add X divide.exe –Numerator 3 –denominator 6

Job add X divide.exe –Numerator 3 –denominator 9

Job add X divide.exe –Numerator 3 –denominator 12

Job submit /id: X

New in HPC Server 2008, we have a much simpler (and faster!) way of running parameter sweeps using wildcards! This approach will save you a lot of time since they're much easier to create and because the Job Scheduler needs to store much less information for a Parametric Task than it does for job with many distinct tasks. An example of how to do the same step above using this new technique follows:

Job submit /parametric:6-12:3 divide.exe –Numerator 3 –denominator *

For more info on how to use the new HPC Server parametric sweeps, see my earlier blog post, Making a Clean Sweep with Windows HPC Server 2008.

For more info on the command line tools in general, check out our Command Line Reference. Unfortunately it's not yet updated for v2 . . . but there should be a new one out on the web in just a few weeks!