What's new in Azure Batch July release (General Availability)

Azure Batch service just announced general availability (see this blog post for detail). Comparing to the preview version, here is a list of what has changed.

The New Portal

Batch is now available in the new Azure portal - https://portal.azure.com. Besides account management functionality, the portal provides Batch pool and job monitoring capability. We'll have a separate article discussing this.

Change of Batch Service REST API 

• Batch URL becomes regional from https:// <account>. batch.core.windows.net to https:// <account> . <region>. batch.azure.com.
• Workitem has been removed. Instead of creating a workitem and wait for job, client application should create a Job object directly. Most properties on the workitem has been moved on to job. The only exception is the functionality of running recurring job. This capability goes to the newly introduced JobSchedule entity. With JobSchedule, client can configure Batch service to launch recurring jobs based on a schedule.
• Pool entity remains the same but TVM is renamed to ComputeNodes. This also impact APIs related to VM operation (for example, List TVM -> List ComputeNodes).
• All entities now have both Id and DisplayName field. If not specified, .Net client library will generate a GUID for the Id field automatically. REST API user have to generate a unique Id for the entities. DisplayName doesn’t have to be unique and is optional.
• A new set of environment variables (prefixed with “AZ_BATCH_”) appear side by side with the old environment (prefixed with “WATASK_”). (See below.)
• Sample rate of resource metrics used in auto-scale formula has been changed from 5 seconds to 30 seconds.
• Added support to list all job preparation and release tasks on a specified job.
• A new task state – Preparing – is added to indicate that the task has been assigned to a compute node but is waiting for job preparation task to complete.
• Change in autoscale formula - autoscale metric $TVMDeallocationOption is renamed to $NodeDeallocationOption
• Change in autoscale formula - autoscale metric $SampleTVMCount is renamed to $SampleNodeCount
• Change in autoscale formula - function doubleVec vec(doubleVecList) is removed.

Change in .Net client library

• Reflect changes on REST API.
• Removing C# interfaces when it's not necessary. This means ICloudPool, ICloudJob, ICloudTask are all going away. Function signature will reference to the corresponding class directly.
• Protocol API (Microsoft.Azue.Batch.Protocol) will be replaced by Hyak-generated client. The interface has been changed.
• Minor changes based on feedback.
  • Refactor VMFile and TaskFile into 1 class - NodeFile.
  • Instead of using BatchCredential, a sub-class - BatchSharedKeyCredential - is created. Client should use the sub-class to create credential object. This is changed to get the client library ready for new authentication mechanism in the future.
  • Toolbox class is renamed to Utilities.
  • *Manager classes are renamed to *Operations, e.g. PoolManager -> PoolOperations.

Backward Compatibility

• To access aforementioned changes, a new REST API version will be exposed (“2015-06-01.2.0”) together with a new .Net client library through Nuget (v2.0.0).
• The current REST API (from public preview) and .Net client will still be available and fully functional. Public preview customer will not lose any functionality or see any behavior change.
• We’ll deprecate the public preview REST and .Net API in the future. We'll announce the timeline and plan later.
• Both set of (old and new) API will have the same view of entities in the service. Specifically, 1 none recurring workitem/job pair in the old API will map to 1 job in the new API. And 1 recurring workitem will map to 1 job schedule in the new API. Each job under the recurring workitem will map to a job in the new API. Operations on the object from either side will reflect in the other set of API.

Migration Guide

For any new application, please start with the new API since the old one will go away.

For current application, moving to the new API is straightforward.

1. If client submits non-recurring workitem, replace it with Job submission.
   1. All workitem properties is included in the new Job object.
   2. There is no need to wait for the job anymore. Client can start adding Tasks to the job immediately.
   3. If client use recurring workitem, replace it with Job Schedule creation.
      1. Job will appear when the schedule is triggered.
      2. JobSchedule contains JobSpecification which is identical to workitem.
      3. If the Task is using environment variables on the VM, replace it with the new one. (Instead of using hard coded directory name, always use the environment variable.)

 NOTE:

1. Pool/VM/Task and other entities are not changed.
2. Remember to change the URL since now it’s regional.
3. For client using protocol layer directly (under namespace Microsoft.Azure.Batch.Protocol), migrate the code to new REST Client in the .Net API. For client using convenience layer (under namespace Microsoft.Azure.Batch), most changes are just a string replacement. (E.g., TVM is renamed to ComputeNode, Task/VMFile is renamed to NodeFile. There is no structural change though.