Testing Hot Fixes, Cumulative Updates and Service Packs

It is often a challenge to keep up with supporting your own applications, let alone changes to the underlying platforms, such as Microsoft SQL Server, that the application supports. In the face of these changing conditions, the only way to ensure that your application remains functional, reliable, and high-performing is through ongoing testing. 

Let's first understand the relationship between Hotfixes, Cumulative Updates (CU), and Service Packs (SP).

Hotfix: The smallest unit of change for SQL Server

Cumulative Update (CU) : A hotfix package containing all the changes to that point

Service Pack (SP) : A collection of all previous cumulative updates (which contains all the previous hotfixes)

The more testing you do along the way, the less painful it is to support each new version and update. That may not be intuitive, but here’s the rationale: By testing at smaller intervals (such as at CUs), you are testing many, if not all, of the changes that eventually go into SPs as well as the next major version of SQL Server. If you are testing at each level of update along the way, testing an SP, for example, should be no big deal because a lot of what is in the SP has already been assessed. The number of changes that could affect your application goes up, not down, as time goes on. If a problem occurs somewhere along the way and, for the sake of argument, needs a fix because it breaks your application, it is easier to deal with the problem in a smaller increment than in a more comprehensive package after you’ve waited months or years between testing of the underlying platform. Although such frequent testing may seem like a lot of work, if you have good automation, the process should not place a great burden on your organization.

One of the hardest things any ISV has to do is manage the risk associated with the need to accept change, whether that change is made to its own application or underlying platforms. When it comes to Microsoft SQL Server, as more time passes, the delta of change between older and newer versions becomes larger and larger. If an application has not been tested against newer versions of SQL Server released since the application was introduced, not only is the ISV facing huge risk with possibly years of change to account for, but supportability may have been compromised for both the ISV and its customers.

Check out the Testing and Developing Supportability Roadmaps for ISV Applications document for more information.