Optimizing Operating Systems Upgrades for Remotely Connected Clients


Hi everyone. My name is Kacey Krehbiel. I'm a service engineer responsible for application and operating system upgrade deployments within Microsoft. Today I'd like to focus on one small aspect of our operating system upgrade process.

Over the past couple of operating system upgrade cycles we have seen a huge increase in the numbers of users that are upgrading their operating system while being connected over VPN. An edge case that in the past would be dismissed with an answer of, "Wait until you're in the office," or "Sorry, the system wasn't built for that," is not acceptable. Users may never go into the office. With that in mind, when designing an OS upgrade using a Configuration Manager task sequence we need to make sure that we are making special consideration for remote clients.

Choosing between Windows 10 Servicing and Operating System Upgrade Task Sequences

A note on Windows 10 Servicing: Windows 10 Servicing is a great solution for installing Windows 10 feature updates for many environments that simplifies many of the challenges of a large-scale feature update deployment. Due to unique internal deployment timelines, as well as some customization specific to our environment, we have opted to use the Operating System Upgrade Task Sequence feature present in Configuration Manager. We encourage everyone to research the capabilities of Windows 10 Servicing, and determine if it is appropriate for your environment.

Pre-Download

The first hurdle we need to overcome with remotely connected clients is ensuring that the download of the upgrade package is successful. If we get one chance at time of enforcement to download a 4GB operating system package, we are going to have some failures if users are getting that one chance while seated at a coffee shop. Pre-download helps to reduce that issue by increasing the number of times we attempt download before mandatory enforcement.

Every one of our required operating system upgrade deployments is available to users for at least 7 days before the mandatory date. This allows for remote users (and in-office users) plenty of warning before the mandatory enforcement.  It allows some remote users to be pro-active and initiate the upgrade when they are in the office that one day of the week, but most germane to our current topic, it allows for 7 days of attempted pre-downloads.

Pre-download is still a pre-release feature, so if you choose to use it, it will first need to be enabled. You can find it under Updates and Servicing -> Features -> "Pre-release - Task Sequence content pre-caching." Once you have the feature enabled, setting up pre-download is a simple process.  In the Upgrade Operating System Package properties, the Data Source tab will have radial buttons for x86 or x64 architectures. Pre-1706 you will need to put in the language code of the package, in 1706 simply select the language you wish from the drop-down menu.

 

If you have multiple upgrade packages in your task sequence you will still need to put in logic to specify which package you want to use.

 

When creating the task sequence deployment, to enable pre-download, select the "Pre-download content for this task sequence" box.

 

Task Sequence Design

Pre-download of the operating system upgrade package solves the largest outright failure scenario for a remotely connected client but we want to make sure that we are optimizing the task sequence itself to reduce other download failures, as well as reduce any delays that loss of connectivity may create.

To reduce any other download failures, we're minimizing the number of other packages we're sending during operating system upgrade. If application or driver updates need to be installed before operating system upgrade occurs, consider sending those separately before the upgrade. If you have your task sequence set to "Download content locally when needed by the running task sequence" keep in mind that after operating system upgrade completes VPN connectivity may no longer exist. In such cases you will want to ensure that any packages are downloaded locally before the upgrade step occurs.

This loss of VPN connectivity after upgrade is important to keep in mind in the actual order of your task sequence steps. The task sequence is going to want to send status updates for every step it completes, even if it is a step that it evaluates as skipped.

In our task sequence we have 32-bit and 64-bit packages of 5 different languages. If a client doesn't have MP connectivity it is still going to attempt to send status updates for each upgrade step it skips, adding a substantial amount of time into the process. Because of this, we put the upgrade steps in reverse order of popularity. The processor architecture/language combination with the fewest devices in our environment goes first, followed by the second fewest, all the way to the most common (in our case English 64-bit.) Organizing it this way allows the most people to have the least amount of delays while keeping a single task sequence.

 

Keeping those status delays in mind, we minimize the number of post-upgrade steps. We also make sure that the post-upgrade steps are not essential to a successful upgrade and a functioning device. Reporting of task sequence execution can be unreliable after MP connectivity is lost (an issue we are working the development team on,) and we don't want unreliable reporting of any essential action.

Reporting

Perhaps the most essential addition we've added within our task sequence is one that doesn't perform a purpose in the upgrade, but assists in reporting and troubleshooting. Right before the upgrade step occurs we run a script to detect if the client is connected via VPN. This allows us to quickly determine if the attempt was made over VPN without checking any log or contacting the user. We're able to categorize top errors based on connectivity type, track the overall success of upgrade over VPN, and see the percentage of upgrades that are being attempted over VPN.

The Remote Connectivity Check can be a simple PowerShell script to look for whatever remote connectivity solutions exist in an environment:

$ErrorActionPreference = "SilentlyContinue"
if((Get-NetConnectionProfile -Name [domain.com]).InterfaceAlias -in ("[VPN Name]","[Second VPN Name]")){exit 2}
else
{
if((Get-DAConnectionStatus).status -eq 'ConnectedLocally' -or (Get-DAConnectionStatus).status -eq 'ConnectedRemotely')
{exit 3}
else
{exit 0}
}

In our case we check for both our main VPN profiles, as well as the older Direct Access connectivity.

 

We then make sure to set that step to "Continue on Error"

 

We can now use the v_TaskExecutionStatus table in the database to gather data on the connectivity of all upgrade attempts. Combining that connection information with the other details of the task sequence execution we are able to track both the number of users upgrading remotely.  We can also track the errors that are occurring on these attempts to improve the process for the next set of upgrades through a PowerBI Dashboard.

 

8/31 - Added clarity regarding enabling the Pre-download feature as it is still in pre-release.

Disclaimer: This task sequence configuration may not be explicitly supported and this blog post is for informational purposes. This post is provided “AS IS” with no warranties and confers no rights.


	
Comments (13)

  1. Sascha Stumpler says:

    Hello Kacey, great article and thanks a lot for sharing! It took me a few minutes to realize that I could not see the data source options you used because it is still a SCCM pre-release feature. I think it would help others if you mentioned this.

    1. Thanks Sascha! The post has been updated to add a bit more clarity that this is indeed a pre-release feature still.

  2. will nimmo says:

    Kacey –
    Thanks for this blog. We are facing the same challenges you are addressing above. A few questions / comments:
    1. When you configure a task sequence to pre-download content, it downloads ALL content associated with the task sequence. In our case we support 17 hardware models. Downloading all of the drivers and model specific apps for all models uses upwards of 17GB. For many of our remote users, this presents a challenge – not having enough free disk space, or when they do their connectivity may make a 17GB download a very time consuming and performance inhibiting challenge. How are you addressing this? Our current strategy is to have a package that is targeted pre-OS upgrade based on each model to put the files in a specific location. We then set that location as the location for drivers during the pre-OS upgrade section of the task sequence.
    2. You mentioned specifying the OS architecture. The 1611 technical preview feature is not mentioned in the 1706 release notes, but your blog seems to indicate the feature is in production 1706? Does the capability allow you to dictate which drivers to download ahead of time, or is the functionality limited to os upgrade packages only..
    3. Please keep hammering the product team about the delay associated with status messages when a PC is offline. There should be 2 variables – 1 to allow it to stop trying wait for them to send, and another for whether they should cache up and send once connectivity is established. We have looked into the possibility of generating MIFs and sending those up once the TS is complete.
    3.

    1. Hi Kacey,
      As far as I have seen and read, the “Pre-download Content” feature only works for “available” task sequences. I understand that you would send this out a week or so in advance to get the content out there as it will only process the various WMI rules, etc so that only the content applicable to the client gets downloaded. But once you change your deployment to be required, it will download the entire task sequence (which would be a ton of data). I’m assuming you then change your required deployed to only “Download Content locally when needed by the running task sequence”? Do those two settings working together result in the client only pre-caching and downloading the content it needs?

  3. Nice, but like Nimmo says, whenever an approach is chosen to put drivers into individual packages for different models, it gets far too big to handle for a pre-download feature. It would just take forever, consume too much capacity on the volume.

    @Nimmo, I’ve seen our Ops team do the same approach in regards to drivers. IMHO – Stop using that approach. I’ve worked/am working to simplify driver management. Bringing it back to 1 package for all models. It saves a lot of capacity, and speed. It’s rules out different driver versions or drivers being imported more than once. Also, I use a different approach to install drivers: During Windows Setup, only apply required Device Driver Classes: Mass Storage, Net, Display (for WinSat), System (only handy ones). After Windows Setup completes, in the Post Install section I apply the remaining driver through my 1 driver package by executing DPinst. DPinst scans and installs drivers that are published to the OS. It is run a few times to allow child-node devices to be found once their parent has been installed. This installs like 95% of all drivers. Last seperate step is to install drivers that require additional software (often) to run properly. Such as WWan, Bluetooth or Intel Management Engine (heci/Mebx/vPro) for optimal functionality.

    Next step I want to use is to let ConfigMgr actively perform driver management itself based on PnP inventory and import updated driver from the first ConfigMgr client that reported it, tracing it back to find any superseeded drivers and remove those as well as to detect whether the PnP information still applies to all models actively being used.

    Kacey – I like your post. What I don’t like is to download more than I need. Hence, I’ve already built a solution to do OSD over wireless with deployment set to Access content directly from distribution point. The solution incorporates 802.1x authentication using the network access account for MSCHAPv2. It works well, still looking only for an option to remove time-outs for the Status Messages that the Task Sequence client wants to send before network connectivity can be restored by my custom made reboot step (w/ WiFi incorporated for both WinPE and Windows). I intend to use this solution too for VPN as it basically works the same for us.

    So, will surely take a look at the pre-download option, but I think it needs more work. In the mean time, I’m with Nimmo: pls help us by hammering the ConfigMgr product team down for options to delay status messages through variables.

    1. will nimmo says:

      @Herman – If you come up with a solution to circumvent the post OS upgrade MP connectivity issue until network access can be established, please share. We are working on it as well, though progress is admittedly slow.

  4. Richard Spaven says:

    How well would this work with a cloud management gateway and Azure DP?

  5. will nimmo says:

    @herman – On the issue of drivers, we deploy laptops from 5 vendors. Having a single set of drivers that works across the board isn’t an option for us. Further, our management and user base expects functionality that PNP drivers simply cannot handle. We have laptops where unless the vendor provided driver AND application GUI are not installed, it is impossible to disable Bluetooth independently from Wifi.
    If it works for you, I’m envious.

    1. Bob Clements says:

      Regarding drivers – still testing this scenario, but currently investigating the application model for pre-staging the drivers on endpoints. The thought process is as follows:

      1 application with multiple deployment types. Each deployment type has a model requirement. For execution, run a script that copies the corresponding drivers to a staging folder (i.e. C:\Temp\Drivers), which you specify in the upgrade operating system step. The application is deployed independent of the main TS.

    2. Will Nimmo – I am sure that you can control ie BT without vendor provided GUI, they generally use an method as described above to allow control of devices. Especially with UEFI, things are getting more easier/universal to control from Windows. So, it’s a matter of discovering how the vendor accomplishes it in their GUI, either through ACPI WMI provider, or specific driver which has it’s functions exposed through some libraries which are either called or part of the GUI. I generally utilize C# (also for unmanaged code) and some debug methods to – attempt – to ‘reverse engineer’ and make available for control what some vendors fail to implement in the first place.

      To add to the driver saga, we too have all major vendors within our company that need to be supported in OSD. What I did, is I created a universal framework that handles BIOS configurations and also BIOS updates.
      BIOS Configuration framework allows us to utilize either ACPI WMI Provider to configure BIOS settings, or using specific vendor supplied driver incl separate MOF to compile into WMI to allow all devices to be handled by the framework in as similar to each other as possible. I utilize an XML doc to group BIOS settings of each individual vendor(/model) in such a way that 1 TS step would configure the settings across all our vendor models for that particular group (ie group Security -> handles TPM settings).
      This took some time to accomplish, as well causes some intermittent maintenance to keep the framework up-to-date whenever new(er) models appear.
      Same applies to the multi-vendor/model BIOS update framework. I made it a self-evaluating framework that based on current version assesses whether to update to a newer version, incorporating too of course the prerequisites like pausing Bitlocker, power adapter check etc.

      With all this now shared with you, the BIOS configuration framework allows on multiple vendors to independently remove ie Wlan/BT radio control from the radio switch, thus allowing the published device to be installed properly during OSD.

      What works for you, works up to a certain point. I decided to split things up:
      – (Mini)-Setup installs only certain device classes, prevents accidental BSODs or problems on ie user-mode drivers
      – Post Install – install PnP drivers using gathered or exported drivers from OEM or part manufacturer (latter preferred)
      – Post Install – install specific software related to control certain devices

      This resulted in 1 TS, with general steps where the scripts inside the steps handle OEM specific items which – imho – is a flexible as it can get within OSD world.

  6. Ray says:

    For all who use driverpackages in the upgrade task sequence, I believe Kacey’s blog is only applicable for scenario’s when upgrading existing Windows 10 PCs (e.g. 1607 –> 1703). This scenario does not require any drivers, since the PCs already have Windows 10 drivers loaded.

  7. Thanks for the post!! Operating system deployment in System Center 2012 Configuration Manager has external dependencies and dependencies within the product.

Skip to main content