MSDN and TechNet are full virtualized right now with Hyper-V, and we had a dramatic cost savings when we went and did the virtualization of these very high volume Web sites within Microsoft. Of course, we'll be deploying it throughout our datacenters both for our internal IT as well as for our external facing properties in the coming months.
So how did we do it? The MSCOM Operations team has just released a White Paper covering the details of the rollout. From the "Lessons Learned" section:
- Based on our results with Hyper-V, and a review of current hardware utilization across our environment, we expect to realize significant benefits by consolidating diverse applications on a virtualized platform. Mixing high and low scale systems on the same physical server resources should enable us achieve improved overall hardware utilization and a reduced physical footprint.
- Although we reduced our hardware supporting MSDN and TechNet by migrating from older physical servers to a smaller deployment of new, more powerful physical servers, there is some minimal overhead associated with virtualization. From our perspective, Hyper-V has clearly delivered enough performance, stability, and scale to drive widespread adoption in our production environment. The flexibility and management gains expected from coupling System Center Virtual Machine Manager (SCVMM) v2 with Hyper-V should justify the measured overhead.
- If the three percent or less in additional overhead from oversubscribing resources is consistent with additional application workloads tested, oversubscription with Hyper-V should provide MSCOM Ops significant flexibility and reasonable performance for application consolidation.
- The results we achieved are based on the application characteristics of MSDN and TechNet on Hyper-V RC0. We are working to build a model based on this data that we hope will allow us to predict physical and VM requirements based on common performance characteristics of a Web site. The model will likely include the current requests per second per 1 percent CPU metric with additional memory and I/O performance characteristics as we virtualize other Web applications and gather more test and production data.
By the way... if you want to see some crazy diagrams of the overall network infrastructure that powers microsoft.com, check out this site. Some stats, if you are into that kind of thing: