Software Defined Networking, Enabled in Windows Server 2012 and System Center 2012 SP1, Virtual Machine Manager

Unlocking Network Flexibility, Efficiency, and Multi-tenancy for the Cloud 

We are very excited about the promise of Software Defined Networking (SDN) for enabling automation, flexibility, and reliability in the multi-tenant cloud.  Traditionally the control plane of networking has been proprietary, resulting in datacenter environments that are unable to respond effectively to the dynamically changing needs of today’s cloud workloads.  By enabling network control via software, we give customers the ability to configure and reconfigure their networks to match the changing requirements of their workloads, without compromising multi-tenant isolation and performance that would be expected from traditional networking.

Windows Server 2012 and System Center 2012 SP1, Virtual Machine Manager (VMM) enable everyone to take advantage of the power of SDN in your datacenters.  Our integrated solution provides unparalleled automation, flexibility, and control.  The solution supports scalability for even the most mission-critical deployments.  At the same time, we provide a standards-based and open platform that is supported by a rich partner ecosystem.  Best of all, everything you need to deploy SDN is built right into these products, so you do not need to acquire separate management tools or product licenses.

Of course, these attributes of our SDN solution did not come about by accident.  Windows Server 2012 builds on our years of experience running massive datacenters for properties such as Hotmail, Bing, and Windows Azure.  This foundation of experience is why we can confidently say that Windows Server 2012 is the first operating system specifically built for the Cloud – for enabling the public, private, and hybrid cloud.

In this post, we introduce Software-Defined Networking and talk about its origins within our own datacenters.  We then discuss how Windows Server 2012 and VMM deliver an end-to-end SDN solution and how partners are extending the solution.  We then discuss our own experience using SDN and how you can get started deploying this exciting technology today.

What Is Software-Defined Networking (SDN)
Traditionally, networks were defined by their physical topology, how the servers, switches, and routers were cabled together.  That meant that once you built out your network, changes were costly and complex.  Certainly, this type of networking is simply not compatible with the notion of a lights-out datacenter or a cloud environment that needs flexibility to support varying workload demands.

With Software Defined Networking (SDN), software can dynamically configure the network, allowing it to adapt to changing needs.  An SDN solution can accomplish several things:

• Create virtual networks that run on top of the physical network.  In a multi-tenant cloud, a virtual network might represent a tenant’s network topology, complete with the tenant’s own IP addresses, subnets, and even routing topology.  Through SDN, virtual networks can be created dynamically, and they can support VM mobility throughout the datacenter while preserving the logical network abstraction.
• Control traffic flow within the datacenter.  Some classes of traffic may need forwarding to a particular appliance (or VM) for security analysis or monitoring.  You may need to create bandwidth guarantees or enforce bandwidth caps on particular workloads.   Through SDN, you can create these policies and dynamically change them according to the needs of your workloads.
• Create integrated policies that span the physical and virtual networks.  Through SDN, you can ensure that your physical network and endpoints handle traffic similarly.  For example, you may want to deploy common security profiles, or you may want to share monitoring and metering infrastructure across both physical and virtual switches.

In summary, SDN is about being able to configure end hosts and physical network elements, dynamically adjust policies for how traffic flows through the network, and create virtual network abstractions that support real-time VM instantiation and migration throughout the datacenter.  This definition of SDN is, in fact, broader, than the definition currently used by many industry players who only focus on configuration of physical network elements.  Our broader SDN definition includes programmability of end hosts, enabling end-to-end software control in the datacenter.  Our definition also supports real-time changes in response to VM placement and migration.  As we will see below, the integration of VM management and network control is important to facilitate automation and reliability in large-scale datacenters.

Origins of Software Defined Networking
As mentioned above, we at Microsoft have years of experience running massive datacenters for properties such as Bing, Hotmail, and Windows Azure.  This experience taught us several important principles about datacenter network design:

• Automation is critical:  We have found that the vast majority of network outages arise because of human error.  Networks need to be configured and managed in an autonomous fashion.
• Multi-tenancy demands network flexibility:  In environments such as Windows Azure, customers expect to have easy ways to on ramp their workloads.  They don’t want to change IP addresses or other network settings in order to move to the cloud.  The cloud needs to be able to give each tenant the illusion of a dedicated network, even though it is shared by multiple tenants.  Interestingly, we have found the need for multi-tenancy even in single-=use datacenters.  For example, we often need to run a production SharePoint environment as well as a test SharePoint deployment simultaneously within the same datacenter.  As much as possible, our test deployment needs to mirror the production deployment, but it is critical for the test deployment to use its own Active Directory and DNS infrastructure.  Of course, we don’t want to deploy physically separate servers for the production and test environments—that would be unreasonably expensive!
• Centralized control drives simplicity and reliability:  In our experience, virtual machine placement needs to be driven from a central management entity that understands workload needs, hardware capacity, and virtual networks.  This manager drives policies to the end hosts and, therefore, is also best positioned to coordinate the network changes required to support that VM placement.  This approach reduces the possibility of policy inconsistency in the network, reduces delays associated with propagating SDN policies, and simplifies configuration and management.

In fact, based on this datacenter experience, our colleagues in Microsoft Research published seminal work defining new ways to create virtual and physical networks.  This effort heavily influenced our approach to SDN in Windows Azure and Windows Server and in fact, was the foundation for much of the SDN work being done across the industry.

An End-to-End Solution in Windows Server 2012 and System Center 2012 SP1, Virtual Machine Manager
Windows Server 2012 and VMM provide an end-to-end SDN solution for public, private, and hybrid clouds.  By building all the pieces as part of a solution—the hypervisor, the SDN control surface on the end host, and the management software—we ensure a set of seamless experiences for datacenter administrators.  All of the solution components work together to provide the most scalable and flexible platform for the cloud.

Our SDN approach consists of several different capabilities.

Hyper-V Network Virtualization delivers network flexibility for the cloud by providing the ability to create multi-tenant virtual networks on a shared physical network.  Each tenant gets a complete virtual network, including multiple virtual subnets and virtual routing.  (Some network virtualization solutions out there assume the tenant only has a single subnet!)  On each host, Hyper-V uses dynamically updatable SDN policies to associate a tenant network and properly direct traffic to the destination. The SDN policy also determines which VM’s these tenant VM’s are allowed to communicate with, providing the requisite isolation.  As a result, Hyper-V Network Virtualization allows tenant workloads to be placed anywhere in the physical datacenter.  Tenant networks even can use private IP addresses (which might overlap with addresses used by other tenants), allowing tenants to rapidly migrate their existing workloads to the cloud by bringing their own IP addresses.  In fact, Windows Server 2012 supports interoperable cross-premise connectivity, so you can seamlessly link your subnets in the public cloud back to your local network.

VMM plays a key role in automating configuration of SDN policies for Hyper-V Network Virtualization.  In VMM, you define and create tenant virtual networks as needed.  Note that because these networks are defined entirely in software, no reconfiguration of the physical network is needed.  VMM takes care of placing VM workloads and applying the necessary SDN policies to the hosts to create those virtual networks.  By applying VM placement decision and the SDN policy updates together, VMM provides a high degree of automation and centralized control, in keeping with our datacenter experience.  In addition, this integrated control plane speeds up policy distribution, reducing downtime and enabling more flexible VM placement and optimization.

Our SDN solution is further enabled through rich traffic control policies on the Hyper-V virtual switch.  On a per-VM basis, you can configure security policies that limit the types of traffic (and destinations).  You can reserve bandwidth to particular VMs, ensuring that mission-critical services can always access necessary network capacity.  You can even apply bandwidth caps, allowing you to avoid traffic starvation or enforce a variety of charging models.  What’s more, these network control policies are dynamic, so they can be adjusted in real-time.

VMM allows customers to unify the individual virtual switches on each Hyper-V host in the datacenter into a distributed logical switch that is dynamically programmed with SDN traffic control policies.  For example, you can define a profile for a set of VMs.  That profile might include the security and bandwidth controls that should be applied.  As it brings VMs up, VMM automatically programs the host virtual switch with the appropriate profile.  The profile moves from host to host as the VM is migrated.  The administrator is essentially defining a single logical datacenter switch, with VMM automating deployment of per-host and per-VM policies, ensuring consistency of SDN policies, and (as we have seen before) providing central control.

With Windows Server 2012, we are excited to introduce the Hyper-V Extensible Switch.  The switch provides a platform through which our partners can extend SDN policies within the switch.  In fact, one of the most common use cases for this extensibility is to integrate the virtual switch with the rest of the physical network infrastructure. A unique aspect of this extensibility is that multiple partners can extend the switch at the same time.  For example, InMon has built an extension that allows traffic monitoring to be done on the Hyper-V switch in the same way it is done on physical switches.  Another partner, NEC, has integrated the Hyper-V switch with their OpenFlow controller.  The NEC OpenFlow controller defines exactly how traffic from the source VM to the destination VM should be routed through the network; NEC solution is completely compatible with Hyper-V Network Virtualization, which defines the origin and destination VMs within the virtual network.  The NEC solution allows for easy configuration of virtual appliances such as load balancers, intrusion detection systems, and network monitoring solutions.

VMM handles the lifecycle and configuration of Hyper-V switch extensions.  In fact, these switch extensions essentially become part of the SDN language that VMM speaks to Hyper-V.  As VMs migrate across the datacenter, VMM and Hyper-V ensure that state information associated with the switch extension is also migrated to the new host.  VMM ensures that the destination host has the switch extensions required by the guest VM or tenant network.  This level of seamless extensibility is unique to the Hyper-V / System Center SDN solution.

Of course, our end-to-end solution recognizes that Hyper-V hosts are not the only components of a datacenter network.  VMM is able to dynamically provision key network elements such as load balancers, site-to-site VPNs, and Hyper-V Network Virtualization gateways.  At the end of the day, SDN is about end-to-end automation, flexibility, and control throughout the data center.

Built for Partners, Built with Partners
Our SDN solution is, from the ground up, designed with partners in mind.  It is open and flexible, allowing partners to offer value added capabilities.  Moreover, the SDN solution supports a close relationship between software and hardware.  Even though it is software-driven, SDN needs to take advantage of capabilities provided by network cards, switches, and routers.

We disagree with many in the industry who say that SDN should “commoditize” the network infrastructure.   In our view, SDN should provide the automation, flexibility, and control to allow you easily to take advantage of the capabilities of the infrastructure.  In fact, SDN should create new innovation opportunities for network hardware.  Customers can only benefit from new innovations across their datacenter.

Within our SDN solution, we have already touched on how partners can build extensions for the Hyper-V Extensible Switch.  In fact, multiple extensions can co-exist in the hypervisor switch, and they can all work in tandem with our other SDN elements, Hyper-V Network Virtualization and rich traffic control policies.  We support our partners with certification tests, interoperability plug fests, development tools, and close engineering support.

This spirit of partner cooperation is evident throughout our SDN solution.  Hyper-V Network Virtualization builds on IETF standard protocols (Generic Routing Encapsulation, or GRE), and together with partners from a variety of network silicon and switch manufacturers, we have published guidance on how GRE enables network virtualization.  This standards-based approach means that network cards and network switches can support and accelerate tenant logical network traffic.  In fact, our design includes tenant ID information in the packet, enabling network equipment to do tenant-specific accounting, policy control, or advanced processing.

Our open approach has enabled several partners to announce solutions that work with Hyper-V Network Virtualization.  For example, nAppliance and IVO Networks have both announced plans for network appliances that provide Hyper-V Network Virtualization gateways.  Stay tuned for more partner announcements shortly!

In addition, VMM supports pluggable interfaces, allowing it to configure arbitrary load balancers, site-to-site VPNs, and network virtualization gateways.  VMM can therefore interoperate with other SDN solutions or network control servers.

Production Tested, Production Used
As we have discussed, our SDN solution grew out of our experience running large datacenters and cloud services.  Needless to say, we have been able to validate our solution in these environments.  Within Microsoft, we are running a large, multi-tenant private cloud used for several mission-critical workloads.  Hyper-V Network Virtualization is in active use within that cloud today, orchestrating communication for tens of thousands of VMs running on over 4000 physical hosts.  As you might expect, our SDN algorithms and protocols are in active use within the Windows Azure datacenter, supporting our Infrastructure as a Service (IaaS) offering that was announced last month.

At the same time, throughout the development of Windows Server 2012 and VMM, we have been working closely with enterprise and hoster customers to validate and deploy our SDN solution.  Many of these customers are already running production services using these cloud components.

Ready for You – and Built Right In!
Software Defined Networking (SDN) holds the promise to revolutionize cloud networks by bringing a new level of automation, flexibility, and control to the network environment.  As we have seen, our SDN approach takes an integrated, end-to-end view which brings simplicity, performance, and reliability to the solution.  At the same time, we have built our solution using open standards and pluggable interfaces.  Just as important, we have been developing a rich partner ecosystem, so you can integrate best-of-breed capabilities across the industry with Windows Server 2012 and System Center 2012 SP1, Virtual Machine Manager.

Most important, all of the tools you need to deploy Software Defined Networking are built right in to Windows Server 2012 and System Center 2012 SP1, Virtual Machine Manager.  You do not need to buy separate management tools or acquire separate product editions.  Windows Server 2012 and System Center 2012 SP1, Virtual Machine Manager deliver the best value for public, private, and hybrid clouds.

With the Release to Manufacturing (RTM) and impending launch of Windows Server 2012, our SDN solution is ready for you to deploy.  We are looking forward to hearing about your experiences building public, private, and hybrid clouds on our SDN platform.

Appendix: Some Resources for Getting Started with SDN
Windows Server® 2012 Hyper-V Network Virtualization Survival Guide helps you get started deploying SDN and network virtualization in your datacenter.

The Hyper-V Network Virtualization Overview gives you a technical overview of the feature and how it works.

The Internet RFC titled NVGRE: Network Virtualization using Generic Routing Encapsulation gives you the details behind the packet encapsulation format Hyper-V network virtualization uses for virtualizing network traffic.

The Hyper-V Extensible Switch article gives you an architectural overview about Hyper-V switch extensions.  You can also learn about Writing Hyper-V Switch Extensions.

The blog article about Cloud Datacenter Network Architecture describes how you can put everything together in order to build a cloud that uses SDN.

Sandeep K. Singhal, GM, Windows Networking

Vijay Tewari, Principal Group Program Manager, System Center Virtual Machine Manager