This post is a part of the nine-part “What’s New in Windows Server & System Center 2012 R2” series that is featured on Brad Anderson’s In the Cloud blog. Today’s blog post covers an overview of networking investments in the R2 release and how it applies to the larger topic of “Transform the Datacenter.” To read that post and see the other technologies discussed, read today’s post: “What’s New in 2012 R2: IaaS Innovations.”
As described in “What’s New in 2012 R2: IaaS Innovations”, Windows Server 2012 R2, System Center 2012 R2 and the Windows Azure Pack together deliver an Infrastructure as a Service (IaaS) solution for service providers. With the R2 release, networking, storage and compute come together end to end with a rich management plane for both tenants and service providers to enable a low cost, easy to operate private cloud solution.
Looking back, we spent a significant portion of time during the planning phase understanding the challenges faced by our enterprise customers, service providers and our own large datacenters (such as Azure, Bing). For networking, we got three main pieces of feedback:
- Reduce networking capex by maximizing utilization of infrastructure that is already deployed. No specialized equipment, no rip and replace. Instead, get more out of what is already available.
- Enable choice and flexibility to plug and play across networking equipment while being able to deploy workloads across any cloud.
- Ensure network automation across the service provider network and the tenant network. Azure makes thousands of changes in its network every day – without automation, this would be impossible.
These learnings helped crystallize our core customer vision for the R2 release – Transform existing datacenter networks into a pooled, automated resource that provide flexibility to move workloads across any cloud, while offering high performance and easy diagnosability.
This vision brought focus to the investment areas we targeted: Cloud-scale performance and diagnosability, an inbox comprehensive Software-Defined-Networking (SDN) solution and Core network infrastructure enhancements for the cloud.
The rest of this post will give an overview of the set of features we invested across these three large bucket areas.
Comprehensive Software Defined Networking (SDN) solution
Historically, workloads were tied to physical machines. With virtualization, we created a pool of compute resources that could be dynamically allocated to apps/VMs enabling flexibility for the IT admin. The next barrier was storage, because frequently storage was directly attached to the workload. This meant that if a workload were to be migrated, then the storage would have to migrate to the same machine as well. In addition, storage was not cost effective either. With SMB 3.0 and Storage Spaces, we now provide disaggregated storage that solves this problem. The last barrier is networking. As much as VMs and storage can be migrated, their corresponding networks don’t automatically move without lot of changes in the physical network infrastructure for the private cloud. For hybrid environments, this is even harder.
Software Defined Networking (SDN) is the ability of software to dynamically convert your existing network into a pooled resource that can be flexibly allocated to workloads independent of machine, rack, stamp or cloud. Based on our deployments in our large scale datacenters, we see five broad technology enablers for SDN – many of which we started down the path in Windows Server 2012 and System Center 2012 SP1.
Hyper-V Extensible switch as the policy edge
The first step towards gaining flexibility in your existing network is to move the policy edge from the physical switch in the network to the virtual switches on the host where possible. That is, instead of applying ACLs, QoS, isolation etc. on the physical switches – apply them on the virtual switches instead. By doing so, your existing network becomes consistently manageable and automated via software. A key aspect of our switch is that it is extensible, allowing capture extensions (such as InMon’s sFlow extension), filter extensions (such as 5Nine’s firewall extension) and forwarding extensions (such as NEC’s OpenFlow extension or Cisco’s Nexus 1000V extension) to co-exist with each other.
In Windows Server 2012 R2, we made the policy model richer by allowing firewall like capabilities in the switch. With Extended ACLs, you now get the ability to apply weighted, stateful rules that allow or deny traffic not just based on source/destination IP addresses, but port numbers as well. This allows setting ACLs not just for a VM, but workloads running in the VM as well. A future blog post will describe this capability in detail.
Hyper-V Network Virtualization (HNV) for constructing isolated tenant overlays
Once the policy edge is the virtual switch, you now can construct overlays on your existing network to model tenant networking needs. HNV in Windows Server 2012 and System Center 2012 Virtual Machine Manager SP1 provides the ability for tenants to bring their IP addresses to the cloud, with the virtual switches providing isolation amongst tenants.
With the R2 release, we now allow customers to bring in more of their network topology into the cloud, including their DHCP servers, and guest clusters. HNV now offers more advanced diagnostics allowing customer reported networking issues to be quickly isolated down to a problem in the service provider network or the tenant network. HNV works with NIC Teaming and includes new capabilities (such as task offloads) to drive up performance. To learn more, refer to What’s new in Hyper-V Network Virtualization in R2.
One of the key facets we introduced in R2 is to allow multiple network virtualization protocols to co-exist on the same switch. This capability, called hybrid forwarding, enables HNV traffic to get forwarded natively by the switch while enabling forwarding extensions (such as the Cisco Nexus 1000V) to forward all other traffic. To learn more, refer to Hyper-V Extensible Switch Enhancements in Windows Server 2012 R2.
Windows Server Gateways to enable hybrid connectivity
Hyper-V Network virtualization lets you create tenant network overlays, and the Hyper-V Extensible switch gives the mechanism to enforce the isolation requirements. Frequently, tenants need to communicate outside the network overlay they are a part of. For this, you need a gateway. In Windows Server 2012, we offer a diverse set of partner gateways that make this possible.
With the R2 release, we supplement our partner offerings with an inbox one across Site to Site, Network Address Translation and Forwarding. The S2S gateway allows tenants to communicate back to their on-premise datacenters (for instance, the web front-end in the public cloud and the SQL back-end in the private cloud), the NAT gateway allows tenants to connect to the Internet (or for any scenario requiring address translation), and the forwarding gateway allows tenants within a private cloud to connect to a shared physical resource (such as storage). Stay tuned for a future blog post on this topic.
Together with HNV and the virtual switch, this trifecta of gateways allows the ultimate flexibility in enabling VMs and workloads to be placed and migrated across machines and clouds. Your existing network is now a pooled resource that can be dynamically managed based on your needs.
Physical switch management using standards-based schemas
Thus far, we’ve been talking about virtual switches. Virtual switches provide the foundation to gain flexibility on top of an existing physical network. With Windows Server 2012 R2, you also get the ability to use PowerShell to deploy your physical network as well. This helps automate the setup of your network (such as configuring switch ports, setting VLANs etc.) that works consistently across vendors. We worked closely with major industry partners to introduce a new standards based switch management schema. We will be logo’ing switches that implement this industry standard.
In addition, we are lighting up specific customer scenarios through SCVMM. One of the major pain points we hear from customers is matching the VLAN configuration across the physical switch and the virtual switch. With the R2 release, SCVMM will monitor the VLAN configuration across both the virtual and physical switches, notify the admin if something in the VLAN configuration is out of sync, and allow the administrator to easily fix the misconfiguration.
To learn more, refer to DAL in action: Managing Network Switches using PowerShell and CIM.
SCVMM and Windows Azure Pack for automation and self-service
Perhaps the most critical component to enable all of the above is automation for the service provider and self-service for the tenant. With System Center Virtual Machine Manager 2012 R2, we provide end to end automation for transforming your datacenter. From setting up policies and extensions on the virtual switches, to managing tenant network overlays to streamlined deployment of all the kinds of gateways – SCVMM is a core asset to enabling the overall SDN solution.
At the same time, scaling up the service provider requires self service capabilities for the tenant. With Windows Azure Pack, we make the Azure tenant self-service portal experience available for private clouds as well.
For more details on Microsoft’s SDN solution, refer to Transforming your Datacenter with Software-Defined Networking (SDN): Part I.
Cloud scale performance and diagnosability
The key challenge we set based on planning conversations was whether we could maximize utilization of the existing infrastructure that customers already have in deployment, so as to enable a low cost IaaS solution for networking. And whether we could make diagnosability easier. Below is a description of some of the key features for this broad bucket.
Traditionally, large networking-intensive, physical workloads use Receive Side Scaling (RSS) to scale up by distributing the processing of networking traffic across multiple physical cores. In a virtualized environment, there is no equivalent. Virtual RSS (vRSS) in Windows Server 2012 R2 enables both the host and the VM to use multiple cores, resulting in bandwidth scaling characteristics that are somewhat similar to what RSS enabled for large physical workloads. In our test environments with a simulated workload, we found vRSS enabling a VM to hit near line rate on a 10Gbps NIC compared to ~5.5Gbps without this capability.
To learn more about vRSS, refer to Drive up networking performance for your most demanding workloads with Virtual RSS.
Dynamic NIC Teaming
We introduced inbox NIC Teaming in Windows Server 2012, and it is amongst our most popular features. We learned that our load distribution algorithms were not working optimally for scenarios where there were a few large flows. Since NIC Teaming affinitized a flow to a NIC, a few large flows could overwhelm the NIC even though there may be spare capacity available in the team. In the R2 release, we break up a flow into smaller flow-lets based on natural gaps that occur in a TCP stream, and load balance flow-lets to NICs. The result is optimal utilization of the NIC team across any kind and any number of flows.
Stay tuned for a future blog post on this topic.
Remote Packet Capture
A common challenge surrounding diagnosability is the need to capture a packet trace from a machine. This happens commonly when networking is identified as the issue for a critical issue, and a given machine’s health needs to get diagnosed. In such situations, getting an immediate packet capture and ETW traces is critical. Traditionally, this involves having to log on to the machine, starting a trace, and then separately collecting both the packet capture and the ETW events. With the R2 release, the next version of Netmon (called Message Analyzer), provides the ability to remotely capture a packet stream (for administrators only) with a correlated set of ETW traces.
To learn more about remote packet capture and other diagnostics capabilities, refer to New Networking Diagnostics with PowerShell in Windows Server R2.
Combined with the set of investments in Windows Server 2012, Windows Server 2012 R2 provides the most high-performance and diagnosable release of Windows Server ever, thereby minimizing networking costs in the cloud.
Cloud optimized core infrastructure enhancements
Given our cloud-first philosophy, it is critical that the core infrastructure servicing such environments be cloud optimized. We started this journey in Windows Server 2012 with the introduction of IPAM which provided centralized tracking and administration of IP addresses (physical and virtual), DHCP and DNS, along with rich PowerShell automation.
This momentum is continued in the R2 release across all services – IPAM, DNS and DHCP.
Unified IP address administration of Physical and Virtual networks
In Windows Server 2012 R2, IPAM enables network administrators to fully streamline IP address space administration of both physical (fabric) and virtual (tenant) networks. The integration between IPAM and SCVMM in the R2 release provides end-to-end IP address space automation for Microsoft-powered cloud networks. A single instance of IPAM detects and prevents IP address space conflicts, duplicates, and overlaps across multiple instances of SCVMM deployed in a large datacenter or across datacenters.
Fine-grained administrative control
As customer clouds grow in size and deployment, there is a need for IPAM to enable appropriate privileges for the different scopes. With the R2 release, IPAM can now enable granular role-based access control (RBAC) and delegated administration. System and network admins can use IPAM to define the roles (collection of admin operations and whether it can be delegated); access scopes (administrative domains in IPAM, determines the entities that the user has access to); and access policies (combines a role with an access scope to assign permission to a user or group). Such capabilities gives admins the needed flexibility to confidently administer their large cloud environments.
Highly scalable and customizable
As noted earlier, automation is key to attain cloud scale, and IPAM is no exception on that front. IPAM provides a comprehensive set of PowerShell cmdlets to facilitate the operations described above, and to enable integration with various other systems in the network. In addition, MS SQL Server is now supported as an optional backend for IPAM database to enable large scale.
As you can see, we targeted three broad investments areas in the R2 release – Cloud-scale performance and diagnosability, an inbox comprehensive Software-Defined-Networking (SDN) solution and Core network infrastructure enhancements for the cloud. We hope that these investments help transform your existing network into a pooled, automated resource with flexibility to move workloads across any cloud, while offering high performance and easy diagnosability.
We look forward to hearing from you on your experiences.
Networking Talks at TechEd 2013
- What’s New in Windows Server 2012 R2 Networking
- Deep Dive on Hyper-V Network Virtualization in Windows Server 2012 R2
- Everything You Need to Know about the Software Defined Networking Solution from Microsoft
- How to Design and Configure Networking in Microsoft System Center – Virtual Machine Manager and Hyper-V (Part 1 of 2)
- How to Design and Configure Networking in Microsoft System Center – Virtual Machine Manager and Hyper-V (Part 2 of 2)
- Networking for Cloud Services in Windows Server 2012 R2
To see all of the posts in this series, check out the What’s New in Windows Server & System Center 2012 R2 archive.
Ravi Rao, Lead Program Manager, Windows Core Networking team