Now that we have all of our equipment, we need to figure out what our deployment footprint looks like. It’s worth noting that while we’re discussing this like it was a linear process, this is actually one of the many interdependencies we had to manage during the planning process that shaped our purchasing.
Racks of Unusual Size
The first bit of uniqueness we have to deal with is our datacenter. The location where this cloud will be deployed is one of our research and development datacenters. In other words, Microsoft’s public services like Office365, Outlook.com, and Xbox Live! do not live there. That’s not to say it’s not a serious datacenter, however. There are literally tens of thousands of servers running there, all serving the needs of our development and test teams.
One of the unique features of the datacenter is special racks throughout. For density purposes, we had custom 52U racks built for the entire facility. Needless to say, that makes for some interesting architecture choices when it comes to racking and wiring, if you’re going to make maximum use of the space in each rack.
Maximizing density is important despite the datacenter’s huge size, because space is still at a premium. Luckily for us, our existing compute nodes were deployed as one large chunk several years ago, and consumed 7+ racks of contiguous space. This fortunate twist meant we only needed one additional rack of space for our new gear, along with a well considered re-racking plan.
The final rack deployment plan took shape around a rack pair design that would be replicated 4 times. You can see a representation of our racks to the right, just click to enlarge. One rack will hold only compute nodes and their associated network equipment, while the other rack will hold a smaller quantity of compute nodes, as well one of our Scale-Out File Servers that will provide storage for the rack pair. Besides helping localize traffic, it gives us flexibility to take a pair of racks and remove them from the fabric to test new ideas, new versions of Windows and System Center, and leave the rest of the fabric unaffected.
In the next post of this series, we’ll dive into the details of the rack layout, and then the logical wiring of the systems and racks.