What not to Learn… Revisited for 2013!
In October, 2011 I posted an article called vPTA: What NOT to take away from my 1-day virtualization training! It was only partly tongue-in-cheek on the environment that I have been using for several years to demonstrate server virtualization from a pair of laptops. A few months later Damir Bersinic took that list and made some modifications, and published it on this blog as Things NOT To Take Away from the IT Virtualization Boot Camp. Because we spend so much time in our IT Camps demonstrating similar environments, I decided it was a good time to rewrite that article.
Normally when I revisit an article I would simply republish it. There are two reasons that I decided to rewrite this one from scratch:
- The improvements in Windows Server 2012, and
- My more official position at Microsoft Canada
Since writing that original article I have tried to revise my writing style so as to not offend some people… I am trying to be a resource to all IT Professionals in Canada, and to do that I want to eliminate a lot of the sarcasm that my older posts were replete with. At the same time there are points that I want to reinforce because of the severity of the consequences.
Creating a lab environment equivalent to Microsoft Canada’s IT Camps, with simple modifications:
1. In our IT Camps we provide the attendees with hardware to use for their labs. Depending on the camp attendees will work in teams on either one or two laptops. While this is fine for the Windows 8 camps, please remember that in your environment – even in a lab where possible – you should be using actual server hardware. With virtualization it is so simple to create a segregated lab environment on the same server as your production environment, using virtual switches and VLAN tagging. In environments where System Center 2012 has already been deployed it is easy enough to provision private clouds for your test/dev environments, but even without that it is a good idea. The laptops that we use for the IT Camps are great for the one- or two-day camps, but for longer than that you are going to risk running into a plethora of crashes that are easy enough to anticipate.
2. You should always have multiple domain controllers in any environment, production or otherwise. Depending on who you speak to many professionals will tell you that at least one domain controller in your domain should be on a physical box (as opposed to a virtual machine). I am still not convinced that this does not fall into the category of ‘Legacy Thinking’ but there is certainly an argument to be made for this. Whether you are going to do this in physical or virtual, you should never rely on a single domain controller. Likewise your domain controllers should be dedicated as such, and should not also be file or application servers.
3. I strongly recommend shared storage for your virtualization hosts be implemented on Storage Area Networks (SANs) . SAN devices are a great method of sharing data between clustered nodes in a failover cluster. In Windows Server 2012 we have included the iSCSI Software Target that was previously an optional download (The Microsoft iSCSI Software Target is now free). While this is still not a good replacement of physical SANs, it is a fully supported solution for Windows Failover Cluster Services, including for Hyper-V virtual machine environments. It is even now recognized as an option for System Center 2012 private clouds. As well the Storage Pools feature in the new Server is a compelling feature to consider. However there are some caveats to consider:
A. Both iSCSI software targets and Storage Pools rely on virtual storage (VHDX files) for their LUNs and Pools. While VHDX files are very stable, putting one VHDX file into another VHDX file is a bad idea… at least for long-term testing and especially for production environments. If you are going to use a software target or Storage Pool (which are both fully supported by Microsoft for production environments) it is strongly recommended that you put them onto physical hardware.
B. While Storage Pools are supported on any available drive architecture (including USB, SATA, etc…) the only architecture that will be supported for clustered environments are iSCSI and SAS (Serial Attached SCSI). Do not try to build a production (or long-term test environment) cluster on inexpensive USB or SATA drives.
C. In our labs we use a lot of thin-provisioned (dynamically expanding, storage-on-demand) disks. While these are fully supported, it is not necessarily a best practice. Especially on drives where you may be storing multiple VHDX files you are simply asking for fragmentation issues.
4. If you are building a lab environment on a single host, you may run into troubles when trying to join your host to the domain. I am not saying that it will not work – as long as you have properly configured your virtual network it likely will – but there are a couple of things to remember. Make sure that your virtual domain controller is configured to Always Start rather than Always start if it was running when the service stopped. As well it is a good idea to configure a static IP address for the host, just in case your virtual DHCP server fails to start properly, or in a timely fashion.
5. Servers are meant to run. Shutting down your servers on a daily basis has not been a recommended practice for many years, and the way we do things – at the end of the camp we re-image our machines, pack them into a giant case and ship them to the next site – is a really bad idea. If you are able I strongly recommend leaving your lab servers running at all times.
6. While it is great to be able to demo server technologies, when at all possible you should leave your servers connected (and turned on) in one place. If you are able to bring your clients to you for demos that is ideal, but it is so easy these days to access servers remotely on even the most basic of Internet connections. If your company does not have a static IP address I would recommend using a dynamic DNS service (such as dyndns.com) with proper port-forwarding configured in your gateway router to access then remotely.
7. I am asked all the time how many network adapters you need for a proper server environment. I always answer ‘It depends.’ There are many factors to consider when building your hosts, and in a demo environment there are concessions you can make. However unless you have absolutely no choice it should be more than one. For a proper cluster configuration (excluding multi-pathing and redundancy) you should have a production network, a storage network, and a heartbeat network… and that is three just for the bare minimum. Some of these can share networks and NICs by configuring VLANs, but again, preferably only in lab environments. Before building your systems consider what you are willing to compromise on, and what is absolutely required. Then build your architectural plan and determine what hardware is required before making your purchase.
7a. While on the subject of networks, in our demo environment the two laptop-servers are connected to each other by a single RJ-45 cable. BUY SWITCHES… and the ones that are good enough for you to use at home are usually not good enough for your production environment!
8. When it is at all possible your storage network should be physically segregated from your production network. When physical segregation is not possible then at least separating the streams by using vLANs is strongly recommended. The first offers security as well as bandwidth management, the second only security.
9. Your laptop and desktop hardware are not good-enough substitutes for server-grade hardware. I know we mentioned this before, but I still feel it is important enough to state again.
10. In Windows Server 2008 R2 we were very adamant that snapshots, while handy in labs and testing, were a bad idea for your production environment. With the improvements to Hyper-V in Windows Server 2012 we can be a little less adamant, but remember that you cannot take a snapshot and forget about it. When you delete or apply a snapshot it will now merge the VHDX and AVHDX files live… but snapshots can still outgrow your volume so make sure that when you are finished with a snapshot you clean up after yourself.
11. Breaking any of these rules in a production environment is not just a bad idea, it would likely result in an RGE (Resume Generating Event). In other words, some of these can be serious enough for you to lose your job, lose customers, and possibly even get you sued. Follow the best practices though and you should be fine!