Failover Clustering for Windows Server 2008 Hyper-V with File Server Storage

[LEFT][CODE]http://blogs.technet.com/josebda/archive/2008/07/16/failover-clustering-for-hyper-v-with-file-server-storage.aspx[/CODE]

[B]Overview[/B]
[INDENT] In a previous blog post, I described 5 different ways to implement Windows Server Failover Clustering with Windows Server 2008 Hyper-V. Those options included: Parent-based Failover Clustering with two physical servers, Child-based Failover Clustering with two physical servers, Mixed Physical/Virtual Failover Clustering, Failover Clustering with two child partitions on one physical server and Standalone demo laptop with Virtual iSCSI SAN. If you missed that one, check [url=http://blogs.technet.com/josebda/archive/2008/06/17/windows-server-2008-hyper-v-failover-clustering-options.aspx]Jose Barreto's Blog : Windows Server 2008 Hyper-V Failover Clustering Options[/url].
However, I failed to mention in that post the option to use CIFS/SMB file server share as your option for Failover Clustering storage. This scenario is so unique (with differences in flexibility, cost and performance), that I would argue it constitutes a sixth method. Here’s how you can do it.[/INDENT][B]Before and After Diagrams[/B]
[INDENT] As I did with the previous blog post, let me describe the scenario using two diagrams. First, here is a diagram describing the scenario before a failure:
[IMG]http://blogs.technet.com/photos/josebda/images/3089404/original.aspx[/IMG]
Now, here’s a diagram describing the scenario after a failure in SPTNODE1:
[IMG]http://blogs.technet.com/photos/josebda/images/3089401/original.aspx[/IMG]
As you can see, we use a file server (called SPTSERVER1) for storing the Hyper-V files. The idea is to store the configuration files, the VHD itself and the VHD snapshots in the \\SPTSERVER1\VMSHARE\VM1 folder. As we do when using a SAN for shared storage, the surviving node will take over and start the VM in case of a failure. We can also use the very same scenario for Quick Migration, making the VM move orderly from one node to another by saving the state to the file share and instructing to other node to take over and restore the VM.[/INDENT][B]Pre-requisites[/B]
[INDENT] Before you move forward, you want to make sure you have at least two physical computers running Hyper-V. In our scenario, STPNODE1 and STPNODE2 are running Windows Server 2008 Enterprise (Full or Core installs work fine).
Add the Hyper-V role to STPNODE1 and STPNODE2.
Add the Failover Clustering feature to STPNODE1 and STPNODE2.
You will need to use a general purpose server to act as a file server or a NAS box compatible with CIFS/SMB. You probably want to run Windows Server 2008 for improved performance (new TCP/IP stack and SMBv2 protocol). In our scenario, STPSERVER1 is the file server running Windows Server 2008 Enterprise Edition (Full or Core installs work fine).
Grant the required permissions for \\SPTSERVER1\VMSHARE\ to the computer accounts for STPNODE1 and STPNODE2, as described at [url=http://blogs.technet.com/josebda/archive/2008/06/24/storing-windows-server-2008-hyper-v-files-on-an-cifs-smb-file-share.aspx]Jose Barreto's Blog : Storing Windows Server 2008 Hyper-V files on a CIFS/SMB file share[/url].
You might also want to have a management client which could be your desktop (running Windows Vista SP1) or another server (running a Full install of Windows Server 2008). In our scenario, SPTCLIENT1 is the management client.
Install the Windows Server RTM patch described at [url=http://blogs.technet.com/josebda/archive/2008/06/26/windows-server-2008-hyper-v-released-today-includes-a-list-of-main-hyper-v-related-links.aspx]Jose Barreto's Blog : Windows Server 2008 Hyper-V released today! (includes a list of main Hyper-V related links)[/url] in both nodes and also in the management client.
You will need to have a domain infrastructure (Windows Server Failover Clustering requires a domain). The domain controller is not shown in the diagrams.[/INDENT][B]Steps[/B]
[INDENT] You start the process by creating a cluster with the two Hyper-V nodes. To do this, you will use the Failover Cluster Management MMC from either node. In that tool, you will:

[LIST=1][*]Validate the configuration[*]Create the cluster[*]Adjust the quorum configuration[*]Create the virtual machine in one of the nodes[*]Make the VM highly available[/LIST][/INDENT][B]Running Validation[/B]
[INDENT] Here’s the initial screen of the Failover Cluster Management MMC, when first loaded.
[IMG]http://blogs.technet.com/photos/josebda/images/3089405/original.aspx[/IMG]
Before you create the cluster, you must Validate your Configuration. Be sure to run *all* Validation tests, since solutions are only supported if you do so.
Since we are not using shared storage, the storage tests will generate a warning. Completing validation with a warning is acceptable.
If you run into any errors during Validation, you must fix those before you proceed.[/INDENT][B]Creating the Cluster[/B]
[INDENT] After you run validation, click the option to “Create a Cluster”. First, you must specify the nodes. In this case we’re using SPTNODE1 and SPTNODE2.
[IMG]http://blogs.technet.com/photos/josebda/images/3089406/original.aspx[/IMG]
Second, you specify the name of the cluster.
[IMG]http://blogs.technet.com/photos/josebda/images/3089407/original.aspx[/IMG]
After confirming the data entered, the cluster is created, as shown below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089408/original.aspx[/IMG]
Note that we end up with a warning (yellow triangle). If you click the “View Report” button, you find what the issue is:
[INDENT] [I]No appropriate disk could be found for the quorum disk.[/I] [/INDENT]This is expected. With only two nodes with no shared storage, you don’t have a valid quorum configuration and a single node failure will cause the cluster to fail.
You can see that in the cluster information below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089409/original.aspx[/IMG]
Typically, in a shared storage configuration, you would get that third vote from a shared witness disk (also know as a quorum disk).
We will overcome that in the next step.[/INDENT][B]Configuring the Cluster Quorum Settings[/B]
[INDENT] To get our third vote for the cluster without using shared storage, we will use the new option in Windows Server 2008 Failover Clustering to use a file server witness.
First, you need to add permission for the cluster computer account to the file share. The cluster computer account was created when we created the cluster.
As you did when granting permissions to SPTNODE1 and SPTNODE2, add full control permissions for the SPTDEMO\SPTCLUSTER$ account in the share and in the file system at SPTSERVER1.
Next, use the Failover Cluster Management tool to change the Quorum Configuration.
You will find this option by right-clicking the cluster name, then selecting “More Actions”, as shown below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089413/original.aspx[/IMG]
The wizard will guide you through the process. You will select the option for “Node and File Share Majority”, as shown below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089414/original.aspx[/IMG]
In the next screen, you will specify the actual shared folder path for the file share witness resource. We will use \\SPTSERVER1\VMSHARE\WITNESS. See below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089415/original.aspx[/IMG]
After you confirm the operation, you will see the update in the quorum configuration, now showing no warning signs.
[IMG]http://blogs.technet.com/photos/josebda/images/3089697/original.aspx[/IMG]
I would recommend that you also check the status of the storage in the cluster.
You do this by clicking on the “Storage” node under the cluster name in the Failover Cluster Management tool. Here’s what you should see at this point:
[IMG]http://blogs.technet.com/photos/josebda/images/3089698/original.aspx[/IMG]
As you can see, this is one of the cases where you have a healthy cluster with no shared storage. Exchange Server 2007 CCR clusters also do that.[/INDENT][B]Creating a regular Virtual Machine on a cluster node[/B]
[INDENT] At this point, if you check the Hyper-V Manager tool, you will see no virtual machines:
[IMG]http://blogs.technet.com/photos/josebda/images/3089699/original.aspx[/IMG]
Now we will use the Hyper-V Manager to create a new VM in SPTNODE1 using only a file share for storage. If you’re doing this from SPTNODE1, you should have no isses. If you’re doing this from any other computer (like the management client SPTCLIENT1), be sure to check this post on how to configure Constrained Delegation to allow remote management of Hyper-V when using file shares: [url=http://blogs.technet.com/josebda/archive/2008/06/27/using-constrained-delegation-to-remotely-manage-a-server-running-hyper-v-that-uses-cifs-smb-file-shares.aspx]Jose Barreto's Blog : Using Constrained Delegation to remotely manage a server running Hyper-V that uses CIFS/SMB file shares[/url]
Again, this is done through a wizard. This is a regular VM creation, except for the fact that we’re using UNC paths (file share paths) for the storage, instead of regular folders on a local disk. In my specific case, we’re storing this new VM at \\SPTSERVER1\VMSHARE\VM1.
Here you see the virtual machine configuration folder:
[IMG]http://blogs.technet.com/photos/josebda/images/3089701/original.aspx[/IMG]
Then the location of the new VHD file for the VM:
[IMG]http://blogs.technet.com/photos/josebda/images/3089703/original.aspx[/IMG]
And even the ISO file we’re mounting will also come from that file server:
[IMG]http://blogs.technet.com/photos/josebda/images/3089704/original.aspx[/IMG]
Once all is confirmed, we have a new VM, which you should keep in an “off” state for now:
[IMG]http://blogs.technet.com/photos/josebda/images/3089705/original.aspx[/IMG][/INDENT][B]Making the Virtual Machine Highly Available[/B]
[INDENT] Now we go back to the Failover Cluster Management tool to make the newly created VM highly available.
Click on the “Services and Applications” node under the cluster name and select the option to “Configure a Service or Application”. Again, it’s a wizard:
[IMG]http://blogs.technet.com/photos/josebda/images/3089706/original.aspx[/IMG]
After selecting “Virtual Machine” as the type of service, you will select from a list of existing VMs. In our case, there’s only VM1:
[IMG]http://blogs.technet.com/photos/josebda/images/3089707/original.aspx[/IMG]
After confirming your settings, the VM is made highly available, with a warning:
[IMG]http://blogs.technet.com/photos/josebda/images/3089708/original.aspx[/IMG]
Again, if you click on the “View Report” button, you find the issues
[INDENT] [I]The path '\\SPTSERVER1\VMSHARE\VM1' where the virtual machine configuration is stored is not on a failover cluster and might not be highly available. To achieve the highest availability, store the virtual machine configuration on a clustered file server (configured within a failover cluster).[/I]
[I]The path '\\SPTSERVER1\VMSHARE\VM1' where the virtual machine snapshots are stored is not on a failover cluster and might not be highly available. To achieve the highest availability, store the virtual machine snapshots on a clustered file server (configured within a failover cluster).[/I]
[I]The path '\\SPTSERVER1\VMSHARE\VM1\VM1.vhd' where the virtual hard disk is stored is not on a failover cluster and might not be highly available. To achieve the highest availability, store the virtual hard disk on a clustered file server (configured within a failover cluster).[/I]
[I]The path '\\SPTSERVER1\VMSHARE\ISO\WindowsServer2008-amd64.iso' where the virtual hard disk is stored is not on a failover cluster and might not be highly available. To achieve the highest availability, store the virtual hard disk on a clustered file server (configured within a failover cluster).[/I][/INDENT]As it usually does, the Failover Cluster Management tool is being very careful, pointing out that the file server share you are using is a potential single point of failure.
In order to have true high availability, you need to make sure that file share is also highly available. To achieve that, you need to place that file share in Failover Cluster as well.
The wizard has no way to detect if the file share is also clustered, so you will always get these warnings.
Now, you can go back and check the properties of the new highly available VM and bring it online.
One interesting thing you will notice is that you will not have any storage associated with that service, as you can see below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089709/original.aspx[/IMG]
In the summary page, you also confirm that, since you do not have the typical clustered disk listed in the summary for the virtual machine:
[IMG]http://blogs.technet.com/photos/josebda/images/3089721/original.aspx[/IMG][/INDENT][B]Moving the VM to another node[/B]
[INDENT] The last step is to prove that you can fail or move the VM to another node.
To do this, I use the option to “Move this service or application to another node”, which you can find when you right-click the virtual machine. See below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089724/original.aspx[/IMG]
When you do this, you will see that the VM will be taken offline in the source node (the state is saved first), as you can see below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089725/original.aspx[/IMG]
Then the VM will be brought online on the destination node (by restoring the state). Check below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089726/original.aspx[/IMG]
This process takes only a moment, and will depend only on how much memory you VM has and how long it takes to save the state to the file server share (from SPTNODE1) and then to restore the state from that same file share (from SPTNODE2).
You can see the final state, after the move to SPTNODE2 is completed, below:
[IMG]http://blogs.technet.com/photos/josebda/images/3089727/original.aspx[/IMG][/INDENT][B]Conclusion[/B]
[INDENT] I hope these steps will give you enough to recreate this scenario in your environment and validate that you can create a Failover Cluster for Hyper-V using only a CIFS/SMB file share for storage.
Keep in mind that, if your configuration can pass the Failover Cluster Validation with only a warning due to the missing shared storage, you solution is supported.
Also remember that this solution will only be truly highly available if you file share is also highly available. I’m sure you will find plenty of documentation out there on how to make file servers highly available.
Last, there is the question of performance. As I mentioned in the previous blog post using Hyper-V with file shares, I was impressed with the increased performance of a Windows Server 2008 file server, specially for this type of workload. However, I will leave it to you to test this configuration for yourself and draw your own conclusions about performance. With faster IP networks (including 10Gb Ethernet) and the improvements in the SMB v2 protocol, you might be tempted to trade performance for the added flexibility and reduced cost this could bring to your Hyper-V storage management.[/INDENT]


[/LEFT]