Microsoft Small Business Server 2008 bundles Active Directory, Exchange 2007, IIS and Sharepoint into one supposedly easy to use package, tailored to run on a single server. It is aimed at networks of up to 75 users, and is great value if you want or need to run this stuff on-premise.
A new feature in Windows Server 2008 is Hyper-V, for server virtualisation; and a new feature of Small Business Server 2008 is the Premium Edition which comes with a license for a second server. One of the supported scenarios is that you use the second server to run a Hyper-V host, and run SBS 2008 itself as a virtual server.
Well, it is more or less supported. However, there is one significant oversight in Microsoft’s product design. SBS 2008 comes with a new backup solution based on imaging to an external hard drive rather than tape. Hyper-V guests do not support USB pass-through, so you cannot use the backup system in the way that is intended.
How then do you backup virtual SBS? Microsoft’s grown-up backup system is Data Protection Manager, which supports Hyper-V; but that is extra cost and an extra server, hardly appropriate for SBS.
What about just backing up the host? Since Windows Server Backup is able to backup live virtual machines, that seems a good solution. However, there are a few disadvantages:
1. Especially if your Virtual Hard Drives (VHDs) are fixed size, backing up the host is inefficient. Each VHD looks like a single huge file and will be backed up as such.
2. Backing up a live VM is a complex process. The host tells the guest it is being backed up, and VSS (Volume Shadow Services) gets invoked both on the host and in the guest. I’ve had backup failures when the guest is running, that disappear when the guest is shut down first.
3. There are special considerations when backing up a virtual domain controller, like SBS. It is recommended that you backup a domain controller from the guest:
Only use the standard way to backup and restore Active Directory Domain Controller, since the default checks of Active Directory consistency will only be performed when the Domain Controller is aware of a restore.
Using imaging solutions like Symantec Ghost and Acronis TrueImage seem like smarter backup and restore solutions, but they are imaging solutions (in contrast to proper backup and restore solutions.) Restoring an image of a Domain Controller may cause an Update Sequence Numbers (USN) rollback situation to occur, which may lead to replication problems and other undesired effects.
4. SBS 2008 backup has a very easy wizard for restoring individual files, such as a document that is inadvertently deleted. If you only have a host backup, getting at that individual file will not be quite so easy – though it should be possible to mount the backup VHD and copy the file from there.
I don’t mean to suggest that you should never do a host backup. It is a good idea; but should not be your only backup.
Is there a way to use the built-in SBS backup when running virtual? Here’s what I have come up with.
1. Use Hyper-V 2008 R2, not the first release. There is a free version, provided you can cope with server core. The reason is that R2 lets you attach and detach VHDs from the virtual SCSI controller while the guest is running.
2. Use a VHD as the backup target. A VHD attached to the virtual SCSI controller actually looks like a removable drive to the guest, so SBS backup will be happy. You need a large VHD – I suggest a dynamic VHD, which can be located on a cheap Sata drive if you like, since if the drive fails it can be easily replaced with minimal impact on the main server.
Backing up to a VHD actually works very nicely, since you can restore individual files using the wizard.
3. The tricky bit: how do we safely copy the backup VHD to an external drive each day, for off-site storage? Ideally this should not require the user to do more than simply plug and unplug a USB drive from the server.
You can accomplish this with a scheduled PowerShell script. First, grab the PowerShell Management Library for Hyper-V from Codeplex and install it on the host.
4. Next, go into the guest and open device manager. Under Disk drives, right-click the drive that is the backup VHD, click Policies, and choose Optimize for quick removal.
What if you have several drives, all called Msft Virtual Disk SCSI Disk Device? You can tell which is which by the LUN, which shows on the General tab. You can also see this number in the Hyper-V manager. I suggest you always use the same LUN for your backup VHD; in the example that follows it is LUN 6.
The reason you have to do this is that there is no way (that I know of) to script the Safe Remove Hardware wizard in order to safely remove the backup VHD. You will need to safely remove it, in order to back up the VHD with confidence. Turning off write caching means safe removal is not necessary.
5. Schedule your backup using the SBS wizard. The script will be detaching the backup target, so clearly you need to leave enough time for the backup to complete before detaching the drive. Fortunately, the SBS backup is incremental and usually completes relatively quickly; but leave plenty of margin. If a problem occurs, the SBS report will tell you.
6. Download sync.exe from sysinternals and install it on the host. It doesn’t matter where you put it, provided your script can find it. This enables you to flush the external backup drive so the user can safely disconnect it.
Why didn’t we use sync.exe on the guest as well? Because it requires a drive letter, and the SBS backup target does not have one.
7. Here’s a PowerShell script you can schedule on the host:
IPMO HyperV
Remove-VMdrive "your-vm-name" 0 6 -SCSI -DiskOnly
Copy-Item path-to-source.vhd path-to-target-vhd
Add-VMDisk "your-vm-name" 0 6 "path-to-source.vhd" -SCSI
sync.exe target-drive-letter:
In this script, your-vm-name is the name of your SBS virtual machine; path-to-source.vhd is the full path to the backup VHD; path-to-target-vhd is the full path to where you will be copying that VHD on the external drive; and target-drive-letter is the drive letter that is assigned to the external drive.
The backup VHD will be large, so this script will take a while to complete.
It is helpful to keep details of how you have configured the SBS VM, in case you need to do a restore. In particular, note the sizes of any VHDs that are attached, and whether they are SCSI or IDE, and make sure you know their internal drive letters as well. It makes sense to keep one drive letter per VHD; I suggest avoiding multiple partitions on a single VHD.
Restoring Virtual SBS
If you follow the above procedure, you will have no problem restoring individual files or Exchange mailboxes from the internal backup VHD, but there are a few complications if you need to do a bare-metal restore. Imagine that your SBS server is stolen and all you have is your recent copy of the backup.vhd.
Here’s how you can do a bare metal restore. First, install Hyper-V Server R2 on the new machine. You will need wbadmin on this machine; you can install this using ocsetup.
Next, create a new VM using the same or similar settings to the original. Details such as the exact amount of RAM or the number of virtual processors don’t matter too much, but having the same number and size of VHDs attached is important.
Attach the backup VHD to the IDE controller, not SCSI. Otherwise the restore will not be able to see it.
Now boot the VM from the SBS install media. Go into system repair, and do a full recovery from the backup.
You will find that the recovery works for the C drive but not for any drives attached as SCSI. The recovery reports:
The following disks were not restored … The backup is restored to available disks.
Fortunately there is a workaround. Shutdown the VM once recovery completes; don’t try to restart it yet. Now log onto the Hyper-V host. Use Diskpart to mount your VHDs, the backup as well as any target VHDs that were not recovered. Assign a drive letter to the backup VHD. Partition, quick format, and assign a drive letter to the target VHDs.
Now you can use wbadmin to restore individual volumes. Continue until they are all complete. Take care to use the same version as used for the system recovery; normally you would just use the latest one.
Finally, use diskpart to detach the VHDs. Go back to the Hyper-V manager and move the backup VHD to the SCSI controller. Start the VM.
All going well, you will get a successful boot. In my tests, I found that the virtual NICs were re-detected by the guest, so it was as if the NICs had been replaced. This meant fixing networking, as the SBS server lost its assigned IP. In other respects, the restored SBS seemed to work fine.
Note that you can also restore the host complete with its VMs, if you have a full backup. However, doing this on a daily basis is more difficult and lengthy than backing up from the guest. My suggestion is to backup the host occasionally as a fallback.
Helpful post? Sponsor ITWriting.com for ad-free access to the site
Thank you for a very informative post. I am running SBS 2008 as a guest on Hyper-V R2, but in Device Manager I do not have the option to set the SCSI VHD to “Optimize for quick removal”. The only options I can select are: “Enable write caching on the disk” (checked) and “Enable advanced performance” (unchecked). Checking or unchecking either option does not give me the “Optimize for quick removal” ability. Did I miss something?
Clay
The “optimize for quick removal” appears when Windows thinks it is a removeable drive. I thought this was always the case with Hyper-V R2 and SCSI drives; it certainly happens by default. However maybe there are some settings or scenarios when a virtual SCSI drive is attached as a non-removeable drive; that’s the only thing I can think of at the moment, if it is definitely R2?
You really need the backup VHD to appear as removeable.
Tim
One other thought: do you have the latest Hyper-V integration services installed in the guest?
Tim
There is an automatic way to do this and it creates convenient incremental backups as well. I wrote about it in my blog:
Peter
This approach with a virtual SCSI drive did not work for Microsoft Hyper-V Server 2008 R2; v1.0; 8/28-2009.
“Optimize for quick removal” is grayed-out.
“Optimize for performance” is grayed-out.
“Enable write caching on the disk” is enabled by default.
Changes to “Enable write caching on the disk” are not persisted when applying the setting.
Tjerk
That’s interesting. The screen grab in the article is from SBS 2008 running on Hyper-V Server 2008 R2. There must be some reason why we are getting different results. This is definitely a VHD and not a pass-through drive?
Tim
Tim – wondering if you have tried a bare metal restore from a virtual SCSI device? I’m testing a modified version of your approach–the Virtual SCSI VHD is actually on an internal drive. I then boot the VM from the SBS 08 DVD, start the Windows Complete PC Restore option, and it can’t see the backup on the SCSI target. If I move the same VHD to an IDE drive, voila, it appears as a restore source. But then my second drive (also attached as SCSI) is not visible.
As for the Quick Removal questions, I suspect that the guest is passing through the options of the host. So if the physical drive is a fixed drive, or maybe an eSATA drive, Quick Removal may not be available. Unfortunately it is not easy in the non-GUI Hyper-V Server R2 to check these options. The utility in MS KB 811392 is useful to check cache setup, but for some reason, it only reports cache setup for internal drives, not for USB drives.
These instructions, from a MS Partner Newsgroup, may be helpful:
[quote]For the safely remove device in Server Core/ Hyper-V server question, I found a chance to talk the owner of Safely Remove Hardware Applet in development team. Here are the summary of the discussion.
1. Safe Removal is only needed if you have specifically enabled caching for that removable drive.
2. Dismounting the volume is sufficient for safe removal.
3. In ServerCore/Hyper-V server, you can use the built-in fsutil to dismount the volume. Here is an example for your information.
C:Windowssystem32>fsutil volume dismount
Usage : fsutil volume dismount
Eg : fsutil volume dismount C:
[end quote]
Hi Tim,
you back up the VM to A VHD attached to the virtual SCSI controller but when you want to restore, like Mark Berry said, the wizard won’t see the virtual SCSI controller. normally you have to load the driver but where do we find the driver ?
Hi Mark,
you said “dismount the volume” from inside the VM. and the script to copy the VHD runs from the host ? so we have 2 scripts and 2 schedules ? one from the VMm the other from the host ?
thanks,
Vu Binh
@Mark @Vu I’m researching these issues. Note that we back up to a VHD on an internal drive too. This is useful even without bare metal restore because you can restore documents, Exchange etc. However this presumes the OS is up and running.
@Vu have you tried this technique for extracting the SCSI drivers:
http://www.windeploy.co.uk/blog/index.php/2009/10/extracting-hyper-v-r2-drivers/
Note that you can also backup the VM from the host.
Tim
@Mark
I’ve successfully completed a bare metal recovery using just the backup VHD. I’ve updated the article with some notes on this. Essentially, I attached the backup initially to the IDE controller for system recovery of the C drive. Then I used wbadmin on the host – but still with the backup made from the guest – to restore the other VHDs.
Tim
Hi Tim,
Thanks so much for such an informative blog. I am just starting to get a handle on backing up the SBS 2008 that I have running on a hyper V system host that is enterprise 2008 R2.
I found that by taking a usb hard drive offline in the host operating system that I could use the usb hard drive to create another virtual HD using an ide controller in the virtual machines settings.
I am able to do an unscheduled backup to this drive but unfortunately it does not appear in the list of available drives with which to schedule a back up.
Also after reading your post I am not sure if the backup on my VHD (located on my external USB drive) would be any good for a restore.
Mark
Mark
Am I right in thinking that the USB drive is mounted as a pass-through drive? That’s the only reason I can think of for taking it offline in the host.
In this case I’d have thought you could do a scheduled backup, but you cannot swap to another USB drive as envisaged by the backup application.
Personally I prefer backing up to an internal VHD. You can then copy the backup VHD to an external drive as described in the article.
Another idea is to use a network USB hub such as a Belkin F5L009 – I’ve not tried this, but it might well work.
Tim
Sorry I didn’t think to look back at this post until today.
@Vu Binh: I was referring to dismounting the volume on the HOST, not from inside the VM. If you can’t set up the drive for safe removal, dismounting it is supposed to flush the cache and make it safe for removal. It might work in the VM too. I have since switched to Windows Server 2008 R2 as my Hyper-V host. In the GUI environment, I use HotSwap! on the host, which I think also dismounts and/or uninstalls the volume (http://mt-naka.com/hotswap/index_enu.htm). That works well at least for eSATA drives. It can also work from the command line but I haven’t tried that.
@Tim 7/12/10: Interesting approach. I’m now running SBS 08 under Hyper-V. When I tested the bare metal restore, I also ran into the can’t-see-SCSI-drives issue. I was told this is because the drivers are not available in the SBS 08 recovery environment. I did not try loading the drivers. IIRC, I did the restore this way:
1. Attach the backup VHD as one IDE and the VHD for SBS 08 Drive C: as the other IDE.
2. Did the bare metal restore using the SBS 08 recovery environment. This gives you bootable SBS with drive C: only.
3. Reset my original VM config (C: and D: on IDE, G: and the backup VHD on SCSI drives).
4. Boot into SBS, which can see the SCSI drives. It’s not happy that shares and Exchange data are missing, but too bad. Use Windows Server Backup to restore D: and G: from the backup VHD. Reboot.
To me this seems more natural than using the host wbadmin to restore guest volumes, but I can see how that should work too.
Also FWIW I copy the VHD file to a second eSATA drive without removing it from SBS. I’m assuming this would fail if the VHD were in use, but it seems to work fine as long as I schedule the copy long after the SBS backup. I used the copied VHD to test the restore and had no problem with that. Have you tried that? Is removing the SCSI drive really necessary?
@Mark
Perhaps it will work without removing and later re-attaching the SCSI drive – but doing so seems more robust to me, and it is very easy using the script.
Tim
Here is a service that does the same job… Team a VM with VHD’s on one ore more USB drives. Then you can insert and remove the drives (and take them offsite), and the service attaches and detaches the VHD’s…
Forgot the link:
http://www.probus-it.se/project/Hyper-VTools
I am running into the same problem that TJerk mentioned about quick removal being greyed out. With that being said I may have found another viable solution and would like your opinions. Using the SCSI adapter to backup SBS to a dynamic VHD on a TB raid within the host, creates a working backup that SBS likes. This VHD location remains permanent, so there is no need for scripts of the drive removal.
From the host, I have it set to backup the “backup VHD” from the SBS guest. All of the backups are staggered so none conflict with each other. Each night the SBS updates it’s dynamic VHD backup file on the TB raid, then the host backs up that VHD file to a USB. Then someone swaps the USB drive daily. Does anyone see any issues with this solution?
Dustin,
Did your solution work out for you as mentioned above?
I’m trying to find a good way to do this in VMware with an SBS 2008 SP2 (Not R2).
The customer originally was backing up to eSATA drives that also have a USB connection on them.
The USB connection is so much slower though that I’m having trouble getting the nightly backup to complete.
I was running a backup to a NAS device but am running into the same speed related issue when trying to get the backup copied to the external drive for offsite storage.
I wanted to hook up an eSATA card in the host but have not found one that is supported.
Thanks,
Marshall