Cloning Windows Azure Virtual Machines

In one of my previous blog articles, I demonstrated how to build a demonstration or development environment for Microsoft Dynamics CRM 2011 using Windows Azure Virtual Machine technology.  Once you get a basic virtual machine installed, you will likely want to back it up, or clone it.  This article will show you how to manage your virtual machines once you get them set up.

Background

Before I go into some of the details, I want to give a little background on how Windows Azure manages hard disk images that it uses for virtual machines.  When you first create a virtual machine, you start with an image, and that image can be one that you have 'captured', or it can be one from the Azure gallery of images.  Microsoft provides basic installs of Windows Server 2008 R2, various Linux distros and even Windows Server 2012 Release Candidate. These base images are basically unattended installs that Microsoft (or you) maintain. When you create a new virtual machine, the Azure fabric controller will start the machine up in provisioning mode, which will allow Azure to specify the password for your virtual machine.  The machine will initialize itself when it boots for the first time.  The new virtual machine will have a single 30GB hard drive that is attached as the C: drive and used for the system installation, as well as D: drive that is used for the swap file and temporary storage.

One of the new features that makes Azure Virtual Machines possible is that the hard drive is now durable, and they do this by storing the hard drive blocks in Azure blobs.  This means that your hard drive now can benefit from the redundancy and durability that is baked into the Azure blob storage infrastructure, including multiple geo-distributed replica copies.  The downside is that you are dealing with blob storage which is not quite as fast as a physical disk.  The operating system volume (C: drive) is stored in Azure blobs, but the temporary volume (D: drive) is not stored in blob storage.  As such, the D: drive is a little faster, but it is not durable and should not be used to install applications or their permanent data on it.

You can create additional drives in Windows Azure and attach them to your Azure virtual machine.  The number of drives that can be attached to a virtual machine is determined by the virtual machine size.  For most real-world installations, you are going to want to create an additional data drive and attach it to your Azure virtual machine.  Keep in mind that your C: drive is only 30GB and it will fill up when Windows applies updates or other middleware components get installed.  If you have a choice of where to install any application, choose your additional permanent data drive over the system drive whenever possible.

Cloning

Cloning a machine can be done in one of two ways.  The default mechanism of which has great documentation is to use the 'capture' mechanism in the portal to make a new image of the virtual machine.  This process involves you running sysprep on the machine and shutting it down.  The sysprep mechanism is a Windows operating system mechanism that will generify the configuration and configure the virtual machine to boot up in provisioning mode.  An image that is captured can then be used just like any other Azure image.  You can prepare an image in this manner and start up as many virtual machines as you wish.  Once you run sysprep and shut down your virtual machine, you then click the Capture button in the portal.  This will add the drive to your list of Azure images and it will delete the virtual machine.  Pay attention here - running sysprep on your Azure virtual machine will start you down a path where that virtual machine will no longer boot, and when you complete the operation by running Capture, the virtual machine will be deleted from your Azure subscription.  If this is not your desired outcome, don't start down this path.

The second mechanism to copy a virtual machine is to make a copy of the underlying VHD stored in blob storage, then register this as a disk.  This process involves you simply shutting down your virtual machine, copying the disk image, then starting it back up again.  You can then register the copy of the VHD file as another disk and make a new virtual machine using that disk image.  Please note that this mechanism is not using sysprep, and that has implications.  Keep in mind that Azure is just a virtual server host and that everything about the operating system still applies.  For example, you do not want to clone a domain controller and put it in the same network otherwise you are likely to corrupt your domain database.  Conversely you cannot run sysprep on a domain controller and have the machine boot successfully.  Pay close attention to operating systems operations before you choose how you are going to copy/clone your new virtual machine.

I will start with a basic virtual machine.  For this article, I am not attaching any additional data drives.  I am going to clone the virtual machine using blob storage copy and not use the sysprem/capture mechanism as that is already well documented.  I shut down the machine so that I can safely clone the hard drive.

When you create a new virtual machine, Windows Azure will create a storage account to host the VHD file.  The storage keys will be automatically managed for you - please do not reset your storage keys unless you don't want to boot those virtual machines again...  Note the storage blob url that is used for your virtual machine:

Click the Virtual Machines link to switch back to the main portal and then select Storage in the left navigation panel.  Select the storage name that is the same as your virtual machine.  In my case where I only have a single virtual machine, this is quite easy to spot - but once you get a few different virtual machines or are managing many different subscriptions, you will have to be careful in what you select.

Clicking on this storage account will bring you to the details for that storage account.  On this screen, click the Manage Keys action button in the footer.

This will bring up a screen that will display your access keys.  Select your primary access key, right-click, and select copy to get it into your clipboard.

Start up your favorite Azure Blob Storage Manager application.  There are many on the market, and for this example, I am going to demonstrate how to use CloudBerry Explorer for Azure Blob Storage. 

Start CloudBerry and select File | Azure Blob Storage Accounts from the menu.

This will bring up a dialog.  Select New Account and then click the Add button.

This will bring up a screen that will allow you to register your storage credentials.  Enter a name that you will recognize for this account, then paste in the shared key that you copied earlier.  Go back to the portal and copy the account name and paste it into the account here as well.  Turn on the Enable SSL option.

In the main window of CloudBerry, select the new account that you just configured for the "Source" in both panes of the tab.  This will ensure that you can make a copy of the blob in blob storage instead of copying it to your local hard drive.  You can certainly make a copy to your local hard drive, and upload fresh copies, but keep in mind that these VHD files can be huge and take an extended period of time and substantial bandwidth to do so.  Copying from one blob to another is very quick.

We will be copying the VHD file to a new container.  In the right pane, click the new container icon and enter a new folder name called 'backups'.

In the left pane, select the vhds container and in the right pane, select the new backups container.  Then select the VHD file and click the Copy icon.

Now that we have a copy of the file, we need to rename it and move it back to the vhds container.  Select the file, then click on the rename button in the toolbar.  Provide a new name and click OK.

At this point, you can either copy the file back to the vhds folder or move it.  If you want to make further copies of the VHD, I recommend copying it and keeping the backup copy in the backups container, but if this is a one-time operation, you can move the file to the vhds folder.  Here is what my vhds folder looks like now in CloudBerry:

Switch back to your Azure portal and select Virtual Machines in the left navigation pane. 

Before we can use this VHD for a virtual machine, we have to define it as a disk in Windows Azure.  Select the Disks tab at the top then click the Create Disk action at the bottom of the screen.

This will pop up a dialog.  Enter a name for the disk, and I highly recommend you use the same name as you used for the VHD file or you will go mad trying to manage your disks.  Select your VHD file, and turn on the checkbox that indicates that this VHD file is bootable (contains an operating system).

Wait for the Azure portal to indicate that it has successfully created your disk.

Switch back to Virtual Machines | VM Instances tab, then select the New action in the bottom, then Virtual Machine, then Gallery.

Switch to the My Disks tab in the left navigation and select your disk image from the list on the right.  Then click the arrow button to go to the next tab in the virtual machine wizard.

Enter a name for this virtual machine.  If you were creating this from a disk image instead of a copy of a VHD file, Windows Azure would start this machine up using the machine name you provide here in the unattended startup file.  However, because we are creating this from a pre-configured VM that has not been prepared with sysprep, the resulting virtual machine will have the same network name it had when it was shut down and the VHD file was copied.  You can boot the machine and give it a different name later, but use a virtual machine name here that you will recognize (as you cannot change it later).  Click the next arrow to move to the next screen in the wizard.

You now have to provide a unique Azure dns name for the virtual machine.  This name needs to be unique across all azure services, so keep trying until you see a green checkmark.  Complete the wizard.

At this point, Windows Azure will create the virtual machine and attempt to start it in provisioning mode.  Since this machine was not prepared with sysprep, it is going to boot in regular mode and not in an unattended installation mode.  Wait until you see a running status.

Wrap Up

At this point you can now connect to your new VM.  If you recall when you set up your first virtual machine, you had to provide a password for the administrator account.  When you created the virtual machine from the disk instead of an image, you did not have to provide a password.  This is because you are not starting up the virtual machine in an unattended install mode.  You will have to log in with the credentials of the virtual machine as they were when you shut it down and copied the VHD file.  Keep in mind that Azure is just another virtual server host and you have to manage your image files like any other virtual server.  Cloning machines can be done using sysprep and capturing the VM image or you can copy the underlying VHD file and define it as a disk.

Comments
  • Thanks, Shan.  Well done as always. 

  • Do you think you could fix this page so that the images display, they're currently broken.

    It will really help me to follow your guide.

    Many thanks,
    Paul.

  • Can't see the images on the blog.

    Can you fix it please ?