HOWTO: On-Premises to Azure - Cirrus Migrate Cloud
This article describes and provides a detailed step-by-step guide on how to use Cirrus Migrate Cloud to move data from On-Premises to Azure Managed Disks.
Cirrus Migrate Cloud is a patented solution created to help customers move data from anywhere to anywhere.
As a cloud-native migration-as-a-service solution, Cirrus Migrate Cloud enables users to easily migrate from any storage in any platform at any location to Microsoft Azure Storage.
To learn more about Cirrus Migrate Cloud, please read: What is Cirrus Migrate Cloud?
Verify that on-premises host is ready for migration. Confirm that the Azure Cloud VM Instance is created and is also ready for the migration.
Validate that CMC supports the source and destination host's Operating Systems, and their kernel is found in Which Operating Systems are Supported by Cirrus Migrate Cloud?
Please check Cirrus Migrate Cloud OS/Kernel Requirements before CMC installation from the link above.
Please also ensure both the on-premises hosts and the Azure Instances meet the minimum CPU-core and Memory requirements in this article: Tested Limits for CMC Migration
In this How-to guide, the scenario is to migrate the Data Disks from on-prem hosts to Azure cloud and not the host’s OS disks. Therefore, the user will need to have a destination host (Azure cloud instance) that has the compatible application ready. Best practices provided by the cloud provider should be followed to deploy a brand-new Virtual Machine (for example using OS images available in marketplace) before migration. Alternatively, you can also utilize the cloud platform's other free tools to migrate the virtual machine OS before using CMC to migrate the data.
Check back soon for announcement for a “Compute-level Migration for Azure” which will migrate not only the data volumes, but also the OS and convert it to run at Azure as a new instance.
Once the Azure cloud VM instance is ready, the production application/database needs to be prepared to run in the destination Azure cloud VM instance such that when the data disks are migrated and cutover, the application can run using the data disks. For example, for Microsoft SQL instances, once the database drives and log drives are migrated to an Azure SQL VM instance, use the SQL Manager GUI to “attach” the newly migrated database and log directories and start up the SQL database.
Please follow this guide if you didn’t already have a Cirrus Data Cloud account: Get Started with Cirrus Data Cloud
The above gives detailed steps on how to obtain license and to transfer license to the project. It also has steps on how to create a project, deploy the CMC Agents, and to perform your first migration. You may want to just read the steps to get an idea of the overall process. This guide provides all the steps necessary to have your first migration done successfully.
If you already have an account, please visit: https://cloud.cirrusdata.com/ to login to the Cirrus Data Cloud Portal and then follow the steps below.
First, we will create a new project reflecting the specific migration characteristics, type, owner of the migration, and any details needed to define the operations. After you login, click “My Projects” at the top-left and click “Create New Project” to create a project.
Fill in all the fields and click “Create Project.”
Follow this article to transfer enough capacity or host-based license to this project: How to Transfer License Credits to Project?
With the new project open, click on “Migration Host” at the left and click “Deploy Cirrus Migrate Cloud”. Copy the curl command for Linux, or click on the “WINDOWS” tab and copy the installation iex command for PowerShell. This same command can be used to install as many hosts as you like for this project.
Run the copied command at the source hosts and at the destination Azure instances. CMC will install and register each host to the Cirrus Data Cloud project automatically without any user interaction. This will install the CMC agents on the on-premises hosts and the cloud instances.
Once registered, the hosts should appear in the Migration Hosts list to confirm that they have been registered to the project and deployed. Pairs of source and destination hosts can now be connected, so that data blocks can be migrated from the source disks to the new destination disks on the Azure Cloud.
To connect a source host and a destination host (always in pairs), click on “H2H Connections” at the left menu and click on “Create New Connection.”
Although it is possible to make the connection in either direction (regardless of the direction of migration), in practice it is much easier to configure the Network Security Group for the destination Azure instance than to request a firewall port being opened at your on-premises datacenter. For the Azure Network Security Group, make sure this In-bound rule is added:
Select the two hosts and enter the connection address. (The destination address is the public IP address of the Azure instance. Once connected, either side can send data to the other. A H2H is needed for each pair of source-destination hosts.
To Learn more about Host-to-Host (H2H) Connections, please read: How to establish host-to-host (H2H) network connections for remote migration?
Before the migration starts, an Azure-Cirrus Data Cloud Integration must be set up. A user only needs to do this once in the project.
Click on “Integrations” from the left menu and scroll down to find “Azure Managed Disks” amongst the many available integrations. Click “ADD” to configure the Azure Integration.
Enter any Name to identify this Azure account, and enter the Azure API Credentials and click save.
Note: The Credentials are obtained after you created a Service Account in Azure Web management portal. This Service Account must have a customer rule defined for enabling the following:
Microsoft.Authorization/permissions/read
Microsoft.Compute/virtualMachines/read
Microsoft.Compute/virtualMachines/write
Microsoft.Compute/disks/read
Microsoft.Compute/disk/write
Microsoft.Network/networkInterfaces/join/action
Please consult this KB article in the creation or configuration of your Azure Service Account: Creating an Azure Service Account for Cirrus Data Cloud
For “Verify Connection”, please click open the drop box and select the Azure Instance. If the Azure Instances registered in this project are managed under different Azure Accounts, you must create additional Azure Integrations and specify the different Credentials.
The integration has been validated and added successfully.
If you need help setting up your Azure account, please read our step-by-step integration guide: Creating an Azure Service Account for Cirrus Data Cloud
Open “Migration Sessions” from the left menu and then click the “New Migration Session” Button.
Select the on-premises host as the source and click “Continue.”
Select “Remote Migration” for the migration session type, and select the H2H that is paired for the destination host.
Then select the option to “Migrate Data Volumes Only,” and click “Continue.”
Since there is no destination volume yet, users can use the auto allocation for Azure Integration to create the destination volume.
Select the devices to migrate to Azure, then click on “Auto Allocate Destination Volumes” button.
Select the previously created integration.
Then users may configure the parameters needed for the new volumes and click on “Allocate Volumes” to start the auto allocation operation.
The destination volumes will be automatically created and discovered/paired correctly with the source volumes. Click “Continue.”
The last step of creating the migration session is to complete it with parameters and additional information.
Users can configure auto re-sync and configure the resync period (default is 1440 minutes which is 24 hours). Let’s change it to 5 so that it will resync every 5 minutes. You can also configure the iQoS setting. IQoS is a purpose-built feature that intelligently minimizes the impact of migration, allowing CMC to migrate as fast as possible without impacting production I/O.
To learn more about iQoS, please read: [article coming soon].
Select the appropriate iQoS level for this migration, for example we can select “Moderate”, and click “Create Session” to start the migration.
The migration session has been created and started. Here we can observe the migration progress: Status, Changed Data, etc.
Click on the small icon at bottom-right of the window (highlighted above) to open the Changed Data Map. From the map, users can see which parts of the volumes have been migrated, and which parts have not. Changes made during the migration will be reflected on the changed data map dynamically in lighter blue dots.
The application remains live, and all changes to the source volumes are tracked throughout the entire time of synchronization. After the initial synchronization is completed, CMC will perform a re-sync every 5 minutes.
After the synchronization is completed and when the migration session is at the “Tracking Changes” state, click on “Trigger cMotion™” to prepare for cutover.
cMotion™ is a Cirrus Migrate Cloud feature that allows users to cutover the workload to the destination volumes without any downtime. Once a migration is in the cMotion™ state, users can immediately enjoy all the benefits of the destination volumes without any downtime to the production application.
Users can also click “Revert cMotion™” to back out from cMotion™ and go back to using the original source storage for production. Since cMotion™ is reversible, only changes made during cMotion™ are required to be re-synchronized.
This feature is more practical for a Local Migration where there is no latency added to the I/O sent to the destination storage. The primary purpose is to test the performance of the destination storage for a Local Migration. For remote migration, the purpose is to ensure that there is no “Final Sync” during the final cutover, because after the cMotion is engaged, all the writes are immediately redirected to the destination, therefore there is no “delta” during the final cutover. This ensures a set of hosts being cutover can do so as fast as what it takes to offline the source volumes and then immediately online the destination volumes, no final resync needed.
To finalize cutover, users will stop the application from the source host. Then, open the “Session Actions” menu and select “Finalize Cutover.”
(Note that this step is immediate, and no final synchronization is need, since cMotion™ already ensured that all updates are redirected to the destination.)
Next, bring all the source disks OFFLINE at the source (on-premises) host (or execute the “unmount” command for Linux).
Once Finalize Cutover is completed, “Rescan” all the devices in the destination Azure instance.
Go back to the migration session. Open the “Session Actions” menu and select “Complete Session” to end the migration.
Migration is now complete, and the Azure instance can start up the application!
Moved block workloads from any on-premises source to Microsoft Azure Managed Disks.
Running with no application impact.
Fully data reduced and encrypted.
Minimal application downtime at final cutover.
To obtain technical support for Cirrus Migrate Cloud, please visit the CMC portal support site at https://support.cirrusdata.cloud and use the chat button in the lower right or search our Knowledge base repository for useful articles.
Migrating to Azure with Cirrus Migrate Cloud
Cirrus Migrate Cloud is a patented solution created to help customers move data from anywhere to anywhere.
As a cloud-native migration-as-a-service solution, Cirrus Migrate Cloud enables users to easily migrate from any storage in any platform at any location to Microsoft Azure Storage.
To learn more about Cirrus Migrate Cloud, please read: What is Cirrus Migrate Cloud?
Preparation
Pre-Installation Considerations:
Verify that on-premises host is ready for migration. Confirm that the Azure Cloud VM Instance is created and is also ready for the migration.
Validate that CMC supports the source and destination host's Operating Systems, and their kernel is found in Which Operating Systems are Supported by Cirrus Migrate Cloud?
Preparing Host and Cloud VM Instance:
Source Host:
Please check Cirrus Migrate Cloud OS/Kernel Requirements before CMC installation from the link above.
Please also ensure both the on-premises hosts and the Azure Instances meet the minimum CPU-core and Memory requirements in this article: Tested Limits for CMC Migration
Destination Cloud VM Instance at Azure:
In this How-to guide, the scenario is to migrate the Data Disks from on-prem hosts to Azure cloud and not the host’s OS disks. Therefore, the user will need to have a destination host (Azure cloud instance) that has the compatible application ready. Best practices provided by the cloud provider should be followed to deploy a brand-new Virtual Machine (for example using OS images available in marketplace) before migration. Alternatively, you can also utilize the cloud platform's other free tools to migrate the virtual machine OS before using CMC to migrate the data.
Check back soon for announcement for a “Compute-level Migration for Azure” which will migrate not only the data volumes, but also the OS and convert it to run at Azure as a new instance.
Once the Azure cloud VM instance is ready, the production application/database needs to be prepared to run in the destination Azure cloud VM instance such that when the data disks are migrated and cutover, the application can run using the data disks. For example, for Microsoft SQL instances, once the database drives and log drives are migrated to an Azure SQL VM instance, use the SQL Manager GUI to “attach” the newly migrated database and log directories and start up the SQL database.
Cirrus Data Cloud Account Setup:
Please follow this guide if you didn’t already have a Cirrus Data Cloud account: Get Started with Cirrus Data Cloud
The above gives detailed steps on how to obtain license and to transfer license to the project. It also has steps on how to create a project, deploy the CMC Agents, and to perform your first migration. You may want to just read the steps to get an idea of the overall process. This guide provides all the steps necessary to have your first migration done successfully.
If you already have an account, please visit: https://cloud.cirrusdata.com/ to login to the Cirrus Data Cloud Portal and then follow the steps below.
Step-by-step Procedures
Create a Migration Project and Transfer License:
First, we will create a new project reflecting the specific migration characteristics, type, owner of the migration, and any details needed to define the operations. After you login, click “My Projects” at the top-left and click “Create New Project” to create a project.
Fill in all the fields and click “Create Project.”
Follow this article to transfer enough capacity or host-based license to this project: How to Transfer License Credits to Project?
Deploy Cirrus Migrate Cloud:
With the new project open, click on “Migration Host” at the left and click “Deploy Cirrus Migrate Cloud”. Copy the curl command for Linux, or click on the “WINDOWS” tab and copy the installation iex command for PowerShell. This same command can be used to install as many hosts as you like for this project.
Run the copied command at the source hosts and at the destination Azure instances. CMC will install and register each host to the Cirrus Data Cloud project automatically without any user interaction. This will install the CMC agents on the on-premises hosts and the cloud instances.
Once registered, the hosts should appear in the Migration Hosts list to confirm that they have been registered to the project and deployed. Pairs of source and destination hosts can now be connected, so that data blocks can be migrated from the source disks to the new destination disks on the Azure Cloud.
Host-to-Host (H2H) Connections:
To connect a source host and a destination host (always in pairs), click on “H2H Connections” at the left menu and click on “Create New Connection.”
Although it is possible to make the connection in either direction (regardless of the direction of migration), in practice it is much easier to configure the Network Security Group for the destination Azure instance than to request a firewall port being opened at your on-premises datacenter. For the Azure Network Security Group, make sure this In-bound rule is added:
Select the two hosts and enter the connection address. (The destination address is the public IP address of the Azure instance. Once connected, either side can send data to the other. A H2H is needed for each pair of source-destination hosts.
To Learn more about Host-to-Host (H2H) Connections, please read: How to establish host-to-host (H2H) network connections for remote migration?
Azure Integration:
Before the migration starts, an Azure-Cirrus Data Cloud Integration must be set up. A user only needs to do this once in the project.
Click on “Integrations” from the left menu and scroll down to find “Azure Managed Disks” amongst the many available integrations. Click “ADD” to configure the Azure Integration.
Enter any Name to identify this Azure account, and enter the Azure API Credentials and click save.
Note: The Credentials are obtained after you created a Service Account in Azure Web management portal. This Service Account must have a customer rule defined for enabling the following:
Microsoft.Authorization/permissions/read
Microsoft.Compute/virtualMachines/read
Microsoft.Compute/virtualMachines/write
Microsoft.Compute/disks/read
Microsoft.Compute/disk/write
Microsoft.Network/networkInterfaces/join/action
Please consult this KB article in the creation or configuration of your Azure Service Account: Creating an Azure Service Account for Cirrus Data Cloud
For “Verify Connection”, please click open the drop box and select the Azure Instance. If the Azure Instances registered in this project are managed under different Azure Accounts, you must create additional Azure Integrations and specify the different Credentials.
The integration has been validated and added successfully.
If you need help setting up your Azure account, please read our step-by-step integration guide: Creating an Azure Service Account for Cirrus Data Cloud
Create Migration Session:
Open “Migration Sessions” from the left menu and then click the “New Migration Session” Button.
Select the on-premises host as the source and click “Continue.”
Select “Remote Migration” for the migration session type, and select the H2H that is paired for the destination host.
Then select the option to “Migrate Data Volumes Only,” and click “Continue.”
Auto Allocation:
Since there is no destination volume yet, users can use the auto allocation for Azure Integration to create the destination volume.
Select the devices to migrate to Azure, then click on “Auto Allocate Destination Volumes” button.
Select the previously created integration.
Then users may configure the parameters needed for the new volumes and click on “Allocate Volumes” to start the auto allocation operation.
The destination volumes will be automatically created and discovered/paired correctly with the source volumes. Click “Continue.”
Migration Parameters:
The last step of creating the migration session is to complete it with parameters and additional information.
Users can configure auto re-sync and configure the resync period (default is 1440 minutes which is 24 hours). Let’s change it to 5 so that it will resync every 5 minutes. You can also configure the iQoS setting. IQoS is a purpose-built feature that intelligently minimizes the impact of migration, allowing CMC to migrate as fast as possible without impacting production I/O.
To learn more about iQoS, please read: [article coming soon].
Select the appropriate iQoS level for this migration, for example we can select “Moderate”, and click “Create Session” to start the migration.
Migration Session Starts:
The migration session has been created and started. Here we can observe the migration progress: Status, Changed Data, etc.
Click on the small icon at bottom-right of the window (highlighted above) to open the Changed Data Map. From the map, users can see which parts of the volumes have been migrated, and which parts have not. Changes made during the migration will be reflected on the changed data map dynamically in lighter blue dots.
The application remains live, and all changes to the source volumes are tracked throughout the entire time of synchronization. After the initial synchronization is completed, CMC will perform a re-sync every 5 minutes.
cMotion™:
After the synchronization is completed and when the migration session is at the “Tracking Changes” state, click on “Trigger cMotion™” to prepare for cutover.
cMotion™ is a Cirrus Migrate Cloud feature that allows users to cutover the workload to the destination volumes without any downtime. Once a migration is in the cMotion™ state, users can immediately enjoy all the benefits of the destination volumes without any downtime to the production application.
Users can also click “Revert cMotion™” to back out from cMotion™ and go back to using the original source storage for production. Since cMotion™ is reversible, only changes made during cMotion™ are required to be re-synchronized.
This feature is more practical for a Local Migration where there is no latency added to the I/O sent to the destination storage. The primary purpose is to test the performance of the destination storage for a Local Migration. For remote migration, the purpose is to ensure that there is no “Final Sync” during the final cutover, because after the cMotion is engaged, all the writes are immediately redirected to the destination, therefore there is no “delta” during the final cutover. This ensures a set of hosts being cutover can do so as fast as what it takes to offline the source volumes and then immediately online the destination volumes, no final resync needed.
Final Cutover:
To finalize cutover, users will stop the application from the source host. Then, open the “Session Actions” menu and select “Finalize Cutover.”
(Note that this step is immediate, and no final synchronization is need, since cMotion™ already ensured that all updates are redirected to the destination.)
Next, bring all the source disks OFFLINE at the source (on-premises) host (or execute the “unmount” command for Linux).
Once Finalize Cutover is completed, “Rescan” all the devices in the destination Azure instance.
Go back to the migration session. Open the “Session Actions” menu and select “Complete Session” to end the migration.
Migration is now complete, and the Azure instance can start up the application!
Migration Conclusion
Moved block workloads from any on-premises source to Microsoft Azure Managed Disks.
Running with no application impact.
Fully data reduced and encrypted.
Minimal application downtime at final cutover.
Getting Support
To obtain technical support for Cirrus Migrate Cloud, please visit the CMC portal support site at https://support.cirrusdata.cloud and use the chat button in the lower right or search our Knowledge base repository for useful articles.
Updated on: 10/03/2023
Thank you!