Online Migration Tool

Last updated: 2020-08-06 17:16:21

    Overview

    Online migration refers to migrating or synchronizing the system and applications on the server or virtual machine from your own IDC, cloud platform, or other source environments to Tencent Cloud without interrupting the system.
    Tencent Cloud provides the migration tool go2tencentcloud. After you run the migration tool on the source server, the entire source server can be migrated to the destination Cloud Virtual Machine (CVM) on Tencent Cloud. With this migration tool, you do not need to prepare, upload, or import images. The source server is directed migrated onto the cloud, which meets service requirements for enterprise cloudification, migration across cloud platforms, migration across accounts or regions, and hybrid cloud deployment.

    Migration Tool

    Supported migration modes

    Default mode

    If both your source server and destination CVM can access the public network, you can use the default migration mode.
    In the current default mode, the source server calls the Tencent Cloud APIs through Internet to initiate a migration request, and transfers data to the destination CVM to complete the migration.

    Private network migration mode

    If your source server or destination CVM is located in a private network or Virtual Private Cloud (VPC), the source server cannot directly set up a connection with the destination CVM through Internet. In this case, you can use the private network migration mode of the tool. If this migration mode is used, you need to set up a connection between the source server and destination CVM by using VPC Peering Connection, VPN Connection, Cloud Connect Network , or Direct Connect.

    • Scenario 1: if your source server or destination CVM cannot access the public network, use a server (such as the gateway) that can access the public network to call the Tencent Cloud APIs through Internet to initiate a migration request, and then transfer data to the destination CVM through the connection to complete the migration. In this scenario, the source server and destination CVM do not need to have the capability to access the public network.
    • Scenario 2: if your source server can access the public network, use the source server to call the Tencent Cloud APIs through Internet to initiate a migration request, and then transfer data to the destination CVM through the connection to complete the migration. In this scenario, the source server must have the capability to access the public network, but the destination CVM does not need to have this capability.
    • Scenario 3: if your source server can access the public network through a proxy, use the source server to call the Tencent Cloud APIs through the network proxy to initiate a migration request, and then transfer data to the destination CVM through the connection to complete the migration. In this scenario, the source server and destination CVM do not need to have the capability to access the public network.

    Supported operating systems

    Operating systems supported by the online migration tool include but are not limited to the following (32-bit or 64-bit):

    Linux Operating SystemWindows Operating System
    CentOS 5/6/7-
    Ubuntu 10/12/14/16/18
    Debian 7/8/9
    SUSE 11/12/15
    openSUSE 42
    Amazon Linux AMI
    Red Hat 7/8

    Files in the compressed package

    File NameDescription
    go2tencentcloud_x64Executable program of the migration tool for the 64-bit Linux operating system.
    go2tencentcloud_x32Executable program of the migration tool for the 32-bit Linux operating system.
    user.jsonConfiguration file of the source server and destination CVM during the migration. You need to modify the configurations based on the description of parameters in the user.json file.
    client.jsonConfiguration file of the migration tool. You need to modify the configurations based on the description of parameters in the client.json file.
    rsync_excludes_linux.txtrsync configuration file, which excludes the files and directories that do not need to be migrated in the Linux system.

    Note:

    The configuration files cannot be deleted. You must store these files under the same folder as the go2tencentcloud executable program.

    • Description of parameters in the user.json file:

      ParameterTypeRequiredDescription
      SecretIdStringYesSecret ID used by your account to access APIs. For more information, see Access Key.
      SecretKeyStringYesSecret key used by your account to access APIs. For more information, see Access Key.
      RegionStringYesRegion of the destination CVM. You only need to enter the region, but not the availability zone. For more information on values, see the region list.
      InstanceIdStringYesInstance ID of the destination CVM. The format is ins-xxxxxxxx.
      DataDisksArrayNoList of data disks to be migrated from the source server. Each element represents one data disk. Up to 20 data disks are supported.
      DataDisks.IndexIntegerYesSerial number of the data disk. Value range: [1,20]. The value 1 indicates that this data disk will be migrated to the first data disk mounted on the destination CVM. Likewise, the value 2 indicates that this data disk will be migrated to the second data disk mounted on the destination CVM.
      DataDisks.SizeIntegerYesSize of the data disk on the source server. Unit: GB. Value range: [10,16000].
      DataDisks.MountPointStringYesMount point of the data disk on the source server, for example, "/mnt/disk1".
      Example 1: to migrate a source Linux server to a CVM located in Guangzhou, configure the user.json file as follows:
      {  
        "SecretId": "your secretId",
        "SecretKey": "your secretKey",  
        "Region": "ap-guangzhou",  
        "InstanceId": "your instance id"
      }  

      Note:

      You need to replace the parameter values with the actual values.

      For example, a source Linux server has one data disk, the mount target is /mnt/disk1, and the size of the data disk is 10 GB. To migrate this server to a CVM (with at least one data disk mounted) located in the Guangzhou region, configure the user.json file as follows:

      {  
        "SecretId": "your secretId",
        "SecretKey": "your secretKey",  
        "Region": "ap-guangzhou",  
        "InstanceId": "your instance id",
        "DataDisks": [
            {
                "Index": 1,
                "Size": 10,
                "MountPoint": "/mnt/disk1"
            }
        ]
      }  

      For example, a source Linux server has two data disks. The mount target for disk 1 is /mnt/disk1, and the size is 10 GB. The mount target for disk 2 is /mnt/disk2, and the size is 20GB. To migrate this server to a CVM (with at least two data disks mounted) located in Guangzhou, with disk 1 and disk 2 of the source server to be mounted to the first and second data disks of the destination CVM, respectively, configure the user.json file as follows:

      {  
        "SecretId": "your secretId",
        "SecretKey": "your secretKey",  
        "Region": "ap-guangzhou",  
        "InstanceId": "your instance id",
        "DataDisks": [
            {
                "Index": 1,
                "Size": 10,
                "MountPoint": "/mnt/disk1"
            },
            {
                "Index": 2,
                "Size": 20,
                "MountPoint": "/mnt/disk2"
            }
        ]
      }  

      Note:

      You need to replace the parameter values with the actual values.

    • Description of parameters in the client.json file:

      ParameterTypeRequiredDescription
      Client.Net.ModeIntegerYesDefault value: 0. Valid values:0 (default mode)), 1 (private network migration mode: scenario 1), 2 (private network migration mode: scenario 2), 3 (private network migration mode: scenario 3). Enter a value based on your actual migration mode or scenario.
      Client.Net.Proxy.IpStringNoIP address of the network proxy. If you select private network migration mode: scenario 3, this parameter must be specified.
      Client.Net.Proxy.PortIntegerNoPort of the network proxy. If you select private network migration mode: scenario 3, this parameter must be specified.
      Client.Net.Proxy.UserStringNoUsername of the network proxy. If you select private network migration mode: scenario 3, this parameter must be specified.
      Client.Net.Proxy.PasswordStringNoPassword of the network proxy. If you select private network migration mode: scenario 3, this parameter must be specified.
      Client.Extra.IgnoreCheckBoolNoDefault value: false. When started, the migration tool automatically checks the environment of the source server. If you need to skip this check, set this parameter to true.
      Client.Rsync.BandwidthLimitStringNoBandwidth limit in KBytes/s. The default value is empty, meaning that there is no bandwidth limit to transfer.
      Client.Rsync.ChecksumBoolNoTransfer checksum. Setting it to true can strengthen transfer consistency check but will increase the CPU load of the source server and slow down the transfer speed. The default value is false, meaning that by default, no check is performed.

      Note:

      Apart from the above parameters, other configuration items in the client.json file usually do not need to be specified.

    • Description of the rsync_excludes_linux.txt file:
      This file is used to exclude files or configuration files under specified directories on the source Linux server, which do not need to be migrated. By default, the rsync_excludes_linux.txt file already excludes the following directories and files. Do not delete these settings.

      /dev/*
      /sys/*
      /proc/*
      /var/cache/yum/*
      /lost+found/*
      /var/lib/lxcfs/*
      /var/lib/docker-storage.btrfs/root/.local/share/gvfs-metadata/*

      If you need to exclude other directories or files, add them at the end of the rsync_excludes_linux.txt file. For example, to exclude all content of the data disk mounted on /mnt/disk1, configure the rsync_excludes_linux.txt file as follows:

      /dev/*
      /sys/*
      /proc/*
      /var/cache/yum/*
      /lost+found/*
      /var/lib/lxcfs/*
      /var/lib/docker-storage.btrfs/root/.local/share/gvfs-metadata/*
      /mnt/disk1/*

    Parameters of the migration tool

    Parameter Description
    --help Prints the help information.
    --check Checks the source server without performing the migration.
    --log-file Sets the log file name, which is log by default.
    --log-level Sets the log output level. Valid values: 1 (ERROR level), 2(INFO level), and 3 (DEBUG level). Default value: 2.
    --clean Enables the destination CVM to forcibly exit the migration mode an cleans the site. For example, if Please execute '--clean' option manually. is displayed on the console, you need to specify this parameter and run the tool so that the destination CVM exits the migration mode.
    --version Prints the version number.

    Checks Before Migration

    Before the migration, check the following items of the source server and destination CVM:

    Destination CVM
    1. Storage: the storage space in its cloud disks (including system disks and data disks) must be sufficient to store the data from the source server.
    2. Security group: the 443 and 80 ports must be open to the Internet in a security group.
    3. Bandwidth: we recommend that you increase inbound and outbound bandwidth for faster migration. The traffic consumed during migration will be approximately equal to the data volume. If needed, change your network billing method in advance.
    4. Operating system: it is recommended that you use the same operating system on both the source server and the destination CVM. Different operating systems will result in inconsistency between the image that will be created later and the actual operating system. For example, when migrating a source server with the CentOS 7 system installed, choose a CVM with the CentOS 7 system installed as the migration destination.
    Linux source server
    1. Check for and install Virtio. For more information, see Checking Virtio Drivers in Linux.
    2. Check whether rsync is installed by running which rsync for verification.
    3. Check whether SELinux is enabled. If yes, disable it.
    4. Ensure the current system time is correct, because the Tencent Cloud API will use the UNIX timestamp to check against the generated token after receiving a migration request.

    Note:

    • You can use tool commands such as sudo ./go2tencentcloud_x64 --check to automatically check the source server.
    • By default, the go2tencentcloud migration tool automatically performs checking when it is started. If you want to skip the checks and directly perform migration, set Client.Extra.IgnoreCheck to true in the client.json file.

    Migration Directions

    If you use go2tencentcloud provided by Tencent Cloud, the entire migration process is completed in three stages. You can learn the migration progress from the tool when the tool is running.

    • Stage 1: initialize the remote instance.
    • Stage 2: transfer data.
    • Stage 3: release the remote instance.

    In each stage, some subtasks are generated to execute related operations. Some time-consuming subtasks also have the maximum timeout by default. The time required for data transfer depends on the size of the data on the source server and network bandwidth. The migration tool supports checkpoint restart of data transfer.

    Note:

    After the migration starts, the destination CVM enters the migration mode. Therefore, do not perform the following operations on the destination CVM until the migration is completed and the destination CVM exits the migration mode: system reinstallation, shutdown, termination, and password reset.

    Migration directions in default mode

    1. In the user.json file, configure the source server and destination CVM for the migration.
      Configure the required parameters based on the description of parameters in the user.json file.
    2. In the client.json file, configure the migration mode and other parameters.
      Set Client.Net.Mode in the client.json file to 0, that is, select the default mode. If necessary, configure other parameters based on description of parameters in the client.json file.
    3. (Optional) Exclude files and directories on the source server that do not need to be migrated.
      When migrating the source Linux server, if you need to exclude some files and directories that do not need to be migrated, add these files and directories at the end of the rsync_excludes_linux.txt file.
    4. Run the tool.
      For example, on a 64-bit Linux source server, execute the following command as root to run the tool.
      sudo ./go2tencentcloud_x64
      If you need to set the log file name and log output level when running the tool, use the following command:
      sudo ./go2tencentcloud --log-file=my.log --log-level=3
      Please wait for the migration process to complete.
      If the migration succeeds in default mode, the console output is as follows:

    Migration directions in private network migration mode

    Scenario 1

    1. Set up a connection between the source server and the destination CVM.
      Set up a connection between the source server and the destination CVM by using VPC peering connection, VPN connection, or CCN.
    2. In the user.json file, configure the source server and destination CVM for the migration.
      Configure the required parameters based on the description of parameters in the user.json file.
    3. In the client.json file, configure the migration mode and other parameters.
      Set Client.Net.Mode in the client.json file to 1, that is, select the private network migration mode: scenario 1. If necessary, configure other parameters based on description of parameters in the client.json file.
    4. (Optional) Exclude files and directories on the source server that do not need to be migrated.
      When migrating the source Linux server, if you need to exclude some files and directories that do not need to be migrated, add these files and directories at the end of the rsync_excludes_linux.txt file.
    5. Run the tool on a server (such as the gateway) that can access the public network.
      For example, on a server that can access the public network, execute the following command to run the tool for migration stage 1.
      sudo ./go2tencentcloud_x64
      If the Stage 1 is finished and please run next stage at source machine. prompt appears, stage 1 has been completed.
    6. Run the tool on the source server.
      After step 5 (stage 1) is completed, copy the entire tool directory in stage 1 to the source server, and then run the tool for migration stage 2.
      Execute the following command to run the tool for migration stage 2.
      sudo ./go2tencentcloud_x64
      If the Stage 2 is finished and please run next stage at gateway machine. prompt appears, stage 2 has been completed.
    7. Run the tool on a server (such as the gateway) that can access the public network.
      After step 6 (stage 2) is completed, copy the entire tool directory in stage 2 to the source server in stage 1, and then run the tool for migration stage 3.
      Execute the following command to run the tool for migration stage 3.
      sudo ./go2tencentcloud_x64
      If the Migrate successfully prompt appears, the entire migration task is completed.

    Scenario 2

    1. Set up a connection between the source server and the destination CVM.
      Set up a connection between the source server and the destination CVM by using VPC peering connection, VPN connection, or CCN.
    2. In the user.json file, configure the source server and destination CVM for the migration.
      Configure the required parameters based on the description of parameters in the user.json file.
    3. In the client.json file, configure the migration mode and other parameters.
      Set Client.Net.Mode in the client.json file to 2, that is, select the private network migration mode: scenario 2. If necessary, configure other parameters based on description of parameters in the client.json file.
    4. (Optional) Exclude files and directories on the source server that do not need to be migrated.
      When migrating the source Linux server, if you need to exclude some files and directories that do not need to be migrated, add these files and directories at the end of the rsync_excludes_linux.txt file.
    5. Run the tool.
      For example, on a 64-bit Linux source server, execute the following command as root to run the tool.
      sudo ./go2tencentcloud_x64
      Please wait for the migration process to complete.
      If the following appears on the console, the migration has been completed successfully.

    Scenario 3

    1. Set up a connection between the source server and the destination CVM.
      Set up a connection between the source server and the destination CVM by using VPC peering connection, VPN connection, or CCN.
    2. In the user.json file, configure the source server and destination CVM for the migration.
      Configure the required parameters based on the description of parameters in the user.json file.
    3. In the client.json file, configure the migration mode and other parameters.
      1. Set Client.Net.Mode in the client.json file to 3, that is, select the private network migration mode: scenario 3.
      2. Set Client.Net.Proxy.Ip and Client.Net.Proxy.Port in the client.json file to the IP address and port of the network proxy.
        If your network proxy needs to be authenticated, set Client.Net.Proxy.User and Client.Net.Proxy.Password to the username and password of the network proxy. If authentication is not needed, you do not need to set these two parameters. If necessary, you can configure other parameters based on the description of parameters in the client.json file.
    4. (Optional) Exclude files and directories on the source server that do not need to be migrated.
      When migrating the source Linux server, if you need to exclude some files and directories that do not need to be migrated, add these files and directories at the end of the rsync_excludes_linux.txt file.
    5. Run the tool.
      For example, on a 64-bit Linux source server, execute the following command as root to run the tool.
      sudo ./go2tencentcloud_x64
      Please wait for the migration process to complete.
      If the following appears on the console, the migration has been completed successfully.

    Checks after the Migration

    1. If the migration fails, check the error information in the log files (under the migration tool directory by default), guides, or About service migration, and rectify the faults.
    2. If the migration succeeds, check whether the destination CVM can be started normally, whether the data on the destination CVM is consistent with that on the source server, whether the network is normal, and whether other system services run normally.
    3. If you have any doubt or encounter migration errors, check About service migration or submit a ticket.

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help