tencent cloud

Feedback

Migration with DTS

Last updated: 2022-06-24 17:17:32

    Background Overview

    Tencent Cloud Data Transmission Service (DTS) is a data transmission service that integrates such features as data migration, sync, and subscription. It helps you migrate your databases to the cloud without interrupting your business and build a high-availability database disaster recovery architecture through real-time sync channels. Its data subscription feature meets your requirements for commercial data mining and async business decoupling.

    DTS for Redis currently supports the data migration feature for you to migrate data to TencentDB in a non-stop manner at a time. In addition, in its full + incremental data migration mode, historical data in the source database written before migration and incremental data written during migration can be migrated together.

    Use Cases

    DTS for Redis data migration supports source and target databases in the following deployment modes:

    Source Database Target Database Description
    Self-built Redis database, such as self-built databases in IDC and CVM TencentDB for Redis -
    Third-party Redis TencentDB for Redis The third-party cloud vendor should grant the SYNC or PSYNC command permission.
    TencentDB for Redis TencentDB for Redis Data migration, version upgrade, and cross-region migration are supported between database instances under the same Tencent Cloud account.

    Migration Support Description

    Note:

    For compatibility issues with migration from Standalone Edition to Memory Edition (Cluster Architecture), see Check on Migration from Standard Architecture to Cluster Architecture.

    Supported versions

    • DTS supports Redis 2.8, 3.0, 3.2, 4.0, and 5.0. We recommend you migrate from an earlier version to a later version; otherwise, compatibility problems may occur.
    • Standard architecture and cluster architecture instances can be migrated to each other; however, such heterogeneous migration may has compatibility problems.
    • Supported architectures include single-node, Redis cluster, Codis, and twemproxy.
    • Migration permission requirements: To migrate data through DTS, the source instance must support SYNC or PSYNC commands.

    Supported networks

    DTS supports data migration based on the public network, CVM instances, Direct Connect gateways, VPN gateways, and CCN.

    • Public Network: The source database can be accessed through a public IP.

    • Self-Build on CVM: The source database is deployed in a CVM instance.

    • Direct Connect: The source database can be interconnected with VPCs through Direct Connect.

    • VPN Access: The source database can be interconnected with VPCs through VPN Connections.

    • CCN: The source database can be interconnected with VPCs through CCN.

    Limitations

    • To ensure migration efficiency, cross-region migration of CVM-based self-built instances is not supported.
    • To migrate instances over a public network, make sure that the source instance can be accessed over the public network.
    • Only instances that are running normally can be migrated, while instances with no password initialized or with ongoing tasks cannot.
    • The target instance must be empty. During the migration process, the target instance will be read-only and cannot be written to.
    • After the successfully migrated data is verified by your business, you can disconnect the source instance and connect to the target instance.

    Environment Requirements

    Note:

    The system will automatically check the following environment requirements before starting a migration task and report an error if a requirement is not met. You can also check them in advance.

    TypeEnvironment Requirement
    Requirements for source database
  • The source and target databases can be connected.
  • The number of databases in the source database must be less than or equal to that in the target database.
  • Requirements for target database
  • The target database version should be later than or equal to the source database version; otherwise, an alarm will be triggered for compatibility problems during verification.
  • The space of the target database must be larger than the volume of the data to be migrated in the source database.
  • The target database must be empty.
  • Migration Directions

    1. Create a migration task

    (1) Log in to the DTS console and click Create Migration Task on the data migration page.
    (2) On the Create Migration Task page, select the types and regions of the source and target databases and click Buy Now.

    2. Set the source and target databases

    Configure the Source Database Settings and Target Database Settings and click Test Connectivity. After the test is passed, click Save to proceed to the next step.

    Setting TypeConfiguration ItemDescription
    Task Configuration Task Name Set a meaningful name for easy task identification.
    Running Mode You can set Immediate execution or Scheduled execution.
    • If a scheduled task is modified and passes verification, you need to click Scheduled start again to make the task start at the scheduled time.
    • If the specified time has passed, the task will be started immediately. You can also click Immediate start to start the task immediately.
    Tag Tags are used to manage resources by category in different dimensions. If the existing tags do not meet your requirements, go to the console to create more.
    Source Database Settings Source Database TypeThe source database type selected during purchase, which cannot be changed.
    RegionThe region selected during purchase, which cannot be changed.
    Access TypeFor a third-party cloud database, you can select Public Network generally or select VPN Access, Direct Connect, or CCN based on your actual network conditions.
    In this scenario, select Direct Connect or VPN Access You need to configure VPN-IDC interconnection in this scenario. For the preparations for different access types, see Overview.
    • Public Network: The source database can be accessed through a public IP.
    • Self-Build on CVM: The source database is deployed in a CVM instance.
    • Direct Connect: The source database can be interconnected with VPCs through Direct Connect.
    • VPN Access: The source database can be interconnected with VPCs through VPN Connections.
    • Database: The source database is a TencentDB instance.
    • CCN: The source database can be interconnected with VPCs through CCN.
    VPC-based Direct Connect GatewayOnly VPC-based Direct Connect gateway is supported. Confirm the network type associated with the gateway.
    VPCSelect a VPC and subnet associated with the VPC-based Direct Connect gateway.
    Node TypeSingle-Node Migration and Cluster Migration are supported. Cluster Migration is used as an example here.
    Currently, there are no limits on the number of shards and replicas in migration from Cluster Edition Redis to Cluster Edition Redis.
    Node InfoEnter the addresses and passwords (IP:port:password or IP:port) of all shards of the source database cluster and separate the information of different nodes with line breaks.
    We strongly recommend you migrate data from a replica node of the source database to avoid any impact on business access to the source database.
    Target Database Settings Target Database TypeThe target database type selected during purchase, which cannot be changed.
    RegionThe target database region selected during purchase, which cannot be changed.
    Access TypeSelect a type based on your scenario. In this scenario, select Database.
    Database InstanceSelect the instance ID of the target database.

    3. Verify and start the task

    On the task verification page, verify the task. After the verification is passed, click Start Task.

    • Failed: It indicates that a check item failed and the task is blocked. You need to fix the problem and run the verification task again.
    • Alarm: It indicates that a check item doesn't completely meet the requirements, and the task can be continued, but the business will be affected. You need to assess whether to ignore the alarm or fix the problem and continue the task based on the alarm message.

    Return to the data migration task list, and you can see that the task has entered the Preparing status. After 1–2 minutes, the data migration task will be started.

    4. Complete the migration task

    (1) (Optional) If you want to view, delete, or perform other operations on a task, click the task and select the target operation in the Operation column. For more information, see Viewing Task.

    (2) If the keys of the source and target databases are the same, click Done in the Operation column to stop the data migration task.

    (3) After the migration task status becomes Task successful, verify the data in the target database. If the verification is passed, you can formally cut over the business. For more information, see Cutover Description.

    Event Alarming and Metric Monitoring

    (1) DTS can automatically report event alarms triggered upon migration interruption to keep you informed of any exceptions. For detailed directions, see Configuring Alarm Policy for Data Migration.
    (2) DTS allows you to view the monitoring data of various metrics during migration to understand the performance metrics of the system. For more information, see Viewing Monitoring Metric.

    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support