Introduction of Tools
redis-port (Linux 64-bit) is an open-source toolset mainly used for database synchronization, data import, and data export among Redis nodes. It supports cross-version data migration for Redis. The toolset includes the following tools: redis-sync：Supports data migration among Redis instances.
redis-restore：Allows importing Redis backup files (RDB) into the specified Redis instance.
redis-dump：Allows backing up Redis data into RDB format files.
redis-decode：Allows parsing Redis backup files (RDB) into readable files.
Compatible Versions
Supports the source Redis 2.8, 3.0, and 4.0.
Supports target instances for the Redis 2.8, 3.0, 3.2, and 4.0, as well as all editions of TencentDB, including Redis Memory Edition and CKV Edition.
redis-sync Online Migration
Migration Principle
The redis-sync tool is divided into two main modules. It simulates a binary replica node continuously synchronizing data from the source instance and translating the replicated data into write commands to update the target instance.
The data replication process consists of two phases: the full synchronization phase and the incremental synchronization phase.
Note:
The number of databases (DB) in the target instance should be greater than that in the source instance; otherwise, the migration will fail.
If the migration is interrupted due to network issues or other reasons, the target instance should be cleared before restarting the migration, otherwise, dirty data may appear.
Migration progress. When the log displays sync: rdb = 9063349 - [100.00%], it indicates that the full data synchronization is completed, and the tool is now performing incremental data synchronization. When speed=(0/0,0/0,0) appears, it indicates that the incremental data synchronization is completed.
To stop the migration, you can terminate the tool by pressing Ctrl+C or using another method to stop the tool's execution, which will stop the data synchronization process.
Parameter Description
-n: number of concurrent write tasks. It is recommended to leave it empty or set it to the value calculated by using the formula: Number of CPU cores x 2.
-m：The source instance address, formatted as "password"@ip:port; for passwordless authentication, useip:port.
-t：The target instance address, formatted as"password"@ip:port; for passwordless authentication, useip:port.
--tmpfile=FILE：The temporary file name.
--tmpfile-size=SIZE：The maximum size of the temporary file.
--help: help command.
Use Case
./redis-sync -m 127.0.0.1:6379 -t "xxx2018"@10.0.5.8:6379
Output Log
[root@VM_5_16_centos bin]
2019/02/21 09:56:00 sync.go:76: [INFO] sync: master = "127.0.0.1:6379", target = "xxx2018@10.0.5.8:6379"
2019/02/21 09:56:01 sync.go:103: [INFO] +
2019/02/21 09:56:01 sync.go:109: [INFO] sync: runid = "f63e2ad58e2fcc15c8cc122f15778389a012c1a4", offset = 18576271
2019/02/21 09:56:01 sync.go:110: [INFO] sync: rdb file = 9063349 (8.64mb)
2019/02/21 09:56:01 sync.go:208: [INFO] sync: (r/f,s/f,s) = (read,rdb.forward,rdb.skip/rdb.forward,rdb.skip)
2019/02/21 09:56:02 sync.go:250: [INFO] sync: rdb = 9063349 - [100.00%] (r/f,s/f,s)=(1703936/71754,0/0,0) ~ (1.62mb/-,-/-,-) ~ speed=(1.62mb/71754,0/0,0)
2019/02/21 09:56:03 sync.go:250: [INFO] sync: rdb = 9063349 - [100.00%] (r/f,s/f,s)=(3407872/153850,0/0,0) ~ (3.25mb/-,-/-,-) ~ speed=(1.62mb/82096,0/0,0)
2019/02/21 09:57:54 sync.go:250: [INFO] sync: rdb = 9063349 - [100.00%] (r/f,s/f,s)=(80487526/411969,0/1587212,0) ~ (76.76mb/-,-/-,-) ~ speed=(0/0,0/0,0)
Data Import by Using redis-restore
The redis-restore tool supports importing Redis backup files (RDB) into specified instances, and also supports importing AOF files. It is compatible with RDB file formats from Redis versions 2.8, 3.0, 3.2, and 4.0.
Parameter Description:
-n: number of concurrent write tasks. It is recommended to leave it empty or set it to the value calculated by using the formula: Number of CPU cores x 2.
-i: RDB file path.
-m: address of the target instance. The format is "password"@ip:port. For password-free authentication, the format is ip:port.
-a: AOF file path.
--db=DB: database ID of the target Redis instance for backup file import, which should be the same as that of the source instance.
--unixtime-in-milliseconds=EXPR: the key expiration time updated during data import.
--help: help command.
Use Case
./redis-restore dump.rdb -t 127.0.0.1:6379
Data Backup by Using redis-dump
redis-dump supports backing up data into RDB files while also supporting the backup of AOF incremental data.
Note:
Tencent Cloud Distributed Cache currently does not support backups using the redis-dump tool. You can perform data backups and downloads via the Distributed Cache console or APIs, while using the redis-dump tool to back up your self-built Redis instances.
Parameter Description:
-n: number of concurrent write tasks. It is recommended to leave it empty or set it to the value calculated by using the formula: Number of CPU cores x 2.
-m: Redis instance address, in the format of "password"@ip:port , the format for password-free authentication is ip:port .
-o: path of the output RDB file for backup.
-a: path of the output AOF file for backup.
--help: help command.
Use Case
./redis-dump 127.0.0.1:6379 -o dump.rdb