VOD Migrate Tool is an all-in-one tool that integrates data migration features. By writing a simple configuration file, you can quickly migrate media files at the source address to VOD.
Windows, Linux, and macOS.
You can install the SDK into your project through pip. If you haven't installed pip in your project environment yet, install it first as instructed at pip's official website.
pip install vodmigrate
Click here to download the source code.
Download the latest code and decompress:
git clone https://github.com/tencentyun/vod-migrate.git
cd vod-migrate
python setup.py install
Run the following command:
vodmigrate config.toml
Note:
After the migration is completed, the result will be output to the directory corresponding to the configuration item
migrateResultOutputPath
, and the filename will bevod_migrate_result.txt
.
The configuration file is in TOML format (for more information, please see config_template.toml. Make sure that the file is encoded in UTF-8). The file content can be divided into the following parts:
type
indicates the migration type, which should be filled in based on your migration needs. For example, to migrate local data to VOD, you need to configure type=migrateLocal
for [migrateType]
.
[migrateType]
type="migrateLocal"
Currently, the following migration types are supported:
migrateType | Description |
---|---|
migrateLocal | From local system to VOD |
migrateUrl | From download URL to VOD |
migrateCos | From Tencent Cloud COS to VOD |
migrateAws | From AWS S3 to VOD |
migrateAli | From Alibaba Cloud OSS to VOD |
migrateQiniu | From Qiniu Kodo to VOD |
You can configure a migration task based on your actual needs, including information configuration for VOD and task configuration.
# Common configuration for the migration tool
[common]
secretId = "SECRETID"
secretKey = "SECRETKEY"
region = 'REGION'
subAppId = 0
concurrency = 5
supportMediaClassification = [ 'video', 'audio', 'image' ]
excludeMediaType = [ ]
migrateDbStoragePath = ''
migrateResultOutputPath = ''
Name | Description |
---|---|
secretId | SecretId of your key. Replace SECRETID with your real key information, which can be obtained on the TencentCloud API key page in the CAM Console |
secretKey | SecretKey of your key. Replace SECRETKEY with your real key information, which can be obtained on the TencentCloud API key page in the CAM Console |
region | Access point region, i.e., the region where to request a VOD server. This is different from the storage region. For more information, please see the list of supported regions |
subAppId | VOD Subapplication ID. If you need to migrate files in a subapplication, enter the subapplication ID in this field; otherwise, leave it empty. |
concurrency | Number of concurrently migrated files. Maximum value: 50 |
supportMediaClassification | List of media types supported for migration. Valid values: video, audio, image |
excludeMediaType | List of file types to be excluded |
migrateDbStoragePath | Save path of the migrated db . If this parameter is empty, it means the current directory |
migrateResultOutputPath | Save path of the migration result (one migration record corresponds to one line of JSON string). If this parameter is empty, it means the current directory |
File type description:
Configure the applicable section according to the migration type described in [migrateType]
. For example, if the configuration item of [migrateType]
is type=migrateLocal
, you only need to configure the [migrateLocal]
section.
If you migrate data from a local system to VOD, configure this section. The specific configuration items and descriptions are as follows:
# Configuration section for migration from a local system to VOD
[migrateLocal]
localPath = ''
excludes = [ ]
Configuration Item | Description |
---|---|
localPath | Local path, which should be an absolute path |
excludes | Absolute path of the directory to be excluded, which means that some files in the directory at localPath are not to be migrated |
If you migrate data from a specified URL list to VOD, configure this section. The specific configuration items and descriptions are as follows:
# Configuration section for migration from a URL list download to VOD
[migrateUrl]
urllistPath = 'D:\folder\urllist.txt'
Configuration Item | Description |
---|---|
urllistPath | Absolute path of the file storing the URL list. The file content is URL text containing one original URL address per line |
If you migrate data from Tencent Cloud COS to VOD, configure this section. The specific configuration items and descriptions are as follows:
# Configuration section for migration from Tencent Cloud COS to VOD
[migrateCos]
region = 'ap-shanghai'
bucket = 'examplebucket-1250000000'
secretId = 'COS_SECRETID'
secretKey = 'COS_SECRETKEY'
prefix = ''
Configuration Item | Description |
---|---|
region | Bucket region. For more information, please see Regions and Access Domain Names |
bucket | Bucket name in the format of <BucketName-APPID> . The bucket name must include the APPID , such as examplebucket-1250000000 |
secretId | secretId of the key of the account owning the bucket, which can be viewed in TencentCloud API Key |
secretKey | secretKey of the key of the account owning the bucket, which can be viewed in TencentCloud API Key |
prefix | Prefix of the path to be migrated. If all data in the bucket needs to be migrated, leave the prefix empty |
If you migrate data from AWS S3 to VOD, configure this section. The specific configuration items and descriptions are as follows:
# Configuration section for migration from AWS S3 to VOD
[migrateAws]
region = 'ap-northeast-2'
bucket = 'bucket-aws'
accessKeyId = 'AccessKeyId'
accessKeySecret = 'AccessKeySecret'
prefix = ''
Configuration Item | Description |
---|---|
region | AWS S3 region |
bucket | AWS S3 bucket name |
accessKeyId | Replace AccessKeyId with your real key information |
accessKeySecret | Replace AccessKeySecret with your real key information |
prefix | Prefix of the path to be migrated. If all data in the bucket needs to be migrated, leave the prefix empty |
If you migrate data from Alibaba Cloud OSS to VOD, configure this section. The specific configuration items and descriptions are as follows:
# Configuration section for migration from Alibaba Cloud OSS to VOD
[migrateAli]
bucket = 'bucket-aliyun'
accessKeyId = 'yourAccessKeyId'
accessKeySecret = 'yourAccessKeySecret'
endPoint = 'oss-cn-hangzhou.aliyuncs.com'
prefix = ''
Configuration Item | Description |
---|---|
bucket | Alibaba Cloud OSS bucket name |
accessKeyId | Replace yourAccessKeyId with your real key information |
accessKeySecret | Replace yourAccessKeySecret with your real key information |
endPoint | Alibaba Cloud endpoint address |
prefix | Prefix of the path to be migrated. If all data in the bucket needs to be migrated, leave the prefix empty |
If you migrate data from Qiniu Kodo to VOD, configure this section. The specific configuration items and descriptions are as follows:
# Configuration section for migration from Qiniu Kodo to VOD
[migrateQiniu]
bucket = 'bucket-qiniu'
accessKeyId = 'AccessKey'
accessKeySecret = 'SecretKey'
endPoint = 'www.bkt.clouddn.com'
prefix = ''
Configuration Item | Description |
---|---|
bucket | Qiniu Kodo bucket name |
accessKeyId | Replace AccessKey with your real key information |
accessKeySecret | Replace SecretKey with your real key information |
endPoint | Download address of Qiniu Kodo, which corresponds to downloadDomain |
prefix | Prefix of the path to be migrated. If all data in the bucket needs to be migrated, leave the prefix empty |
db
file must be cleared (i.e., deleting migrate.db
or modifying the db
storage path) to avoid errors with configuration file MD5 verification.FileId
, all of which are not related to each other.
Was this page helpful?