- Release Notes
- Product Introduction
- Purchase Guide
- Getting Started
- Console Guide
- Console Overview
- Bucket Management
- Bucket Overview
- Creating Buckets
- Deleting Buckets
- Querying Buckets
- Empty Buckets
- Setting Access Permission
- Setting Bucket Encryption
- Setting Hotlink Protection
- Setting Origin-Pull
- Setting Cross-Origin Access
- Setting Versioning
- Setting up a Static Website
- Setting Lifecycle
- Setting Logging
- Accessing Bucket List Using Sub-Account
- Adding Bucket Policies
- Setting INTELLIGENT TIERING
- Enabling Inventory
- Domain Name Management
- Setting Bucket Tags
- Setting Cross-Bucket Replication
- Enabling Global Acceleration
- Object Management
- Uploading Objects
- Downloading Objects
- Copying Objects
- Viewing Object Information
- Searching for Objects
- Sorting and Filtering Objects
- Direct Upload Archiving
- Modifying Storage Class
- Deleting File Fragments
- Setting Object Access Permission
- Setting Object Encryption
- Custom Headers
- Deleting Objects
- Restoring an Archived Object
- Folder Management
- Data Extraction
- Setting Object Tags
- Batch Operation
- Monitoring Reports
- Data Processing
- Application Integration
- User Tools
- Best Practice
- Overview
- Access Control and Permission Management
- ACL Practices
- Cloud Access Management Practices
- Granting Sub-accounts Access to COS
- Cases of Permission Setting
- Working with COS API Access Policies
- Temporary Key Security Guide for Frontend Direct Upload to COS
- Generating and Using Temporary Keys
- Grant Sub-account Permissions to Pull Tag List
- Granting Sub-account Under One Root Account Permission to Manipulate Buckets Under Another Root Account
- Performance Optimization
- Data Migration
- Accessing COS Using the AWS S3 SDK
- Data Disaster Recovery and Backup
- Direct Data Upload
- Domain Name Management Practice
- Data Security
- Data Verification
- Big Data Practice
- Using COS in the Third-party Applications
- Introduction to COS Data Security Solution
- Developer Guide
- Creating Request
- Bucket
- Object
- Data Management
- Data Disaster Recovery
- Data Security
- Cloud Access Management (CAM)
- Batch Processing
- Monitoring and Alarms
- Global Acceleration
- Data Processing
- Troubleshooting
- API Documentation
- Introduction
- Common Request Headers
- Common Response Headers
- Error Codes
- Request Signature
- Operation List
- Service APIs
- Bucket APIs
- Basic Operations
- Access Control List (acl)
- Cross-Origin Resource Sharing (cors)
- Lifecycle
- Bucket policy(policy)
- Hotlink protection（referer)
- Tag (tagging)
- Static website(website)
- Intelligent Tiering
- Bucket inventory(inventory)
- Versioning
- Cross-Bucket Replication（replication）
- Log Management(logging)
- Global Acceleration (Accelerate)
- Bucket Encryption (encryption)
- Object APIs
- Batch Operation APIs
- Data Processing APIs
- SDK Documentation
- SDK Overview
- Preparations
- Android SDK
- Object Operations
- Uploading and Copying Objects
- Downloading Objects
- Listing Objects
- Deleting an Object
- Restoring an Archived Object
- Querying Object Metadata
- Generating a Pre-signed Link
- Configuring Preflight Requests for Cross-origin Access
- Moving an Object
- Server-Side Encryption
- Single-Connection Bandwidth Limit
- Extracting Object Content
- Getting Started
- Quick Experience
- Bucket Operations
- Remote Disaster Recovery
- Data Management
- Cloud Access Management
- Setting Custom Headers
- Setting Access Domain Names
- Image Processing
- Troubleshooting
- Object Operations
- C SDK
- C++ SDK
- .NET(C#) SDK
- Getting Started
- Bucket Operations
- Object Operations
- Uploading and Copying Objects
- Moving an Object
- Downloading Objects
- Listing Objects
- Deleting Objects
- Restoring Archived Objects
- Querying Object Metadata
- Generating Pre-Signed URLs
- Configuring Preflight Requests for Cross-Origin Access
- Server-Side Encryption
- Single-Connection Bandwidth Limit
- Extracting Object Content
- Cross-Region Disaster Recovery
- Data Management
- Cloud Access Management
- Image Processing
- Setting Custom Headers
- Setting Access Endpoints
- Troubleshooting
- Go SDK
- iOS SDK
- Java SDK
- JavaScript SDK
- Node.js SDK
- PHP SDK
- Python SDK
- Mini Program SDK
- Error Codes
- FAQs
- Service Level Agreement
- Glossary
- Release Notes
- Product Introduction
- Purchase Guide
- Getting Started
- Console Guide
- Console Overview
- Bucket Management
- Bucket Overview
- Creating Buckets
- Deleting Buckets
- Querying Buckets
- Empty Buckets
- Setting Access Permission
- Setting Bucket Encryption
- Setting Hotlink Protection
- Setting Origin-Pull
- Setting Cross-Origin Access
- Setting Versioning
- Setting up a Static Website
- Setting Lifecycle
- Setting Logging
- Accessing Bucket List Using Sub-Account
- Adding Bucket Policies
- Setting INTELLIGENT TIERING
- Enabling Inventory
- Domain Name Management
- Setting Bucket Tags
- Setting Cross-Bucket Replication
- Enabling Global Acceleration
- Object Management
- Uploading Objects
- Downloading Objects
- Copying Objects
- Viewing Object Information
- Searching for Objects
- Sorting and Filtering Objects
- Direct Upload Archiving
- Modifying Storage Class
- Deleting File Fragments
- Setting Object Access Permission
- Setting Object Encryption
- Custom Headers
- Deleting Objects
- Restoring an Archived Object
- Folder Management
- Data Extraction
- Setting Object Tags
- Batch Operation
- Monitoring Reports
- Data Processing
- Application Integration
- User Tools
- Best Practice
- Overview
- Access Control and Permission Management
- ACL Practices
- Cloud Access Management Practices
- Granting Sub-accounts Access to COS
- Cases of Permission Setting
- Working with COS API Access Policies
- Temporary Key Security Guide for Frontend Direct Upload to COS
- Generating and Using Temporary Keys
- Grant Sub-account Permissions to Pull Tag List
- Granting Sub-account Under One Root Account Permission to Manipulate Buckets Under Another Root Account
- Performance Optimization
- Data Migration
- Accessing COS Using the AWS S3 SDK
- Data Disaster Recovery and Backup
- Direct Data Upload
- Domain Name Management Practice
- Data Security
- Data Verification
- Big Data Practice
- Using COS in the Third-party Applications
- Introduction to COS Data Security Solution
- Developer Guide
- Creating Request
- Bucket
- Object
- Data Management
- Data Disaster Recovery
- Data Security
- Cloud Access Management (CAM)
- Batch Processing
- Monitoring and Alarms
- Global Acceleration
- Data Processing
- Troubleshooting
- API Documentation
- Introduction
- Common Request Headers
- Common Response Headers
- Error Codes
- Request Signature
- Operation List
- Service APIs
- Bucket APIs
- Basic Operations
- Access Control List (acl)
- Cross-Origin Resource Sharing (cors)
- Lifecycle
- Bucket policy(policy)
- Hotlink protection（referer)
- Tag (tagging)
- Static website(website)
- Intelligent Tiering
- Bucket inventory(inventory)
- Versioning
- Cross-Bucket Replication（replication）
- Log Management(logging)
- Global Acceleration (Accelerate)
- Bucket Encryption (encryption)
- Object APIs
- Batch Operation APIs
- Data Processing APIs
- SDK Documentation
- SDK Overview
- Preparations
- Android SDK
- Object Operations
- Uploading and Copying Objects
- Downloading Objects
- Listing Objects
- Deleting an Object
- Restoring an Archived Object
- Querying Object Metadata
- Generating a Pre-signed Link
- Configuring Preflight Requests for Cross-origin Access
- Moving an Object
- Server-Side Encryption
- Single-Connection Bandwidth Limit
- Extracting Object Content
- Getting Started
- Quick Experience
- Bucket Operations
- Remote Disaster Recovery
- Data Management
- Cloud Access Management
- Setting Custom Headers
- Setting Access Domain Names
- Image Processing
- Troubleshooting
- Object Operations
- C SDK
- C++ SDK
- .NET(C#) SDK
- Getting Started
- Bucket Operations
- Object Operations
- Uploading and Copying Objects
- Moving an Object
- Downloading Objects
- Listing Objects
- Deleting Objects
- Restoring Archived Objects
- Querying Object Metadata
- Generating Pre-Signed URLs
- Configuring Preflight Requests for Cross-Origin Access
- Server-Side Encryption
- Single-Connection Bandwidth Limit
- Extracting Object Content
- Cross-Region Disaster Recovery
- Data Management
- Cloud Access Management
- Image Processing
- Setting Custom Headers
- Setting Access Endpoints
- Troubleshooting
- Go SDK
- iOS SDK
- Java SDK
- JavaScript SDK
- Node.js SDK
- PHP SDK
- Python SDK
- Mini Program SDK
- Error Codes
- FAQs
- Service Level Agreement
- Glossary
Migrating Data in Non-Tencent Cloud Storage to COS
Last updated: 2020-01-06 14:20:25
Background
COS offers the following two migration methods to help you quickly migrate data in a non-Tencent Cloud storage platform to COS.
| Migration Method | Interactivity | Threshold for Large/Small Files | Concurrence | HTTPS Secure Transfer |
|---|---|---|---|---|
| Migration Service Platform (MSP) | Visual operations | Default configuration | The same for all files | Enabled |
| COS Migration | Non-visual operations by modifying configuration files | Customizable | Different for large and small files | Optional; disabling it can speed up migration |
Both migration methods support viewing data migration progress, checking file consistency, uploading again after failure, resuming upload from breakpoint and other features, which can meet your basic migration requirements. However, the two differ in interactivity, specific features, and other aspects as shown in the table above. You can choose an appropriate method in light of the comparisons.
Migration Practice
Migration Service Platform (MSP)
MSP is a platform that integrates multiple migration tools and provides visual interfaces to help you monitor and manage large-scale data migration tasks with ease. The File Migration Tool on it can help you migrate data from various public clouds or data origin servers to COS.
The steps are as follows:
- Log in to MSP.
- Click Migration Tool on the left sidebar, locate "File Migration Tool", and click Get Started.
- Create a migration task and configure task information.
- Start the task.
For more information, please see the following documents:
- Migrating from Alibaba Cloud OSS
- Migrating from Qiniu KODO
- Migrating from UCLOUD UFile
- Migrating from Kingsoft Cloud KS3
- Migrating from Baidu Cloud BOS
- Migrating from AWS S3
Tips
During data migration, how fast the data source can be read depends on the network connection, and selecting a higher QPS concurrence value when creating a file migration task will help speed up the migration.
COS Migration
COS Migration is an all-in-one tool integrating COS data migration features. You can migrate data to COS quickly after completing simple configurations.
The steps are as follows:
- Install the Java environment.
- Install the COS Migration tools.
- Modify the configuration file.
- Launch the tool.
For more information, please see COS Migration Tool.
Tips
Below describes how to configure COS Migration to maximize the migration speed:
- Adjust the threshold and concurrence of large/small files based on your network environment to implement optimal migration where large files are split into multiple parts and small files are transferred concurrently. Adjust the tool execution time and set a bandwidth limit to ensure that bandwidth usage by data migration will not affect your business operations. Such adjustments can be made by modifying the following parameters in the
[common]section of theconfig.iniconfiguration file:Parameter Name Parameter Description smallFileThreshold Threshold for small files. If the size of a file is higher than or equal to this threshold, multipart upload will be used. The default value is 5 MB. bigFileExecutorNum Concurrence of large files, which is 8 by default.
If COS is connected to over the public network with low bandwidth, reduce this value.smallFileExecutorNum Concurrence of small files, which is 64 by default.
If COS is connected to over the public network with low bandwidth, reduce this value.executeTimeWindow This parameter defines the time range when the migration tool will execute a task every day, which will enter sleep mode at other times. In sleep mode, the migration will be paused and the migration progress will be retained until the next time window when the migration will be resumed automatically. - Distributed parallel transfer can further speed up the migration. You can install COS Migration on multiple servers and perform migration tasks separately for different data sources.
Yes
No
Was this page helpful?