Tencent Cloud Object Storage (COS) offers a service availability of 99.95% and reliability of 99.999999999%. Due to uncontrollable factors such as natural disasters and fiber-optic cable failures, neither the availability nor the reliability can reach 100% for in-cloud data; however, extremely high availability and reliability are required for certain businesses like finance.
Given this background, COS provides a high-availability disaster recovery solution based on cross-bucket replication. When using COS, you are advised to make disaster recovery plans and backups for your in-cloud data based on your actual needs to keep your business uninterrupted.
This document describes a COS backup and disaster recovery solution (i.e., master/slave switch for cloud-based businesses) as well as a COS high-availability solution based on cross-bucket replication. Different COS products and features such as cross-bucket replication, origin-pull, SCF, and CDN achieve high availability.
Disaster recovery entails three elements: redundancy, remote, and replication.
COS cross-bucket replication enables cross-bucket syncing of incremental data. Data uploaded to a bucket can be replicated to another bucket in seconds or minutes, depending on file size and distance. Cross-bucket replication allows you to make remote redundant backups of your data for disaster recovery and business continuity. For more information, please see Cross-Bucket Replication Overview. To enable this feature, you need to enable versioning first. For more information on versioning, please see Versioning Overview.
The schematic diagram of the backup and disaster recovery architecture based on cross-bucket replication is as shown below:
Under this architecture, your bucket A and bucket B mutually back up each other. If your data is stored in bucket A, then bucket B is the backup bucket. In order to ensure business continuity and stability, you have configured cross-bucket replication rules for bucket A and bucket B respectively. According to the rules, incremental data in bucket A will be automatically replicated to bucket B, and vice versa.
After the incremental data in bucket A is replicated to bucket B, although it is "incremental" in bucket B, it will not be replicated to bucket A.
Normally, all your read/write requests point to bucket A where all incremental data will be automatically replicated to bucket B as backups. You can add a network quality detection module to your upload or download program at the business side, allowing you to quickly switch to bucket B when a failure is detected in bucket A.
Network quality can be tested based on Serverless Cloud Function (SCF) by changing the automated testing addresses to the domain names of master and slave buckets, and modifying the alarm code snippets as needed.
Despite all the benefits, the aforementioned solution may not always be able to guarantee high availability due to the complex, ever-changing real businesses. This section proposes a high-availability solution based on cross-bucket replication and used with different COS products and features such as origin-pull, SCF, and CDN.
The schematic diagram of the high availability architecture based on cross-bucket replication is as shown below:
This architecture consists of the following layers:
The following explains how this architecture guarantees high availability:
- As full redundant backups are costly, you can choose to make redundant backups of only hot data (such as files uploaded in just a few hours) so as to reduce data storage costs.
- If you choose an origin server as part of the high availability architecture, please be sure to assess the bandwidth of the origin server and the possible impact of the limit on it when designing the architecture.
- If you want to read data from your bucket directly, your client should be able to follow 302 redirects in the HTTP protocol.
- Tencent Cloud CDN boasts nearly a thousand edge servers which provide adjacent access nodes to increase the data read speed. You can bind multiple origin servers to CDN as master and slave servers in order to ensure high availability. For more information, please see Origin Server Configuration.
- If you want to secure your origin servers as much as possible, you can set private-read/write permission for them and enable CDN origin-pull authentication so as to allow your end users to anonymously access the data cached on the CDN edge servers whiling protecting the security of the data on the origin servers.
The following documents can help you easily implement the high-availability disaster recovery architecture: