Feature Introduction
In Tencent Cloud Object Storage (COS), you can send HTTP anonymous requests or HTTP signed requests to COS via the REST API. For signed requests, the COS server authenticates the requester's identity. Authentication for COS is performed using a custom scheme based on the HMAC (Hash Message Authentication Code) key.
Signed requests and anonymous requests are detailed as follows:
Signed request: An HTTP request carries a signature. Upon receiving the message, the COS server authenticates the identity. If authentication succeeds, the server accepts and executes the request. Otherwise, it returns an error message and discards the request.
Anonymous request: An HTTP request carries no identity or authentication information. You perform the HTTP request operation via the REST API.
Applicable Scenarios
In scenarios where COS is used, for data that needs to be published externally, you can typically set objects to public read and private write. This means everyone can view them, while only accounts specified by the ACL policy can write. In this case, you can combine the ACL policy with API request signatures to authenticate access and control operation permissions and validity periods.
In scenarios where signatures are used, you can implement multi-faceted security protection for API requests:
Requester identity authentication: Verify the requester's identity through their unique ID and key.
Transmitted data tampering prevention: Signs and verifies data to ensure the integrity of the transmitted content.
Signature theft prevention: Sets a validity period for the signature to avoid its theft and reuse.
In COS, there are three methods for request signing. We recommend using Method 1 and Method 2:
Method 1: SDK Signature: This method is suitable for scenarios where developers directly use the SDK to request COS. Method 2: Signature URL: This method is suitable for scenarios where COS users directly access COS via a signed URL. Method 3: Online Signature Tool: This method is suitable for scenarios such as temporarily generating signed URLs or understanding the signature algorithm. Must-Knows
This document applies only to the COS XML version.
This document does not apply to HTTP requests for POST Object. This document applies only when you intend to perform secondary development using the raw API and need to follow the steps described herein.
Method 1: SDK Signing
The COS SDK has implemented the signature functionality. You do not need to concern yourself with signature issues when using the SDK to initiate requests or obtain signatures. If you need to implement the signature yourself or understand the specific implementations in various languages, you can refer to the signature implementation files in the respective language SDKs:
|
Android SDK | |
C SDK | |
C++ SDK | |
.NET(C#) SDK | |
Go SDK | |
iOS SDK | |
Java SDK | |
JavaScript SDK | |
Node.js SDK | |
PHP SDK | |
Python SDK | |
Mini Program SDK | |
Method 2: Signed URL
The SDK for each language currently supported by COS provides the feature to generate a signed URL. The generated URL contains a signature with a limited validity period, and this signature supports PUT or GET requests. Therefore, you can directly use this URL to upload or download objects without generating an additional signature. When generating a signed URL for upload, you can also specify headers such as Content-Type or Content-MD5 to restrict the media type of the upload or to ensure the uploaded content matches the specified content. For details on request header settings during upload, refer to the PUT Object documentation. When generating a signed URL for download, you can also specify the response-xxx request parameter to temporarily modify the relevant headers in the response during download. For details on request parameter settings during download, refer to the GET Object documentation. For detailed information on generating signed URLs with each language SDK, see the following SDK documentation for each language:
Note:
We recommend that you use temporary keys to generate pre-signed URLs. This approach, which leverages temporary authorization, further enhances the security of requests such as pre-signed uploads and downloads. When applying for temporary keys, follow the principle of least privilege to prevent the exposure of resources beyond the target bucket or object. If you must use a permanent key to generate a pre-signed URL, we recommend that you limit the key's permissions to upload or download operations only to mitigate risks.
|
Android SDK | |
C SDK | |
C++ SDK | |
.NET(C#) SDK | |
Go SDK | |
iOS SDK | |
Java SDK | |
JavaScript SDK | |
Node.js SDK | |
PHP SDK | |
Python SDK | |
Mini Program SDK | |
Method 3: Online Signing Tool
COS currently provides the COS Signature Tool. You can use this tool to temporarily generate a signature or to reference the signature generation process. For detailed usage instructions, see the COS Signature Tool documentation. Signature Algorithm Principles and Usage
Preparations
Step 1: Obtain Basic Information and Determine the Development Language.
1. Obtain the APPID, SecretId, and SecretKey on the API Key Management page in the CAM console. 2. Determine the development language.
3. Supported languages include, but are not limited to, Java, PHP, .NET, C++, Node.js, and Python. Based on your chosen development language, identify the corresponding HMAC-SHA1, SHA1, and UrlEncode functions. For algorithm examples of calling HMAC-SHA1 in different languages, see Message Digest Algorithm Examples. The HMAC-SHA1 and SHA1 functions take UTF-8 encoded strings as input and output lowercase hexadecimal strings. UrlEncode is based on UTF-8 encoding. Additionally, for printable characters within the ASCII range, the following special symbols must also be encoded.
|
(Space) | 32 | 20 | ; | 59 | 3B |
! | 33 | 21 | < | 60 | 3C |
" | 34 | 22 | = | 61 | 3D |
# | 35 | 23 | > | 62 | 3E |
$ | 36 | 24 | ? | 63 | 3F |
% | 37 | 25 | @ | 64 | 40 |
& | 38 | 26 | [ | 91 | 5B |
' | 39 | 27 | \\ | 92 | 5C |
( | 40 | 28 | ] | 93 | 5D |
) | 41 | 29 | ^ | 94 | 5E |
* | 42 | 2A | ` | 96 | 60 |
+ | 43 | 2B | { | 123 | 7B |
, | 44 | 2C | | | 124 | 7C |
/ | 47 | 2F | } | 125 | 7D |
: | 58 | 3A | None. | None. | None. |
Step 2: Generate the Signature Validity Period KeyTime.
1. Obtain the Unix timestamp StartTimestamp corresponding to the current time. The Unix timestamp counts the total number of seconds from 00:00:00 on January 1, 1970 (UTC or GMT time, that is, 08:00:00 on January 1, 1970, Beijing time) to the present.
2. Based on the preceding timestamp and the expected signature validity period, calculate the Unix timestamp EndTimestamp corresponding to the signature expiration time.
3. Concatenate the signature validity period in the format StartTimestamp;EndTimestamp to form the KeyTime. For example: 1557902800;1557910000.
Step 3: Generate UrlParamList and HttpParameters.
1. Traverse the HTTP request parameters to generate a key-to-value mapping Map and a list of keys, KeyList:
Encode the key using UrlEncode and convert it to lowercase. Encode the value using UrlEncode. If a parameter has no value, treat the value as an empty string. For example, if the request path is /?acl, treat it as /?acl=. Note:
HTTP request parameters refer to the part of the request path after ?. For example, if the request path is /?versions&prefix=example-folder%2F&delimiter=%2F&max-keys=10, the request parameters are versions&prefix=example-folder%2F&delimiter=%2F&max-keys=10.
2. Sort KeyList in lexicographical order.
3. Concatenate each key-value pair in the Map in the order of KeyList, using the format key1=value1&key2=value2&key3=value3. The resulting string is HttpParameters.
4. Concatenate each item in KeyList in the order of KeyList, using the format key1;key2;key3. The resulting string is UrlParamList, the request parameter list.
Example:
Example 1:
Request Path: /?prefix=example-folder%2F&delimiter=%2F&max-keys=10
UrlParamList: delimiter;max-keys;prefix
HttpParameters: delimiter=%2F&max-keys=10&prefix=example-folder%2F
Note:
When requests are sent, the request parameters in the request path are also encoded using UrlEncode. Therefore, do not perform UrlEncode multiple times.
Example 2:
Request Path: /exampleobject?acl
UrlParamList: acl
HttpParameters: acl=
Step 4: Generate HeaderList and HttpHeaders.
1. Traverse the HTTP request headers to generate a key-to-value mapping Map and a list of keys, KeyList. Encode the key using UrlEncode and convert it to lowercase. Encode the value using UrlEncode. Note:
When traversing HTTP request headers, you do not need to process all headers. You can filter them as needed.
2. Sort KeyList in lexicographical order.
3. Concatenate each key-value pair in the Map in the order of KeyList, using the format key1=value1&key2=value2&key3=value3. The resulting string is HttpHeaders.
4. Concatenate each item in KeyList in the order of KeyList, using the format key1;key2;key3. The resulting string is HeaderList, the request header list.
Example:
Request header:
Host: examplebucket-1250000000.cos.ap-shanghai.myqcloud.com
Date: Thu, 16 May 2019 03:15:06 GMT
x-cos-acl: private
x-cos-grant-read: uin="100000000011"
Calculate:
HeaderList = date;host;x-cos-acl;x-cos-grant-read
HttpHeaders = date=Thu%2C%2016%20May%202019%2003%3A15%3A06%20GMT&host=examplebucket-1250000000.cos.ap-shanghai.myqcloud.com&x-cos-acl=private&x-cos-grant-read=uin%3D%22100000000011%22
Signature Calculation
Step 1: Generate the API Key SignKey.
Use HMAC-SHA1 with SecretKey as the key and KeyTime as the message to calculate the message digest (hash value, in lowercase hexadecimal format). For generation methods in different languages, refer to Appendix. This yields SignKey, for example: eb2519b498b02ac213cb1f3d1a3d27a3b3c9bc5f. Step 2: Generate the Process Parameter HttpString.
Generate HttpString based on the HTTP method (HttpMethod), HTTP request path (UriPathname), HttpParameters, and HttpHeaders. The format is HttpMethod\\nUriPathname\\nHttpParameters\\nHttpHeaders\\n. Among them:
Convert HttpMethod to lowercase, for example, get or put.
UriPathname is the request path, for example, / or /exampleobject.
\\n represents a newline character. If any string within it is empty, the preceding and following newline characters must be retained, for example, get\\n/exampleobject\\n\\n\\n.
Step 3: Generate the Process Parameter StringToSign.
Generate StringToSign based on KeyTime and HttpString. The format is sha1\\nKeyTime\\nSHA1(HttpString)\\n.
Among them: sha1 is a fixed string.
\\n represents a newline character.
SHA1(HttpString) is the message digest calculated using SHA1 on HttpString, in lowercase hexadecimal format, for example: 54ecfe22f59d3514fdc764b87a32d8133ea611e6. Step 4: Generate the Process Parameter Signature.
Use HMAC-SHA1 with SignKey as the key (in string form, not raw binary) and StringToSign as the message to calculate the message digest, which is the Signature, for example: 01681b8c9d798a678e43b685a9f1bba0f6c01234. Step 5: Generate the Signature.
q-sign-algorithm=sha1
&q-ak=************************************
&q-sign-time=KeyTime
&q-key-time=KeyTime
&q-header-list=HeaderList
&q-url-param-list=UrlParamList
&q-signature=Signature
Note:
Line breaks in the preceding format are for easier reading only and are not included in the actual format.
Using a Signature
Signed HTTP requests sent to COS through the RESTful API can pass the signature in the following ways:
1. Through the standard HTTP Authorization header, for example, Authorization: q-sign-algorithm=sha1&q-ak=...&q-sign-time=1557989753;1557996953&...&q-signature=...
2. When they are used as HTTP request parameters, ensure they are URL-encoded, for example, /exampleobject?q-sign-algorithm=sha1&q-ak=...&q-sign-time=1557989753%3B1557996953&...&q-signature=...
Note:
The preceding example uses ... to omit some specific signature details.
If temporary security credentials (temporary keys) are used when the signature is calculated, you must also pass the security token field `x-cos-security-token` when sending the request. The method for passing this field varies depending on how the signature is used:
1. When passing the signature through the standard HTTP Authorization header, you must also pass the security token field through the `x-cos-security-token` request header, for example:
Authorization: q-sign-algorithm=sha1&q-ak=************************************&q-sign-time=1557989753;1557996953&...&q-signature=****************************************
x-cos-security-token: ...
2. When passing the signature through HTTP request parameters, you must also pass the security token field through the `x-cos-security-token` request parameter, for example:
/exampleobject?q-sign-algorithm=sha1&q-ak=************************************&q-sign-time=1557989753%3B1557996953&...&q-signature=****************************************&x-cos-security-token=...
Note:
The preceding example uses ... to omit some specific signature and security token details.
Signature Example
Example 1: Uploading an Object
Original Request
PUT /exampleobject(%E8%85%BE%E8%AE%AF%E4%BA%91) HTTP/1.1
Date: Thu, 16 May 2019 06:45:51 GMT
Host: examplebucket-1250000000.cos.ap-beijing.myqcloud.com
Content-Type: text/plain
Content-Length: 13
Content-MD5: mQ/fVh815F3k6TAUm8m0eg==
x-cos-acl: private
x-cos-grant-read: uin="100000000011"
ObjectContent
Intermediate Variables
KeyTime = 1557989151;1557996351
SignKey = eb2519b498b02ac213cb1f3d1a3d27a3b3c9bc5f
UrlParamList = (empty string)
HttpParameters = (empty string)
HeaderList = content-length;content-md5;content-type;date;host;x-cos-acl;x-cos-grant-read
HttpHeaders = content-length=13&content-md5=mQ%2FfVh815F3k6TAUm8m0eg%3D%3D&content-type=text%2Fplain&date=Thu%2C%2016%20May%202019%2006%3A45%3A51%20GMT&host=examplebucket-1250000000.cos.ap-beijing.myqcloud.com&x-cos-acl=private&x-cos-grant-read=uin%3D%22100000000011%22
HttpString = put\\n/exampleobject(Tencent Cloud)\\n\\ncontent-length=13&content-md5=mQ%2FfVh815F3k6TAUm8m0eg%3D%3D&content-type=text%2Fplain&date=Thu%2C%2016%20May%202019%2006%3A45%3A51%20GMT&host=examplebucket-1250000000.cos.ap-beijing.myqcloud.com&x-cos-acl=private&x-cos-grant-read=uin%3D%22100000000011%22\\n
StringToSign = sha1\\n1557989151;1557996351\\n8b2751e77f43a0995d6e9eb9477f4b685cca4172\\n
Signature = 3b8851a11a569213c17ba8fa7dcf2abec6931234
Here, `(empty string)` represents an empty string with a length of 0, and `\\n` represents a newline character.
Signed Request
PUT /exampleobject(%E8%85%BE%E8%AE%AF%E4%BA%91) HTTP/1.1
Date: Thu, 16 May 2019 06:45:51 GMT
Host: examplebucket-1250000000.cos.ap-beijing.myqcloud.com
Content-Type: text/plain
Content-Length: 13
Content-MD5: mQ/fVh815F3k6TAUm8m0eg==
x-cos-acl: private
x-cos-grant-read: uin="100000000011"
Authorization: q-sign-algorithm=sha1&q-ak=************************************&q-sign-time=1557989151;1557996351&q-key-time=1557989151;1557996351&q-header-list=content-length;content-md5;content-type;date;host;x-cos-acl;x-cos-grant-read&q-url-param-list=&q-signature=****************************************
ObjectContent
Example 2: Downloading an Object
Original Request
GET /exampleobject(%E8%85%BE%E8%AE%AF%E4%BA%91)?response-content-type=application%2Foctet-stream&response-cache-control=max-age%3D600 HTTP/1.1
Date: Thu, 16 May 2019 06:55:53 GMT
Host: examplebucket-1250000000.cos.ap-beijing.myqcloud.com
Intermediate Variables
KeyTime = 1557989753;1557996953
SignKey = 937914bf490e9e8c189836aad2052e4feeb35eaf
UrlParamList = response-cache-control;response-content-type
HttpParameters = response-cache-control=max-age%3D600&response-content-type=application%2Foctet-stream
HeaderList = date;host
HttpHeaders = date=Thu%2C%2016%20May%202019%2006%3A55%3A53%20GMT&host=examplebucket-1250000000.cos.ap-beijing.myqcloud.com
HttpString = get\\n/exampleobject(Tencent Cloud)\\nresponse-cache-control=max-age%3D600&response-content-type=application%2Foctet-stream\\ndate=Thu%2C%2016%20May%202019%2006%3A55%3A53%20GMT&host=examplebucket-1250000000.cos.ap-beijing.myqcloud.com\\n
StringToSign = sha1\\n1557989753;1557996953\\n54ecfe22f59d3514fdc764b87a32d8133ea611e6\\n
Signature = 01681b8c9d798a678e43b685a9f1bba0f6c01234
Here, `\\n` represents a newline character.
Signed Request
GET /exampleobject(%E8%85%BE%E8%AE%AF%E4%BA%91)?response-content-type=application%2Foctet-stream&response-cache-control=max-age%3D600 HTTP/1.1
Date: Thu, 16 May 2019 06:55:53 GMT
Host: examplebucket-1250000000.cos.ap-beijing.myqcloud.com
Authorization: q-sign-algorithm=sha1&q-ak=************************************&q-sign-time=1557989753;1557996953&q-key-time=1557989753;1557996953&q-header-list=date;host&q-url-param-list=response-cache-control;response-content-type&q-signature=****************************************
Appendix
Pseudocode
KeyTime = [Now];[Expires]
SignKey = HMAC-SHA1([SecretKey], KeyTime)
HttpString = [HttpMethod]\\n[HttpURI]\\n[HttpParameters]\\n[HttpHeaders]\\n
StringToSign = sha1\\nKeyTime\\nSHA1(HttpString)\\n
Signature = HMAC-SHA1(SignKey, StringToSign)
Example Message Digest Algorithms
The following examples describe how to call HMAC-SHA1 in different languages.
$sha1HttpString = sha1('ExampleHttpString');
$signKey = hash_hmac('sha1', 'ExampleKeyTime', 'YourSecretKey');
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.codec.digest.HmacUtils;
String sha1HttpString = DigestUtils.sha1Hex("ExampleHttpString");
String signKey = HmacUtils.hmacSha1Hex("YourSecretKey", "ExampleKeyTime");
import hmac
import hashlib
sha1_http_string = hashlib.sha1('ExampleHttpString'.encode('utf-8')).hexdigest()
sign_key = hmac.new('YourSecretKey'.encode('utf-8'), 'ExampleKeyTime'.encode('utf-8'), hashlib.sha1).hexdigest()
var crypto = require('crypto');
var sha1HttpString = crypto.createHash('sha1').update('ExampleHttpString').digest('hex');
var signKey = crypto.createHmac('sha1', 'YourSecretKey').update('ExampleKeyTime').digest('hex');
import (
"crypto/hmac"
"crypto/sha1"
)
h := sha1.New()
h.Write([]byte("ExampleHttpString"))
sha1HttpString := h.Sum(nil)
var hashFunc = sha1.New
h = hmac.New(hashFunc, []byte("YourSecretKey"))
h.Write([]byte("ExampleKeyTime"))
signKey := h.Sum(nil)