Help & DocumentationElastic MapReduceData MigrationManually Activating COS in Console

Manually Activating COS in Console

Last updated: 2020-05-28 16:56:25


Use Cases

This operation is applicable to clusters for which you want to activate COS based on your business needs.


Step 1. View the COS access key

Get SecretId and SecretKey in Key Management.

Step 2. Get the download path of files in the bucket

In the COS Console, create or select a bucket and enter the bucket file management page.

You are strongly recommended to select private read/write as the access permission when creating or selecting a bucket for higher data security. For more information, please see Setting Access Permission.

To manually activate COS in the EMR Console, you need to provide an authentication address to check whether the SecretId and SecretKey are correct. Therefore, you should make sure that there is at least one downloadable file under the account corresponding to the entered SecretId and SecretKey.

For example, if there is a file t1.txt in the root directory of the bucket, you can enter its object address as the authentication address. In the bucket file list, click Details of the file to enter the file details page, and copy the file object address.

Step 3. Set COS in the cluster

  1. Enter the EMR Console and click a cluster ID in Cluster List to enter the cluster details page.
  2. In Software Info > COS Access, click Set.
  3. In the pop-up window, enter the SecretId and SecretKey and authentication address obtained in the previous steps and click OK.

Step 4. Conduct a test

On the command line of the master node, run the following commands to view bucket files.

[root@172 ~]# su hadoop
[hadoop@172 root]$ hadoop fs -ls cosn://{bucket-name}/
Found 8 items
-rw-rw-rw-   1 hadoop hadoop       1366 2019-07-17 18:51 cosn://{bucket-name}/README.txt
drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/emr
drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/flume
drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/hive
-rw-rw-rw-   1 hadoop hadoop   95169691 2019-06-25 20:28 cosn://{bucket-name}/spark-test.jar
drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/test
drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/test2
drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/usr