Manually Activating COS in Console

Last updated: 2020-08-18 16:07:56

    Use Cases

    This operation is applicable to clusters for which you want to activate COS based on your business needs.


    Step 1. View the COS access key

    Get SecretId and SecretKey in Key Management.

    Step 2. Get the download path of files in the bucket

    In the COS Console, create or select a bucket and enter the bucket file management page.

    You are strongly recommended to select private read/write as the access permission when creating or selecting a bucket for higher data security. For more information, please see Setting Access Permission.

    To manually activate COS in the EMR Console, you need to provide an authentication address to check whether the SecretId and SecretKey are correct. Therefore, you should make sure that there is at least one downloadable file under the account corresponding to the entered SecretId and SecretKey.

    For example, if there is a file t1.txt in the root directory of the bucket, you can enter its object address as the authentication address. In the bucket file list, click Details of the file to enter the file details page, and copy the file object address.

    Step 3. Set COS in the cluster


    The COS bucket and the cluster must be in the same region.

    1. Log in to the EMR Console, and click the ID/Name of a target cluster on the Cluster page to enter the cluster details page.
    2. In Software Info > COS Access, click Set.
    3. In the pop-up window, enter the SecretId and SecretKey and authentication address obtained in the previous steps and click OK.

    Step 4. Conduct a test

    On the command line of the master node, run the following commands to view bucket files.

    [root@172 ~]# su hadoop
    [hadoop@172 root]$ hadoop fs -ls cosn://{bucket-name}/
    Found 8 items
    -rw-rw-rw-   1 hadoop hadoop       1366 2019-07-17 18:51 cosn://{bucket-name}/README.txt
    drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/emr
    drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/flume
    drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/hive
    -rw-rw-rw-   1 hadoop hadoop   95169691 2019-06-25 20:28 cosn://{bucket-name}/spark-test.jar
    drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/test
    drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/test2
    drwxrwxrwx   - hadoop hadoop          0 1970-01-01 08:00 cosn://{bucket-name}/usr

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback