Last updated: 2020-08-18 10:55:21

    Feature Description

    COSCMD enables you to perform batch object operations such as upload/download/delete by using simple command lines.

    Operating Environment

    System environment

    Windows, Linux, and macOS.

    Note:

    • Please make sure that local characters are in UTF-8 format; otherwise, exceptions will occur when operating on Chinese files.
    • Please also make sure that local time is in sync with UTC, as COSCMD will malfunction when there is a large deviation between the two.

    Software requirements

    • Python 2.7/3.5/3.6.
    • Latest version of pip.

    Installation and configuration

    • For more information on the installation and configuration of the environment, please see Python.
    • For more information on the installation and configuration of pip, please see Installation.

    Download and Installation

    Installing pip

    Run the pip command to install:

    pip install coscmd

    After installation is complete, you can view the version information by running the -v or --version command.

    Updating pip

    Run the pip command to update:

    pip install coscmd -U

    Note:

    If the pip version number is greater than or equal to 10.0.0, a failure may occur when upgrading or installing dependent libraries. We recommend using pip v9.x (pip install pip==9.0.0).

    Installation through source code (not recommended)

    Click here to download the source code.

    git clone https://github.com/tencentyun/coscmd.git
    cd coscmd
    python setup.py install

    Note:

    For Python v2.6, we recommend using this method as it is easy for a failure to occur when installing pip dependent libraries.

    Offline installation

    Note:

    Please make sure that the same version of Python is installed on the two devices; otherwise, the installation will fail.

    # Run the following commands on a device that is connected to the internet
    mkdir coscmd-packages
    pip download coscmd -d coscmd-packages
    tar -czvf coscmd-packages.tar.gz coscmd-packages
    
    # Copy the installation package to a device that is not connected to the internet and run the following commands
    tar -xzvf coscmd-packages.tar.gz
    pip install coscmd --no-index -f coscmd-packages

    Directions

    Viewing help information

    You can view the tool's help information with the -h or --help command.

    coscmd -h  // View information on the current version

    The help information is as shown below:

    usage: coscmd [-h] [-d] [-b BUCKET] [-r REGION] [-c CONFIG_PATH] [-l LOG_PATH]
                  [-v]
    
                  {info,restore,createbucket,signurl,listparts,mget,list,upload,deletebucket,abort,getbucketversioning,putbucketacl,getobjectacl,download,putobjectacl,copy,config,putbucketversioning,getbucketacl,delete}
                  ...
    
    An easy-to-use but powerful command-line tool. Try 'coscmd -h' to get more
    information. Try 'coscmd sub-command -h' to learn the usage of all commands, for example
    'coscmd upload -h'
    
    positional arguments:
      {info,restore,createbucket,signurl,listparts,mget,list,upload,deletebucket,abort,getbucketversioning,putbucketacl,getobjectacl,download,putobjectacl,copy,config,putbucketversioning,getbucketacl,delete}
        config              Config your information at first
        upload              Upload file or directory to COS
        download            Download file from COS to local
        delete              Delete file or files on COS
        abort               Aborts upload parts on COS
        copy                Copy file from COS to COS
        move                move file from COS to COS
        list                List files on COS
        listparts           List upload parts
        info                Get the information of file on COS
        mget                Download file from COS to local
        restore             Restore
        signurl             Get download url
        createbucket        Create bucket
        deletebucket        Delete bucket
        putobjectacl        Set object acl
        getobjectacl        Get object acl
        putbucketacl        Set bucket acl
        getbucketacl        Get bucket acl
        putbucketversioning
                            Set the versioning state
        getbucketversioning
                            Get the versioning state
        probe               Connection test
    
    optional arguments:
      -h, --help            show this help message and exit
      -d, --debug           Debug mode
      -b BUCKET, --bucket BUCKET
                            Specify bucket
      -r REGION, --region REGION
                            Specify region
      -c CONFIG_PATH, --config_path CONFIG_PATH
                            Specify config_path
      -l LOG_PATH, --log_path LOG_PATH
                            Specify log_path
      -v, --version         show program's version number and exit

    In addition, you can also enter -h after each command (with no parameter appended) to see how to use the command. For example:

    coscmd upload -h  // View how to use the upload command

    Configuring parameters

    You need to configure certain parameters before using COSCMD. Run the following command to begin configuration:

    coscmd config [OPTION]...<FILE>...
                  [-h] --help
                  [-a] <SECRET_ID>
                  [-s] <SECRET_KEY>
                  [-t] <TOKEN>
                  [-b] <BucketName-APPID>
                  [-r] <REGION> | [-e] <ENDPOINT>
                  [-m] <MAX_THREAD>
                  [-p] <PART_SIZE>
                  [--do-not-use-ssl]
                  [--anonymous]   

    Generally, you only need to perform simple configurations, as shown in the sample below.

    coscmd config -a AChT4ThiXAbpBDEFGhT4ThiXAbp**** -s WE54wreefvds3462refgwewe**** -b examplebucket-1250000000 -r ap-beijing

    Note:

    Here, fields enclosed in "[]" are optional, while those in "<>" are required.

    The parameters are as detailed below:

    Option Parameter Description Required Valid Value
    -a Key ID, which can be obtained on the API Key Management Page on the console Yes String
    -s Key, which can be obtained on the API Key Management Page on the console Yes String
    -t Temporary key token, which should be specified in the x-cos-security-token header if a temporary key is used No String
    -b Name of the specified bucket in the format: BucketName-APPID. For more information, please see Naming Conventions Yes String
    -r Bucket region. For more information, please see Regions and Access Domain Names Yes String
    -e ENDPOINT of the request. Once configured, the REGION parameter will be invalidated. The format of this parameter is cos.<region>.myqcloud.com for default endpoint domain names and cos.accelerate.myqcloud.com for global acceleration endpoint domain names No String
    -m Maximum number of threads in a multi-thread operation (default value: 5; value range: 1–30) No Number
    -p Part size in MB in a multipart operation (default value: 1 MB; value range: 1–1000) No Number
    --do-not-use-ssl Specifies to use HTTP instead of HTTPS protocol No String
    --anonymous Anonymous operation (i.e. one that does not carry a signature) No String

    Note:

    1. You can directly edit the ~/.cos.conf file (which is a hidden file in My Documents on Windows). This file does not exist at first and is generated by the coscmd config command. You can also create it manually.
      The file content of the configured .cos.conf file is as shown below:
     [common]
    secret_id = AChT4ThiXAbpBDEFGhT4ThiXAbp****
    secret_key = WE54wreefvds3462refgwewe****
    bucket = examplebucket-1250000000
    region = ap-beijing
    max_thread = 5
    part_size = 1
    schema = https
    1. You can add schema in the configuration file to select http/https. The default value is https.
    2. You can use anonymous mode (i.e., the signature is kept empty) by selecting True/False for the anonymous item.

    Using commands to specify a bucket and region

    • Specify a bucket with -b <BucketName-APPID>.
    • The bucket name entered here must be in the format of BucketName-APPID.
    • Specify a region with -r <region>.
    # Command format
    coscmd -b <BucketName-APPID> -r <region> <action> ...
    # Sample - Create a bucket
    coscmd -b examplebucket-1250000000 -r ap-beijing createbucket
    # Sample - Upload a file
    coscmd -b examplebucket-1250000000 -r ap-beijing upload exampleobject exampleobject

    Creating bucket

    • We recommend using this command in conjunction with -b <BucketName-APPID> and -r <region>.
    # Command format
    coscmd -b <BucketName-APPID> createbucket
    # Sample
    coscmd createbucket
    coscmd -b examplebucket-1250000000 -r ap-beijing createbucket

    Deleting a bucket

    • We recommend using this command in conjunction with -b <BucketName-APPID> and -r <region>.
    # Command format
    coscmd -b <BucketName-APPID> deletebucket
    # Sample
    coscmd deletebucket
    coscmd -b examplebucket-1250000000 -r ap-beijing deletebucket
    coscmd -b examplebucket-1250000000 -r ap-beijing deletebucket -f
    • Using the -f parameter will forcibly delete the bucket, including all files, noncurrent folders (if versioning is enabled), and incomplete multipart uploads.

    Uploading a file or folder

    • The command for uploading a file is as follows:
    # Command format
    coscmd upload <localpath> <cospath>
    # Sample
    # Upload the local `/data/exampleobject` file to the `data/exampleobject` path in COS
    coscmd upload /data/exampleobject data/exampleobject 
    coscmd upload /data/exampleobject data/
    # Upload the file by specifying the header
    # Specify the object type to upload an archived file
    coscmd upload /data/exampleobject data/exampleobject -H "{'x-cos-storage-class':'Archive'}"
    # Configure metadata
    coscmd upload /data/exampleobject data/exampleobject -H "{'x-cos-meta-example':'example'}"
    • The command for uploading a folder is as follows:
    # Command format
    coscmd upload -r <localpath> <cospath>
    # Sample
    coscmd upload -r /data/examplefolder data/examplefolder
    # The storage path in COS is `examplefolder2/examplefolder`
    coscmd upload -r /data/examplefolder examplefolder2/
    # Upload to the bucket root directory
    coscmd upload -r /data/examplefolder/ /
    # Upload files synchronously while skipping those with the same MD5 value
    coscmd upload -rs /data/examplefolder data/examplefolder
    # Upload files synchronously while deleting those that have been deleted locally
    coscmd upload -rs --delete /data/examplefolder data/examplefolder
    # Ignore .txt and .doc files
    coscmd upload -rs /data/examplefolder data/examplefolder --ignore *.txt,*.doc

    Replace parameters enclosed in "<>" with the path of the local file to be uploaded (localpath) as well as the COS storage path (cospath).

    Note:

    • When uploading a file, you need to enter the COS path plus the name of the file and/or folder (please see the samples for details).
    • COSCMD supports checkpoint restart to resume the upload of large files. When the multipart upload of a large file fails, only the parts that failed to upload will be uploaded when the operation is resumed instead of starting over from scratch (please ensure that the file content of the re-uploaded directory is the same as that of the uploaded directory).
    • COSCMD performs MD5 verification on each part during multipart upload.
    • The x-cos-meta-md5 header is included by default when COSCMD uploads a file, and its value is the same as the file's MD5 value.
    • Use the -s parameter to upload files synchronously while skipping those with the same MD5 value (please note that the source files in COS must have been uploaded by using COSCMD v1.8.3.2 or above; the x-cos-meta-md5 header is included by default).
    • When setting the HTTP header using the -H parameter, please make sure that it is in JSON format, such as coscmd upload -H "{'x-cos-storage-class':'Archive','Content-Language':'zh-CN'}" <localpath> <cospath>. For more headers, please see PUT Object.
    • You can ignore certain types of files by using the --ignore parameter when uploading folders. Multiple shell wildcard rules (separated by commas ,) are supported. To ignore a specified file extension, , must be added at the end, or "" must be added to enclose the file extension.
    • Currently, a single file of up to 40 TB can be uploaded.

    Downloading a file or folder

    • The command for downloading a file is as follows:
    # Command format
    coscmd download <cospath> <localpath>
    # Sample
    coscmd download data/exampleobject /data/exampleobject
    coscmd download data/exampleobject /data/
    • The command for downloading a folder is as follows:
    # Command format
    coscmd download -r <cospath> <localpath>
    # Sample
    coscmd download -r data/examplefolder/ /data/examplefolder
    coscmd download -r data/examplefolder/ /data/
    # Download all the files in the current bucket root directory and overwrite local files
    coscmd download -rf / /data/examplefolder
    # Download all the files in the current bucket root directory synchronously while skipping those with the same MD5 value
    coscmd download -rs / /data/examplefolder
    # Download all the files in the current bucket root directory synchronously while deleting existing local files that have been deleted in the cloud
    coscmd download -rs --delete / /data/examplefolder
    # Ignore .txt and .doc files
    coscmd download -rs / /data/examplefolder --ignore *.txt,*.doc

    Replace parameters enclosed in "<>" with the COS path (cospath) of the file to be downloaded as well as the local storage path (localpath).

    Note:

    • Since the old mget API is no longer in use, please use the download API for multipart downloads.
    • If a file with the same name exists locally, the download will fail. In this case, you need to use the -f parameter to overwrite the local file.
    • Use the -s or --sync parameter to skip identical files that already exist locally when downloading a folder (provided that the downloaded files were uploaded through the COSCMD upload API and the x- cos-meta-md5 header was included).
    • You can ignore certain types of files by using the --ignore parameter when downloading folders. Multiple shell wildcard rules (separated by commas ,) are supported. To ignore a specified file extension, , must be added at the end, or "" must be added to enclose the file extension.

    Deleting a file or folder

    • The command for deleting a file is as follows:
    # Command format
    coscmd delete <cospath>
    # Sample
    coscmd delete data/exampleobject
    
    • The command for deleting a folder is as follows:
    # Command format
    coscmd delete -r <cospath>
    # Sample
    coscmd delete -r /data/examplefolder/
    coscmd delete -r /

    Replace parameters enclosed in "<>" with the COS path (cospath) of the file to be deleted. You will be prompted to confirm this operation.

    Note:

    You need to enter y to confirm a batch deletion operation. This step can be skipped if the -f parameter is used.

    Viewing incomplete multipart uploads

    • The command is as follows:
    # Command format
    coscmd listparts <cospath>
    # Sample
    coscmd listparts examplefolder/

    Clearing incomplete multipart uploads

    • The command is as follows:
    # Command format
    coscmd abort
    # Sample
    coscmd abort

    Copying a file or folder

    • The command for copying a file is as follows:
    # Command format
    coscmd copy <sourcepath> <cospath> 
    # Sample
    # Copy the object `data/exampleobject` in the bucket `examplebucket2-1250000000` to `data/examplefolder/exampleobject` in the bucket `examplebucket1-1250000000`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou copy examplebucket2-1250000000.ap-beijing.myqcloud.com/data/exampleobject data/examplefolder/exampleobject
    # Change the storage class of the file to `STANDARD_IA`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou copy examplebucket2-1250000000.ap-beijing.myqcloud.com/data/exampleobject data/examplefolder/exampleobject -H "{'x-cos-storage-class':'STANDARD_IA'}"
    # Change the storage class of the file to `ARCHIVE`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou copy examplebucket2-1250000000.ap-beijing.myqcloud.com/data/exampleobject data/examplefolder/exampleobject -H "{'x-cos-storage-class':'Archive'}"
    • The command for copying a folder is as follows:
    # Command format
    coscmd copy -r <sourcepath> <cospath>
    # Sample
    # Copy the directory `examplefolder` in the bucket `examplebucket2-1250000000` to the directory `examplefolder` in the bucket `examplebucket1-1250000000`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou copy -r examplebucket2-1250000000.cos.ap-guangzhou.myqcloud.com/examplefolder/ examplefolder
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou copy -r examplebucket2-1250000000.cos.ap-guangzhou.myqcloud.com/examplefolder/ examplefolder/
    

    Replace parameters enclosed in "<>" with the path of the COS source file (sourcepath) to be copied as well as the path of the COS destination file (cospath) to be copied to.

    Note:

    • The sourcepath is in the format of <BucketName-APPID>.cos.<region>.myqcloud.com/<cospath>.
    • Use the -d parameter to set the x-cos-metadata-directive header (valid values: Copy, Replaced; default value: Copy).
    • When setting the HTTP header by using the -H parameter, please make sure that it is in JSON format, such as coscmd copy -H -d Replaced "{'x-cos-storage-class':'Archive','Content-Language':'zh-CN'}" <localpath> <cospath>. For more headers, please see PUT Object - Copy.

    Moving a file or folder

    • The command for moving a file is as follows:
    # Command format
    coscmd move <sourcepath> <cospath> 
    # Sample
    # Move the object `data/exampleobject` in the bucket `examplebucket2-1250000000` to `data/examplefolder/exampleobject` in the bucket `examplebucket1-1250000000`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou move examplebucket2-1250000000.ap-beijing.myqcloud.com/data/exampleobject data/examplefolder/exampleobject
    # Change the storage class of the file to `STANDARD_IA`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou move examplebucket2-1250000000.ap-beijing.myqcloud.com/data/exampleobject data/examplefolder/exampleobject -H "{'x-cos-storage-class':'STANDARD_IA'}"
    # Change the storage class of the file to `ARCHIVE`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou move examplebucket2-1250000000.ap-beijing.myqcloud.com/data/exampleobject data/examplefolder/exampleobject -H "{'x-cos-storage-class':'Archive'}"
    • The command for moving a folder is as follows:
    # Command format
    coscmd move -r <sourcepath> <cospath>
    # Sample
    # Move the directory `examplefolder` in the bucket `examplebucket2-1250000000` to the directory `examplefolder` in the bucket `examplebucket1-1250000000`
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou move -r examplebucket2-1250000000.cos.ap-guangzhou.myqcloud.com/examplefolder/ examplefolder
    coscmd -b examplebucket1-1250000000 -r ap-guangzhou move -r examplebucket2-1250000000.cos.ap-guangzhou.myqcloud.com/examplefolder/ examplefolder/
    

    Replace parameters enclosed in "<>" with the path of the COS source file (sourcepath) to be moved as well as the path of the COS destination file (cospath) to be moved to.

    Note:

    • The sourcepath is in the format of <BucketName-APPID>.cos.<region>.myqcloud.com/<cospath>.
    • Use the -d parameter to set the x-cos-metadata-directive header (valid values: copy, Replaced; default value: copy).
    • When setting the HTTP header by using the -H parameter, please make sure that it is in JSON format, such as coscmd move -H -d Replaced "{'x-cos-storage-class':'Archive','Content-Language':'zh-CN'}" <localpath> <cospath>. For more headers, please see PUT Object - copy.

    Querying a file list

    The query command is as follows:

    # Command format
    coscmd list <cospath>
    
    # Sample
    # Recursively query the list of all files in the bucket
    coscmd list -ar
    # Recursively query the list of all files prefixed with `examplefolder`
    coscmd list examplefolder/  -ar

    Replace parameters enclosed in "<>" with the COS path (cospath) of the file list to be queried.

    • Use -a to query all files.
    • Use -r to query files recursively. The number and total size of the files will be listed at the end of the returned result.
    • Use -n num to set the maximum number of files to be queried.

    Note:

    If <cospath> is empty, the current bucket root directory will be queried by default.

    Displaying file information

    The command is as follows:

    # Command format
    coscmd info <cospath> 
    
    # Sample
    coscmd info exampleobject

    Replace parameters enclosed in "<>" with the COS path (cospath) of the file to be displayed.

    Getting signed download URL

    The command is as follows:

    # Command format
    coscmd signurl <cospath>
    
    # Sample
    coscmd signurl exampleobject
    coscmd signurl exampleobject -t 100

    Replace parameters enclosed in "<>" with the COS path (cospath) of the file for which you need to get the download URL.
    Use -t time to configure the validity period in seconds of the query signature.

    Enabling/Suspending versioning

    The command is as follows:

    # Command format
    coscmd putbucketversioning <status>
    
    # Enable versioning
    coscmd putbucketversioning Enabled
    
    # Suspend versioning
    coscmd putbucketversioning Suspended

    Replace parameters enclosed in "<>" with the desired versioning status.

    Note:

    Once versioning is enabled for a bucket, this operation cannot be reversed (i.e., versioning cannot be disabled). However, you can suspend versioning for the bucket so that subsequent object uploads will not generate multiple versions.

    Restoring an archived file

    • The command for restoring a file is as follows:
    # Command format
    coscmd restore <cospath>
    # Sample
    coscmd restore -d 3 -t Expedited exampleobject
    • The command for restoring archived files in batches is as follows:
    # Command format
    coscmd restore -r <cospath>
    # Sample
    coscmd restore -r -d 3 -t Expedited examplefolder/

    Replace parameters enclosed in "<>" with the COS path (cospath) of the file list to be queried.

    • Use -d day to configure the validity period of the temporary copy. Default value: 7.
    • Use -t tier to specify the restoration type. Enumerated values: Expedited, Standard, Bulk. Default value: Standard.

    Running commands in debugging mode

    If -d or -debug is added before a command, detailed operation information will be displayed when the command is executed as shown in the example below:

    # Display upload details by using a command in the following format:
    coscmd -d upload <localpath> <cospath>
    
    # Sample
    coscmd -d upload exampleobject exampleobject

    FAQs

    If you have any questions about COSCMD, please see COSCMD FAQs.

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help