Batch Operation

Last updated: 2020-03-17 17:27:04

    Overview

    The batch operation feature of COS allows you to implement large-scale batch operations on objects in the bucket. Currently, you can perform the following batch operations:

    • Replicating Objects
    • Restoring an archived object

    You can put objects to be operated on in an inventory file, which is based on the inventory report generated by the inventory feature (you need to enable Inventory feature first), or you can create one in the specified format. COS batch operation will operate on the objects in batches based on this inventory file. For more information, see Overview.

    Steps

    1. Log into the COS Console and click Batch Operation on the left sidebar to enter the batch operation management page.

    2. Click Create Job to start creating a batch operation job. Configuration items are described as follows.

      • Job Region: Select a region for the job to be created. The job region should be the same as the bucket region of the objects to be operated on in your inventory file, otherwise the job will fail. *Chengdu and Chongqing regions are currently in beta test.

      • Inventory Format: Select a format for the inventory objects. There are two formats:

        Inventory Format Field Configuration Instructions
        COS inventory report Please select this if the inventory file comes from an inventory report generated by the inventory feature
        CSV Ignore Placeholder field, which can be used when your inventory file contains irregular fields
        Bucket Bucket name
        Key Name of an object in a bucket. If CSV file format is used, the object name is URL-encoded and must be decoded before use
        VersionId Object version ID. If versioning is enabled for the bucket, COS assigns a version number to the objects added to the bucket. If you do not want to use the latest object version, you can select a version ID that contains the inventory object
      • Inventory Bucket: Select the bucket where the inventory is stored.

      • Inventory File Path: Enter the inventory file or CSV file path in the format of directory/manifest.json or directory/manifest.csv, respectively. For example, if you have an inventory stored in the examplebucket-1250000000 root directory, the inventory path will be manifest.json.

    3. Click Next and the inventory Etag will be displayed, meaning you have selected the correct inventory objects. Then, you will enter the operation configuration page.

      • Replicating data in batch:
        • Destination Bucket: Select the bucket to store the replicas of objects in the inventory list.
        • Storage Type: Standard or Standard_IA.
        • Server Encryption: Select whether to encrypt the object replicas. Options include no encryption or SSE-COS encryption.
        • Access Permissions: Configure access permissions for the object replicas. Options include using permissions of the destination bucket, private read/write, and public read/private write.
        • Object Metadata: Configure metadata for the object replicas. Options include copying the original metadata or replacing all metadata.
      • Restoring archived storage in batch
        • Restoration Mode: Standard mode and batch mode. For more information on restoration mode, see Restoring Archived Objects
        • Replica Validity Period: Configure the number of days after which the copy will expire and be deleted automatically. The configuration range has a minimum of 1 day and a maximum of 365 days.
    4. Click Next to enter the "Other Configurations" page where you can configure the following items:

      • Job Description (Optional): Description of the job, which is optional.
      • Job Priority: A job of a higher priority will be operated on first. The value can be a positive integer. The greater the value, the higher the priority.
      • Job Report: Select whether to generate a job report.
      • CAM Role: You can create a CAM role or select an existing one to grant operation permissions to COS.

        To grant batch operation permissions to COS, you need to create a CAM role. For more information on CAM roles, see Role Overview.

    5. Click Next to enter the information check page where you can check the configurations of your batch operation job. If you need to make a change, click the corresponding Modify or Previous. Click OK if everything is correct.

    6. After the batch operation job is created, find the newly created job in the job list and click [OK]. To cancel the job, click [Cancel] on the right.

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help