File Decompression

Last updated: 2021-10-08 15:28:08

    Overview

    File decompression is a data processing solution provided by Tencent Cloud COS based on Serverless Cloud Function (SCF). When you upload a compressed file to a bucket with a decompression rule configured, the SCF preset by COS will be triggered automatically to decompress the file into the specified bucket and directory. The decompression flow is shown as follows:

    Decompression Flow

    Notes

    Currently, COS can decompress ZIP files only. Each file contained in the ZIP file must not be larger than 5 GB. If a single file in the ZIP file is larger than 5 GB, the decompression will fail.

    • If you have added a file decompression rule to your bucket in the COS console, a file decompression function will appear in the SCF console. DO NOT delete this function. Otherwise, your rule may not take effect.
    • Regions where SCF is available support ZIP file decompression, including Guangzhou, Shanghai, Beijing, Chengdu, Hong Kong, Singapore, Mumbai, Toronto, and Silicon Valley, and more. For more supported regions, please see SCF Documentation.
    • The directory or file name in your compressed file must be UTF-8 or GB 2312 encoded. Otherwise, the directory or file name might be garbled, or the decompression might be interrupted. If an error is reported, you can click View Log on the right of the function to redirect to the SCF console for viewing the log details.
    • The decompression service is not supported for archived files. To decompress an archived package, please restore it first. For detailed directions, please see Restoring Archived Objects.
    • The maximum processing time for decompressing a single compressed file is 900 seconds, beyond which the decompression task fails. Limits of the COS file decompression feature are subjected to SCF. For more limits, please see SCF Limits.
    • The decompression feature depends on the SCF service, which provides users with a free tier. You will be billed for the part exceeding the free tier according to SCF Product Pricing. Note that the larger your compressed file, the more resources will be used; the more often you decompress your packages, the more calling times will be incurred.

    Directions

    1. Log in to the COS console.

    2. Click Bucket List in the left sidebar.

    3. Click the bucket you want to add a decompression rule for.

    4. Click Function Service > ZIP Decompression Function.

      Note:

      If you haven’t activated SCF, please go to the SCF console to activate it and authorize the service as instructed.

    5. Click Add Function, and configure the following in the pop-up window:

      • Function Name: uniquely identifies a function and cannot be modified after creation. You can view the function in the SCF console.
      • Event Type: an operation that triggers SCF. Take upload as an example. You can initiate an upload by calling the PUT Object” orPOST ObjectAPI. If you choose **Create using PUT method** as the event type, decompression will only be triggered by a package uploaded via thePUT Object` API.
        Note:

        If you intend to upload files to the bucket using multiple ways, such as simple upload, multipart upload, and cross-bucket replication, you are advised to choose File upload as the event type.

      • Trigger Condition: the upload path that will trigger SCF. If you select Specified prefix, SCF will be triggered only when the package is uploaded to a path with the specified prefix. If you choose Not specifying prefix, SCF will be triggered as long as a package is uploaded to any location of the bucket.
        Note:

        If the destination prefix you configured overlaps with the trigger condition, a loop may be triggered, which should be avoided. For example, if the destination prefix is prefix, and the trigger condition ispre, a decompression loop will be triggered when you upload a pref package.

      • SCF Authorization: Required. To decompress a compressed file, SCF should be authorized to read the package from your bucket and upload the decompressed files to the specified location.
    6. Click Next and perform configuration in the pop-up window, as shown below:

      • Decompression Format: the decompression formats you are allowed to use. Currently, only ZIP files are supported.
      • Destination Bucket: a bucket to store the compressed files
      • Destination Path: a path to store the decompressed files of the packages that are matched. To prevent unnecessary fees from triggering the loop, it is recommended that you set a destination path different from the prefix.
      • Extra Prefix:
        • Compressed package name: decompresses the package to a prefix that is named the same as the package itself.
        • Full path of the compressed package: decompresses the package to a prefix that is named the complete path of the package.
        • Empty: decompresses the package directly to the destination path.
      • Forbid Recursive Triggering: Enable does not continue to decompress ZIP packages that are decompressed from the package, while Disable does.
    7. Click Confirm.

      You can perform the following operations on the created function:

      • Click View Logs to view the historical running status of the decompression. If an error is reported, you can click View Logs to quickly redirect to the SCF console for viewing the error log details.
      • Click Edit to modify the file decompression rule.
      • Click Delete to delete the unwanted file decompression rule.