Managed storage refers to the storage space hosted on the Data Lake product, with COS as the underlying storage. Managed storage contains data such as native tables, user program packages, and query results. Therefore, to utilize the capabilities of native tables and data optimization, it is necessary to enable managed storage first. The native tables on managed storage are by default in the Iceberg format, so you don’t need to manage the underlying file contents. For details on managed storage billing, please refer to Billing Overview. Note:
In addition to billing for user data usage, the managed bucket also incurs minimal fees for requests from daily storage routine inspections (including user storage activity level and amount inspections) performed by Data Lake Compute (DLC). To avoid fees from these routine inspections, promptly terminate any managed bucket that is no longer in use.
This document introduces how to enable and configure managed storage.
Enable Managed Storage
Step 1: Enter Managed Storage Configuration
Step 2: Creating a Managed Bucket
1. Click Create Managed Bucket to go to the creation page, select the managed bucket type, and click Confirm to complete the creation.
Here, you can specify the managed bucket type as a metadata acceleration bucket or a regular bucket. Metadata acceleration buckets and regular buckets have the same billing method. However, Metadata acceleration buckets require additional configurations to grant engine access permissions. For details, refer to Binding of the Metadata Acceleration Bucket. 2. The storage path for query results can be modified above. The path is used to temporarily store SQL query result data, Spark job shuffle data, and other related data. You need to specify a path to ensure the normal operation of jobs and tasks. If you have created a managed bucket, it is recommended to configure the query result path as Managed Bucket. You can also configure the query result path to a COS Bucket path under your account. View managed bucket
After enabling managed storage, a bucket will be created, and you can view the buckets and data on managed storage in the Storage Configuration module. Destroy Managed Storage
Destroying data is a high-risk action; only after all database table data has been deleted, can you proceed to destroy managed storage. Destroying managed storage requires administrator privileges.
Step 1: Delete database table data
To destroy managed storage, you must first delete all database table data on the managed storage.
Step 2: Destroy Managed Storage
After deleting the database table data, you can destroy managed storage on the managed storage configuration tab under the Storage Configuration module. Destroying managed storage will delete all DLC managed buckets, so please proceed with caution.