tencent cloud

Feedback

Constraints and Limits

Last updated: 2022-05-24 09:31:13

    Before using EMR, carefully read and understand the following use limits:

    • To ensure the cluster network security, new clusters will be placed in the same VPC. Do not change the VPC of an existing cluster or node; otherwise, the cluster network may fail.
    • When you create a cluster, EMR can help you create a new security group, or you can also manually select an existing EMR security group. Make sure that the selected security group has the necessary inbound/outbound rules for EMR. Do not delete or change the security group in use after creating the cluster; otherwise, cluster communication may fail, thus affecting the service.
    • Plan the storage space of nodes in advance based on your business needs, and promptly add storage nodes, as an insufficient storage space may cause data and node risks. Currently, core, task, and router nodes in the EMR cluster support mounting multiple cloud disks, and a node can have up to 15 cloud disks. Clusters in BM 2.0 and local disk models (IO and D series) don't support mounting multiple cloud disks.
    • When using EMR, do not perform operations in the CVM console, such as shutdown, restart, VPC switch, security group rule adjustment, so as to avoid cluster exceptions. OS reinstallation, instance termination, configuration adjustment, renewal, and billing mode change are also not recommended. You can perform necessary cluster maintenance operations in the EMR console.
    • Public IPs can increase the possibility of master nodes being attacked, so you need to manage and monitor the risks. EIPs (including the IPs on the secondary ENI) will be retained after the cluster is terminated, and the idle IPs will continue to incur fees. If you don't need to retain them, release them on the corresponding resource management page.
    • When you create a cluster, EMR provides component initialization parameters for general scenarios. Before you use components such as HDFS and HBase, we recommend you check whether the component parameters meet the needs of your business scenarios. To get the component initialization guide, contact us.
    • Keep the host login password of your EMR cluster secure. If you configure passwordless login for cross-node access, Tencent Cloud security services may detect vulnerability risks and prompt you.
    • Even if an exception occurs in your cluster, the cluster will continue to be billed. In this case, we recommend you promptly contact us for assistance. If we need to log in to your cluster for troubleshooting, we will gain your consent to request your account and password.

    When you use or maintain your EMR cluster, some unexpected operations may render it unavailable or unstable, and you will receive a risk warning before performing such operations in the console. This document lists some of the prohibited and risky operations:

    Prohibited Operations

    Operation Risk
    Change the private IP of an EMR node in CVM Node communication exception or cluster unavailability
    Modify the security group of a CVM node while the cluster is running Node communication exception or component service unavailability
    Delete an existing process, application, or file from a node Cluster/Component service unavailability
    Delete or modify the `hosts` file in the `/etc` directory Service exception, as the cluster cannot be associated with the service on the node
    Delete or modify the HDFS metadata file `edit log` HDFS cluster unavailability
    Manually modify the data in the Hive metadatabase Service exception caused by a Hive data parsing error
    Delete a ZooKeeper data directory Failure of dependent components
    ## Risky Operations
    Operation Risk Suggestion
    Shut down or restart an EMR cluster node in CVM Service unavailability Check whether the operation is necessary and read the CVM operation limits
    Mount a disk to an EMR node in the CVM console Disk unavailability caused by EMR's failure to recognize and initialize the disk Do this through core node scaling or under technical guidance
    Unmount a disk from an EMR node in the CVM console Data loss or cluster unavailability Do this through core node scaling or under technical guidance
    Modify the component configuration file parameters in CVM Modified parameter overwrite after service restart Modify the parameter configuration in the EMR console and seek technical guidance in special circumstances
    Delete or modify the `resolv.conf` file in the `/etc` directory Service exception, as the cluster cannot be associated with the service on the node Check whether the operation is necessary and perform it under technical guidance
    Modify the MetaDB password EMR depends on the password configured in MetaDB, and after the password is changed, services such as Hive and Ranger will become unavailable Modify the configuration in the EMR console under technical guidance
    Modify the MetaDB floating IP Hive and Ranger service unavailability, as EMR depends on the IP configured in MetaDB Modify the configuration in the EMR console under technical guidance
    Modify the MetaDB security group Interruption of the communication between MetaDB and the cluster, or Hive and Ranger service unavailability Perform the operation under technical guidance
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support