Integrating YARN with Ranger

Last updated: 2021-06-07 17:25:20

    Preparations

    Ranger is available only when it is selected in Optional Components when you purchase a cluster. If you add the Ranger component after purchasing the cluster, the Web UI may be inaccessible. By default, when Ranger is installed, Ranger Admin and Ranger UserSync are deployed on the master node, and Ranger Plugin is deployed on the main daemon node of the embedded component.

    When creating a cluster of the Hadoop type, you can select Ranger in Optional Components. The Ranger version varies depending on the EMR version you choose.

    Note:

    When the cluster type is Hadoop and the Ranger optional component is selected, EMR-Ranger will create services for HDFS and YARN by default and set default policies.

    Ranger Web UI

    Before accessing the Ranger Web UI, make sure that the current cluster is configured with a public IP and click the Ranger Web UI URL on the Cluster Service page.

    After you are redirected, enter the username and password that you set when you purchased the cluster.

    Integrating YARN with Ranger

    Note:

    Make sure that YARN related services are running normally and Ranger has been installed in the current cluster.

    Currently, EMR Ranger YARN only supports ACLs for Capacity Scheduler queues, not for Fair Scheduler queues. Ranger YARN's queue ACLs take effect together with YARN's built-in Capacity Scheduler configuration, but with lower priority. Ranger YARN permissions will be verified only when YARN's built-in Capacity Scheduler configuration denies verification. You are advised to set ACLs via Ranger, instead of in the configuration files.

    1. Add an EMR Ranger YARN service on the EMR Ranger Web UI.
    2. Configure EMR Ranger YARN service parameters.
      Parameter Required Description
      Service Name Yes Service name, which is displayed on the main YARN component on the Ranger Web UI
      description No Service description
      Active Status Default Service status, which is **Enabled** by default
      Username Yes Username of the resource
      Password Yes User password
      NameNode URL Yes YARN URL
      Authorization Enabled Default Select **No** for standard clusters and **Yes** for high-security clusters.
      Authorization Type Yes **Simple**: standard cluster; **Kerberos**: high-security cluster
    3. Configure EMR Ranger YARN resource permissions.
      • Click the configured EMR Ranger YARN service.
      • Configure a policy.

    4. After the policy is added, it will take effect in about 30 seconds, then you can use user1 to submit, kill, or query jobs in the root.default queue of YARN.
    Note:

    When configuring Ranger YARN services and policies, make sure that there are no YARN jobs during this period; otherwise, issues about job submission permissions may occur.