YARN Task Queue Management

Last updated: 2019-07-26 17:44:40

PDF

You can log in to the YARNwebUI, which is the web user interface of YARN, via the shortcut entry provided in EMR. For more information, see the product documentation. After logging in, you will see the following:

YARN

You can see some monitoring information of the entire cluster here:

  • Apps: 67 submitted apps, including 9 pending ones, 7 running ones, and 51 completed ones, where 15 containers are running.
  • Memory: 36 GB in total with 30 GB used and 10 GB reserved.
  • Virtual cores (vCores): 20 vCores in total with 15 used ones and 5 reserved ones.
  • Nodes: 5 available nodes, 0 decommissioned ones, 0 lost ones, 0 unhealthy ones, and 0 rebooted ones.
  • Scheduler: Fair Scheduler, including maximum and minimum memory size and CPU allocation information.

The Application Queues section contains task queuing information of the cluster. Take the root.hadoop queue as an example:

  • Used memory: 30,720 MB (30 GB), 15 vCores
  • Active applications: 7
  • Pending applications: 9
  • Minimum memory occupied: 0 MB, 0 cores
  • Maximum memory occupied: 36,860 MB, 20 cores
  • Steady fair share
  • Instantaneous fair share
  • The proportion of memory occupied by this queue is 83.3%