EMR (HDFS)

Last updated: 2020-07-28 16:16:57

    ## Namespace

    Namespace=QCE/TXMR_HDFS

    Monitoring Metrics

    EMR (HDFS) provides the following metrics: HDFS - Overview, HDFS - OverviewAggregation, HDFS - NameNode, HDFS - DataNode, HDFS - Journal Node, and HDFS - ZKFC.

    For more information on the parameters in each dimension, please see Overview of the Parameters in Each Dimension.

    HDFS - Overview

    Parameter Metric Name Unit Description Dimension
    EmrHdfsOverview
    HdfsNnBlockCapacityTotal
    Cluster storage capacity_CapacityTotal GB Total cluster storage capacity host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockCapacityUsed
    Cluster storage capacity_CapacityUsed GB Used cluster storage capacity host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockCapacityRemaining
    Cluster storage capacity_CapacityRemaining GB Remaining cluster storage capacity host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockCapacity
    UsedNonDFS
    Cluster storage capacity_CapacityUsedNonDFS GB Non-HDFS used cluster capacity host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockTotalLoad
    Cluster load_TotalLoad Count Number of current connections host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockFilesTotal
    Total number of cluster files_FilesTotal Count Total number of files host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockBlockstotal
    Number of blocks_BlocksTotal Count Total number of blocks host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockPending
    ReplicationBlocks
    Number of blocks_PendingReplicationBlocks Count Number of blocks waiting to be backed up host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockUnder
    ReplicatedBlocks
    Number of blocks_UnderReplicatedBlocks Count Number of blocks with insufficient replicas host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockBlocksCorruptblocks
    Number of blocks_CorruptBlocks Count Number of corrupted blocks host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockScheduled
    ReplicationBlocks
    Number of blocks_ScheduledReplicationBlocks Count Number of blocks arranged for backup host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockPending
    DeletionBlocks
    Number of blocks_PendingDeletionBlocks Count Number of blocks waiting to be deleted host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockCorruptblocks
    Number of blocks_CorruptBlocks Count Number of excessive blocks host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockPostponed
    MisreplicatedBlocks
    Number of blocks_PostponedMisreplicatedBlocks Count Number of blocks with exceptions that were postponed to be processed host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockBlockCapacity
    Capacity of blocks_BlockCapacity Count Capacity of blocks host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockNumLiveDataNodes
    Cluster DataNodes_NumLiveDataNodes Count Number of live DataNodes host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockNumDeadDataNodes
    Cluster DataNodes_NumDeadDataNodes Count Number of DataNodes marked as dead host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockNum
    DecomLiveDataNodes
    Cluster DataNodes_NumDecomLiveDataNodes Count Number of decommissioned live nodes host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockNum
    DecomDeadDataNodes
    Cluster DataNodes_NumDecomDeadDataNodes Count Number of decommissioned dead nodes host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockNum
    DecommissioningDataNodes
    Cluster DataNodes_NumDecommissioningDataNodes Count Number of decommissioning nodes host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockNum
    StaleDataNodes
    Cluster DataNodes_NumStaleDataNodes Count Number of current DataNodes marked as expired due to heartbeat delay host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockSnapshots
    Snapshots_Snapshots Count Number of snapshots host4hdfsoverview,
    id4hdfsoverview
    EmrHdfsOverview
    HdfsNnBlockVolumeFailuresTotal
    Disk failures_VolumeFailuresTotal Count Total number of failures on all DataNodes host4hdfsoverview,
    id4hdfsoverview

    HDFS - OverviewAggregation

    Parameter Metric Name Unit Description Dimension
    EmrHdfsOverview
    Aggregation
    HdfsNnBlockCapacityTotal
    Cluster storage capacity_CapacityTotal GB Total cluster storage capacity id4hdfsoverview
    EmrHdfsOverview
    Aggregation
    HdfsNnBlockCapacityUsed
    Cluster storage capacity_CapacityUsed GB Used cluster storage capacity id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockCapacityRemaining
    Cluster storage capacity_CapacityRemaining GB Remaining cluster storage capacity id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockCapacity UsedNonDFS
    Cluster storage capacity_CapacityUsedNonDFS GB Non-HDFS used cluster capacity id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockTotalLoad
    Cluster load_TotalLoad Count Number of current connections id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockFilesTotal
    Total number of cluster files_FilesTotal Count Total number of files id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockBlockstotal
    Number of blocks_BlocksTotal Count Total number of blocks id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockPending ReplicationBlocks
    Number of blocks_PendingReplicationBlocks Count Number of blocks waiting to be backed up id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockUnder ReplicatedBlocks
    Number of blocks_UnderReplicatedBlocks Count Number of blocks with insufficient replicas id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockBlocksCorruptblocks
    Number of blocks_CorruptBlocks Count Number of corrupted blocks id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockScheduled ReplicationBlocks
    Number of blocks_ScheduledReplicationBlocks Count Number of blocks arranged for backup id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockPending DeletionBlocks
    Number of blocks_PendingDeletionBlocks Count Number of blocks waiting to be deleted id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockCorruptblocks
    Number of blocks_CorruptBlocks Count Number of excessive blocks id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockPostponed MisreplicatedBlocks
    Number of blocks_PostponedMisreplicatedBlocks Count Number of blocks with exceptions that were postponed to be processed id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockBlockCapacity
    Capacity of blocks_BlockCapacity Count Capacity of blocks id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockNumLiveDataNodes
    Cluster DataNodes_NumLiveDataNodes Count Number of live DataNodes id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockNumDeadDataNodes
    Cluster DataNodes_NumDeadDataNodes Count Number of DataNodes marked as dead id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockNum DecomLiveDataNodes
    Cluster DataNodes_NumDecomLiveDataNodes Count Number of decommissioned live nodes id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockNum DecomDeadDataNodes
    Cluster DataNodes_NumDecomDeadDataNodes Count Number of decommissioned dead nodes id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockNum DecommissioningDataNodes
    Cluster DataNodes_NumDecommissioningDataNodes Count Number of decommissioning nodes id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockNum StaleDataNodes
    Cluster DataNodes_NumStaleDataNodes Count Number of current DataNodes marked as expired due to heartbeat delay id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockSnapshots
    Snapshots_Snapshots Count Number of snapshots id4hdfsoverview
    EmrHdfsOverview
    AggregationHdfsNn
    BlockVolumeFailuresTotal
    Disk failures_VolumeFailuresTotal Count Total number of failures on all DataNodes id4hdfsoverview

    HDFS - NameNode

    Parameter Metric Name Unit Description Dimension
    HdfsNnPort4007RxtxReceivedbytes Data traffic_ReceivedBytes Bytes/s Data receiving rate host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007RxtxSentbytes Data traffic_SentBytes Bytes/s Data sending rate host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007Qps
    Rpcqueuetimenumops
    QPS_RpcQueueTimeNumOps Calls/s RPC call rate host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007RtRpc
    queuetimeavgtime
    Request processing delay
    _RpcQueueTimeAvgTime
    ms Average RPC delay host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007AuthRpc
    authenticationfailures
    Authentication and authorization
    _RpcAuthenticationFailure
    Count Number of RPC authentication failures host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007AuthRpc
    authenticationsuccesses
    Authentication and authorization
    _RpcAuthenticationSuccesses
    Count Number of RPC authentication successes host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007AuthRpc
    authorizationfailures
    Authentication and authorization
    _RpcAuthorizationFailures
    Count Number of RPC authorization failures host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007AuthRpc
    authorizationsuccesses
    Authentication and authorization
    _RpcAuthorizationSuccesses
    Count Number of RPC authorization successes host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007Connections
    Numopenconnections
    Number of current connections
    _NumOpenConnections
    Count Number of current connections host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPort4007Queue
    LenCallqueuelength
    RPC processing queue length
    _CallQueueLength
    Count Length of current RPC processing queue host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmMemMemnonheapusedm JVM memory
    _MemNonHeapUsedM
    MB Size of the NonHeapMemory currently used by JVM host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmMemMemnon
    heapcommittedm
    JVM memory
    _MemNonHeapCommittedM
    MB JVM memory host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmMemMemheapusedm JVM memory
    _MemHeapUsedM
    MB Size of the HeapMemory currently used by JVM host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmMem
    Memheapcommittedm
    JVM memory
    _MemHeapCommittedM
    MB Size of the HeapMemory committed by JVM host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmMemMemheapmaxm JVM memory_MemHeapMaxM MB Size of the HeapMemory configured by JVM host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmMemMemmaxm JVM memory_MemMaxM MB Maximum memory size that can be used by JVM during runtime host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnBlockReportRt
    Blockreportavgtime
    Block reporting delay
    _BlockReportAvgTime
    Blocks/s Average delay in processing DataNode blocks per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilGcCountFgc GC count_FGC Operations/s Full GC count host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilGcCountYgc GC count_YGC 2 operations/s Young GC count host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilGcTimeYgct GC time_YGCT ms Time consumed by Young GC host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilGcTimeFgct GC time_FGCT ms Time consumed by Full GC host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilGcTimeGct GC time_GCT ms Time used to collect garbage host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmJavaThreadsThreadsnew Number of JVM threads
    _ThreadsNew
    Count Number of threads in the new state host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmJavaThreads
    Threadsrunnable
    Number of JVM threads
    _ThreadsRunnable
    Count Number of threads in the runnable state host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmJavaThreads
    Threadsblocked
    Number of JVM threads
    _ThreadsBlocked
    Count Number of threads in the blocked state host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmJavaThreads
    Threadswaiting
    Number of JVM threads
    _ThreadsWaiting
    Count Number of threads in the WAITING state host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmJavaThreads
    Threadstimedwaiting
    Number of JVM threads
    _ThreadsTimedWaiting
    Count Number of threads in the TIMED WAITING state host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmJavaThreads
    Threadsterminated
    Number of JVM threads
    _ThreadsTerminated
    Count Number of threads in the Terminated state host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmLogTotalLogfatal Number of JVM logs_LogFatal Count Number of Fatal logs host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmLogTotalLogerror Number of JVM logs_LogError Count Number of Error logs host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmLogTotalLogwarn Number of JVM logs_LogWarn Count Number of Warn logs host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnJvmLogTotalLoginfo Number of JVM logs_LogInfo Count Number of Info logs host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilMemoryS0 Memory space percentage_S0 % Percentage of used Survivor 0 memory host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilMemoryS1 Memory space percentage_S1 % Percentage of used Survivor 1 memory host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilMemoryE Memory space percentage_E % Percentage of used Eden memory host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilMemoryO Memory space percentage_O % Percentage of used Old memory host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilMemoryM Memory space percentage_M % Percentage of used Metaspace memory host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnGcUtilMemoryCcs Memory space percentage_CCS % Percentage of memory used by compressed class space host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnStaleStorages
    CountNumstalestorages
    Number of storages marked as expired
    _NumStaleStorages
    Count Number of current DataNodes marked as expired due to heartbeat delay host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnPendingDatanodeMessage
    CountPendingdatanode
    messagecount
    Number of pending block operation messages on the standby NameNode
    _PendingDataNode
    MessageCount
    Requests/s Number of DataNode requests queued on the standby NameNode host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnBlocksMissingNum
    berofmissingblocks
    Total number of missing blocks
    _NumberOfMissingBlocks
    Count Number of missing data blocks host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnBlocksMissingNumberof
    missingblockswithreplication
    factorOne
    Total number of missing blocks_NumberOf
    MissingBlocksWithReplication
    FactorOne
    Count Number of missing blocks (rf = 1) host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOpsAllowsnapshotops Snapshot operations
    _AllowSnapshotOps
    Operations/s Number of AllowSnapshot operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOps
    Disallowsnapshotops
    Snapshot operations
    _DisallowSnapshotOps
    Operations/s Number of DisallowSnapshot operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOps
    Createsnapshotops
    Snapshot operations
    _CreateSnapshotOps
    Operations/s Number of CreateSnapshot operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOps
    Deletesnapshotops
    Snapshot operations
    _DeleteSnapshotOps
    Operations/s Number of DeleteSnapshot operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOps
    Listsnapshottabledirops
    Snapshot operations_ListSnapshottableDirOps Operations/s Number of ListSnapshottableDir operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOps
    Snapshotdiffreportops
    Snapshot operations_SnapshotDiffReportOps Operations/s Number of SnapshotDiffReportOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSnapshotOps
    Renamesnapshotops
    Snapshot operations_RenameSnapshotOps Operations/s Number of RenameSnapshotOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsCreatefileops File operations_CreateFileOps Operations/s Number of CreateFile operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsGetlistingops File operations_GetListingOps Operations/s Number of GetListing operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsTotalfileops File operations_TotalFileOps Operations/s Number of TotalFileOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsDeletefileops File operations_DeleteFileOps Operations/s Number of DeleteFile operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsFileinfoops File operations_FileInfoOps Operations/s Number of FileInfo operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsGetadditional
    datanodeops
    File operations
    _GetAdditionalDatanodeOps
    Operations/s Number of GetAdditionalDatanode operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsCreatesymlinkops File operations_CreateSymlinkOps Operations/s Number of CreateSymlink operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsGetlinktargetops File operations_GetLinkTargetOps Operations/s Number of GetLinkTarget operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnFilesOpsFilesingetlistingops File operations
    _FilesInGetListingOps
    Operations/s Number of FilesInGetListing operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnTransactionOps
    Transactionsnumops
    Transaction operations
    _TransactionsNumOps
    Operations/s Number of Journal transaction operations processed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnTransactionOps
    Transactionsbatchedinsync
    Transaction operations
    _TransactionsBatchedInSync
    Operations/s Number of Journal transaction operations processed in batches per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnImageOpsGeteditnumops Image operations_GetEditNumOps Operations/s Number of GetEditNumOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnImageOpsGetimagenumops Image operations_GetImageNumOps Operations/s Number of GetImageNumOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnImageOpsPutimagenumops Image operations_PutImageNumOps Operations/s Number of PutImageNumOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSyncsOpsSyncsnumops Sync operations_SyncsNumOps Operations/s Number of Journal syncs operations processed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnBlocksOpsBlock
    receivedanddeletedops
    Data block operations
    _BlockOpsQueued
    Operations/s Number of BlockReceivedAndDeletedOps operations executed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnBlocksOpsBlockopsqueued Data block operations
    _BlockOpsQueued
    Operations/s Delay in processing DataNode block reporting operations host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnCacheReportOps
    Cachereportnumops
    Cache reporting
    _CacheReportNumOps
    Operations/s Number of CacheReport operations processed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnBlockReportOps
    Blockreportnumops
    Block reporting
    _BlockReportNumOps
    Operations/s Number of DataNode block reporting operations processed per second host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnSyncsRtSyncsavgtime Sync operation delay_SyncsAvgTime ms Average delay in processing Journal syncs operations host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnCacheReportRt
    Cachereportavgtime
    Cache reporting delay_CacheReportAvgTime ms Average delay in cache reporting operations host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnImageRtGeteditavgtime Image operation delay
    _GetEditAvgTime
    ms Average delay in reading the Edit file host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnImageRtGetimageavgtime Image operation delay
    _GetImageAvgTime
    ms Average delay in reading the image file host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnImageRtPutimageavgtime Image operation delay
    _PutImageAvgTime
    ms Average delay in writing the image file host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnTransactionRt
    Transactionsavgtime
    Transaction operation delay
    _TransactionsAvgTime
    ms Average delay in processing Journal transaction operations host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnStartTimeStarttime Start time_StartTime ms Process start time host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnStateState Primary/secondary status_State - NN status host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnThreadCountPeakthreadcount Number of threads_PeakThreadCount Count Peak number of threads host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnThreadCountThreadcount Number of threads_ThreadCount Count Number of threads host4hdfsnamenode,
    id4hdfsnamenode
    HdfsNnThreadCount
    Daemonthreadcount
    Number of threads_DaemonThreadCount Count Number of background threads host4hdfsnamenode,
    id4hdfsnamenode

    HDFS - DataNode

    Parameter Metric Name Unit Description Dimension
    HdfsDnXceiverXceivercount Number of Xceivers_XceiverCount Xceivers Number of Xceivers id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBytesByteswrittenmb Data read/write rate_BytesReadMB Bytes/s DN byte write rate id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBytesBytesreadmb Data read/write rate_BytesReadMB Bytes/s DN byte read rate id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBytesRemotebytesreadmb Data read/write rate
    _RemoteBytesReadMB
    Bytes/s Rate of bytes read by the remote client id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBytesRemoteby
    teswrittenmb
    Data read/write rate
    _RemoteBytesWrittenMB
    Bytes/s Rate of bytes written by the remote client id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnClientWritesfrom
    remoteclient
    Number of client connections
    _WritesFromRemoteClient
    Count QPS of write operations from the remote client id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnClientWritesfromlocalclient Number of client connections
    _WritesFromLocalClient
    Count OPS of write operations from the local client id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnClientReadsfrom
    remoteclient
    Number of client connections
    _ReadsFromRemoteClient
    Count QPS of read operations from the remote client id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnClientReadsfromlocalclient Number of client connections
    _ReadsFromLocalClient
    Count QPS of read operations from the local client id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksVerifiedFailures
    Blockverificationfailures
    Block check failure
    _BlockVerificationFailures
    Failures/s Number of block check failures id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnVolumeFailures
    Volumefailures
    Disk failures_VolumeFailures Failures/s Number of disk failures id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnNetworkErrors
    Datanodenetworkerrors
    Network errors
    _DatanodeNetworkErrors
    Errors/s Total number of network errors id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnHbRtHeartbeatsavgtime Heartbeat delay_HeartbeatsAvgTime ms Average heartbeat API time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnHbOpsHeartbeatsnumops Heartbeat QPS_HeartbeatsNumOps Heartbeats/s Heartbeat API QPS id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnDatapacketAvgtimeSend
    datapackettransfer
    nanosavgtime
    Packet transfer operations
    QPS_SendDataPacketTransfer
    NanosAvgTime
    ms Average data packet sending time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsRead
    blockopnumops
    Data block operations
    _ReadBlockOpNumOps
    Operations/s OPS of block reads from the DataNode id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsWrite
    blockopnumops
    Data block operations
    _WriteBlockOpNumOps
    Operations/s OPS of block writes to the DataNode id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsBlock
    checksumopnumops
    Data block operations
    _BlockChecksumOpNumOps
    Operations/s OPS of Checksum operations by the DataNode id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsCopy
    blockopnumops
    Data block operations
    _CopyBlockOpNumOps
    Operations/s OPS of block copying operations id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsReplace
    blockopnumops
    Data block operations
    _ReplaceBlockOpNumOps
    Operations/s OPS of Replace Block operations id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsBlock
    reportsnumops
    Data block operations
    _BlockReportsNumOps
    Operations/s OPS of block reporting operations id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsIncremental
    blockreports
    numops
    Data block operations
    _IncrementalBlockReports
    NumOps
    Operations/s OPS of incremental block reporting id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsCache
    reportsnumops
    Data block operations
    _CacheReportsNumOps
    Operations/s OPS of cache reporting id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksOpsPacketack
    roundtriptimenanos
    numops
    Data block operations
    _PacketAckRoundTripTimeNanos
    NumOps
    Operations/s Number of ACK ROUND TRIP operations processed per second id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnFsyncOpsFsync
    nanosnumops
    Fsync operations
    _FsyncNanosNumOps
    Operations/s Number of Fsync operations id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnFlushOpsFlush
    nanosnumops
    Flush operations
    _FlushNanosNumOps
    Operations/s Number of Flush operations processed per second id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtRead
    blockopavgtime
    Data block operation delay
    _ReadBlockOpAvgTime
    ms Average block read time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtWrite
    blockopavgtime
    Data block operation delay
    _ReplaceBlockOpAvgTime
    ms Average block write time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtBlock
    checksumopavgtime
    Data block operation delay_BlockChecksumOpAvgTime ms Average block check time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtCopy
    blockopavgtime
    Data block operation delay
    _CopyBlockOpAvgTime
    ms Average block copy time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRt
    Replaceblockopavgtime
    Data block operation delay
    _Replaceblockopavgtime
    ms Average Replace Block operation time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtBlock
    reportsavgtime
    Data block operation delay
    _BlockReportsAvgTime
    ms Average block reporting time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtIncremental
    blockreportsavgtime
    Data block operation delay
    _IncrementalBlockReportsAvgTime
    ms Average time of incremental block reporting id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtCache
    reportsavgtime
    Data block operation delay
    _CacheReportsAvgTime
    ms Average time of cache reporting id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnBlocksRtPacketack
    roundtriptimenanos
    avgtime
    Data block operation delay
    _PacketAckRoundTripTimeNanos
    AvgTime
    ms Average time of processing ACK ROUND TRIP id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnFlushRtFlushnanosavgtime Flush delay_FlushNanosAvgTime ms Average Flush operation time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnFsyncRtFsyncnanosavgtime Fsync delay_FsyncNanosAvgTime ms Average Fsync operation time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOp
    Ramdiskblockswrite
    RAMDISKBlocks_Ram
    DiskBlocksWrite
    Blocks/s Total number of blocks written to the memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOp
    Ramdiskblockswritefallback
    RAMDISKBlocks_Ram
    DiskBlocksWriteFallback
    Blocks/s Total number of blocks failed to be written to the memory (failover to the disk) id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOpRamdisk
    blocksdeletedbeforelazypersisted
    RAMDISKBlocks_RamDiskBlocks
    DeletedBeforeLazyPersisted
    Blocks/s Total number of blocks deleted before the application is saved to the disk id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOp
    Ramdiskblocksreadhits
    RAMDISKBlocks_Ram
    DiskBlocksReadHits
    Blocks/s Total number of reads from the blocks in the memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOp
    Ramdiskblocksevicted
    RAMDISKBlocks_Ram
    DiskBlocksEvicted
    Blocks/s Total number of blocks cleared in the memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOpRamdisk
    blocksevictedwithoutread
    RAMDISKBlocks_RamDiskBlocks
    EvictedWithoutRead
    Blocks/s Total number of blocks retrieved from the memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOp
    Ramdiskblockslazypersisted
    RAMDISKBlocks_RamDisk
    BlocksLazyPersisted
    Blocks/s Number of disk writes by the lazy writer id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksOp
    Ramdiskbyteslazypersisted
    RAMDISKBlocks_Ram
    DiskBytesLazyPersisted
    Bytes/s Total number of bytes written to the disk by the lazy writer id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRamBlocksBytes
    Ramdiskbyteswrite
    RAM Disk write speed_RamDiskBytesWrite Bytes/s Total number of bytes written to the memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmMem
    Memnonheapusedm
    JVM memory
    _MemNonHeapUsedM
    MB Size of the NonHeapMemory currently used by JVM id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmMem
    Memnonheapcommittedm
    JVM memory
    _MemNonHeapCommittedM
    MB Size of the NonHeapCommittedM configured by JVM id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmMemMemheapusedm JVM memory
    _MemHeapUsedM
    MB Size of the HeapMemory currently used by JVM id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmMem
    Memheapcommittedm
    JVM memory
    _MemHeapCommittedM
    MB Size of the HeapMemory committed by JVM id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmMemMemheapmaxm JVM memory
    _MemHeapMaxM
    MB Size of the HeapMemory configured by JVM id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmMemMemmaxm JVM memory
    _MemMaxM
    MB Maximum memory size that can be used by JVM during runtime id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmJavaThreadsThreadsnew Number of JVM threads
    _ThreadsNew
    Count Number of threads in the new state id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmJavaThreads
    Threadsrunnable
    Number of JVM threads
    _ThreadsRunnable
    Count Number of threads in the runnable state id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmJavaThreads
    Threadsblocked
    Number of JVM threads
    _ThreadsBlocked
    Count Number of threads in the blocked state id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmJavaThreads
    Threadswaiting
    Number of JVM threads
    _ThreadsWaiting
    Count Number of threads in the WAITING state id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmJavaThreads
    Threadstimedwaiting
    Number of JVM threads
    _ThreadsTimedWaiting
    Count Number of threads in the TIMED WAITING state id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmJavaThreads
    Threadsterminated
    Number of JVM threads
    _ThreadsTerminated
    Count Number of threads in the Terminated state id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmLogTotalLogfatal Number of JVM logs_LogFatal Count Number of Fatal logs id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmLogTotalLogerror Number of JVM logs_LogError Count Number of Error logs id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmLogTotalLogwarn Number of JVM logs_LogWarn Count Number of Warn logs id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnJvmLogTotalLoginfo Number of JVM logs_LogInfo Count Number of Info logs id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilMemoryS0 Memory space percentage_S0 % Percentage of used Survivor 0 memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilMemoryS1 Memory space percentage_S1 % Percentage of used Survivor 1 memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilMemoryE Memory space percentage_E % Percentage of used Eden memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilMemoryO Memory space percentage_O % Percentage of used Old memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilMemoryM Memory space percentage_M % Percentage of used Metaspace memory id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilMemoryCcs Memory space percentage_CCS % Percentage of memory used by compressed class space id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilGcCountFgc GC count_FGC Count Full GC count id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilGcCountYgc GC count_YGC Count Young GC count id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilGcTimeYgct GC time_YGCT s Time consumed by Young GC id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilGcTimeFgct GC time_FGCT s Time consumed by Full GC id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnGcUtilGcTimeGct GC time_GCT s Time used to collect garbage id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004RxtxReceivedbytes Data traffic_ReceivedBytes Bytes/s Data receiving rate id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004RxtxSentbytes Data traffic_SentBytes Bytes/s Data sending rate id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004QpsRpc
    queuetimenumops
    QPS_RpcQueueTimeNumOps Queries/s RPC call rate id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004RtRpc
    queuetimeavgtime
    Request processing delay
    _RpcQueueTimeAvgTime
    ms Average RPC delay id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004AuthRpc
    authenticationfailures
    Authentication and authorization
    _RpcAuthenticationFailures
    Failures/s Number of RPC authentication failures id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004AuthRpc
    authenticationsuccesses
    Authentication and authorization
    _RpcAuthenticationSuccesses
    Successes/s Number of RPC authentication successes id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004AuthRpc
    authorizationfailures
    Authentication and authorization
    _RpcAuthorizationFailures
    Failures/s Number of RPC authorization failures id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004AuthRpc
    authorizationsuccesses
    Authentication and authorization
    _RpcAuthorizationSuccesses
    Successes/s Number RPC authorization successes id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004Connections
    Numopenconnections
    Number of current connections
    _NumOpenConnections
    Count Number of current connections id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnPort4004QueueLen
    Callqueuelength
    RPC processing queue length
    _CallQueueLength
    Count Length of the current RPC processing queue id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnThreadTimeCurrent
    threadcputime
    CPU time
    _CurrentThreadCpuTime
    ms CPU time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnThreadTimeCurrent
    threadusertime
    CPU time
    _CurrentThreadUserTime
    ms User time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnStartTimeStarttime Start time_StartTime s Process start time id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnThreadCount
    Peakthreadcount
    Number of threads_PeakThreadCount Count Peak number of threads id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnThreadCount
    Daemonthreadcount
    Number of threads_DaemonThreadCount Count Number of background threads id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRtWrit Read/write delay_Write MB/s Disk write rate id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnRtRead Read/write delay_Read Queries/s Read QPS id4hdfsdatanode,
    host4hdfsdatanode
    HdfsDnDatapacketOps
    Datapacketops
    Packet transfer QPS_DataPacketOps Queries/s Packet transfer QPS id4hdfsdatanode,
    host4hdfsdatanode

    HDFS - Journal Node

    Parameter Metric Name Unit Description Dimension
    HdfsJnJvmMemMemnon
    heapusedm
    JVM memory_MemNonHeapUsedM MB Size of the NonHeapMemory currently used by JVM host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmMemMemnon
    heapcommittedm
    JVM memory_MemNonHeapCommittedM MB JVM memory host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmMemMem
    heapusedm
    JVM memory_MemHeapUsedM MB Size of the HeapMemory currently used by JVM host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmMemMem
    heapcommittedm
    JVM memory_MemHeapCommittedM MB Size of the HeapMemory committed by JVM host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmMemMem
    heapmaxm
    JVM memory_MemHeapMaxM MB Size of the HeapMemory configured by JVM host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmMemMemmaxm JVM memory_MemMaxM MB Maximum memory size that can be used by JVM during runtime host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmJavaThreads
    Threadsnew
    Number of JVM threads_ThreadsNew Count Number of threads in the new state host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmJavaThreads
    Threadsrunnable
    Number of JVM threads_ThreadsRunnable Count Number of threads in the runnable state host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmJavaThreads
    Threadsblocked
    Number of JVM threads_ThreadsBlocked Count Number of threads in the blocked state host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmJavaThreads
    Threadswaiting
    Number of JVM threads_ThreadsWaiting Count Number of threads in the WAITING state host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmJavaThreads
    Threadstimedwaiting
    Number of JVM threads_ThreadsTimedWaiting Count Number of threads in the TIMED WAITING state host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmJavaThreads
    Threadsterminated
    Number of JVM threads_ThreadsTerminated Count Number of threads in the Terminated state host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmLogTotalLogfatal Number of JVM logs_LogFatal Count Number of Fatal logs host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmLogTotalLogerror Number of JVM logs_LogError Count Number of Error logs host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmLogTotalLogwarn Number of JVM logs_LogWarn Count Number of Warn logs host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnJvmLogTotalLoginfo Number of JVM logs_LogInfo Count Number of Info logs host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilMemoryS0 Memory space percentage_S0 % Percentage of used Survivor 0 memory host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilMemoryS1 Memory space percentage_S1 % Percentage of used Survivor 1 memory host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilMemoryE Memory space percentage_E % Percentage of used Eden memory host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilMemoryO Memory space percentage_O % Percentage of used Old memory host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilMemoryM Memory space percentage_M % Percentage of used Metaspace memory host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilMemoryCcs Memory space percentage_CCS % Percentage of memory used by compressed class space host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilGcCountFgc GC count_FGC Count Full GC count host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilGcCountYgc GC count_YGC Count Young GC count host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilGcTimeYgct GC time_YGCT s Time consumed by Young GC host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilGcTimeFgct GC time_FGCT s Time consumed by Full GC host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnGcUtilGcTimeGct GC time_GC s Time used to collect garbage host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005Rxtx
    Receivedbytes
    Data traffic_ReceivedBytes Bytes/s Data receiving rate host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005Rxtx
    Receivedbytes
    Data traffic_ReceivedBytes Bytes/s Data sending rate host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005QpsRpc
    queuetimenumops
    QPS_RpcQueueTimeNumOps Queries/s RPC call rate host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005RtRpc
    queuetimeavgtime
    Request processing delay_RpcQueueTimeAvgTime ms Average RPC delay host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005AuthRpc
    authenticationfailures
    Authentication and authorization_RpcAuthenticationFailures Failures/s Number of RPC authentication failures host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005AuthRpc
    authorizationsuccesses
    Authentication and authorization_RpcAuthorizationSuccesses Successes/s Number of RPC authorization successes host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005AuthRpc
    authenticationsuccesses
    Authentication and authorization_RpcAuthenticationSuccesses Successes/s Number of RPC authentication successes host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005AuthRpc
    authorizationfailures
    Authentication and authorization_RpcAuthorizationFailures Failures/s Number of RPC authorization failures host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005Connections
    Numopenconnections
    Number of current connections_NumOpenConnections Count Number of current connections host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnPort4005QueueLen
    Callqueuelength
    RPC processing queue length_CallQueueLength Count Length of the current RPC processing queue host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnThreadTimeCurrent
    threadcputime
    CPU time_CurrentThreadCpuTime ms CPU time host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnThreadTimeCurrent
    threadusertime
    CPU time_CurrentThreadUserTime ms User time host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnStartTimeStarttime Start time_StartTime s Process start time host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnThreadCount
    Threadcount
    Number of threads_ThreadCount Count Number of threads host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnThreadCount
    Peakthreadcount
    Number of threads_PeakThreadCount Count Peak number of threads host4hdfsjournalnode,
    id4hdfsjournalnode
    HdfsJnThreadCountDaemon
    threadcount
    Number of threads_DaemonThreadCount Count Number of background threads host4hdfsjournalnode,
    id4hdfsjournalnode

    HDFS - ZKFC

    Parameter Metric Name Unit Description Dimension
    HdfsDfzkGcUtilMemoryS0 Memory space percentage_S0 % Percentage of used Survivor 0 memory host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilMemoryS1 Memory space percentage_S1 % Percentage of used Survivor 1 memory host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilMemoryE Memory space percentage_E % Percentage of used Eden memory host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilMemoryO Memory space percentage_O % Percentage of used Old memory host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilMemoryM Memory space percentage_M % Percentage of used Metaspace memory host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilMemoryCcs Memory space percentage_CCS % Percentage of memory used by compressed class space host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilGcCountFgc GC count_FGC Count Full GC count host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilGcCountYgc GC count_YGC Count Young GC count host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilGcTimeYgct GC time_YGCT s Time consumed by Young GC host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilGcTimeFgct GC time_FGCT s Time consumed by Full GC host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller
    HdfsDfzkGcUtilGcTimeGct GC time_GCT s Time used to collect garbage host4hdfszkfailovercontroller,
    id4hdfszkfailovercontroller

    The statistical granularity (period) may vary by metric. The DescribeBaseMetrics API can be used to obtain the period supported by each metric.

    Overview of the Parameters in Each Dimension

    Parameter Name Dimension Name Dimension Description Format
    Instances.N.Dimensions.0.Name id4hdfsoverview Dimension name of the EMR instance ID String-type dimension name, such as id4hdfsoverview
    Instances.N.Dimensions.0.Value id4hdfsoverview Specific EMR instance ID Specific EMR instance ID, such as emr-mm8bs222
    Instances.N.Dimensions.1.Name host4hdfsoverview Dimension name of the node IP in the EMR instance String-type dimension name, such as host4hdfsoverview
    Instances.N.Dimensions.1.Value host4hdfsoverview Specific node IP in the EMR instance Specific node IP, such as 1.1.1.1
    Instances.N.Dimensions.0.Name id4hdfsnamenode Dimension name of the EMR instance ID String-type dimension name, such as id4hdfsnamenode
    Instances.N.Dimensions.0.Value id4hdfsnamenode Specific EMR instance ID Specific EMR instance ID, such as emr-mm8bs222
    Instances.N.Dimensions.1.Name host4hdfsnamenode Dimension name of the node IP in the EMR instance String-type dimension name, such as host4hdfsnamenode
    Instances.N.Dimensions.1.Value host4hdfsnamenode Specific node IP in the EMR instance Specific node IP, such as 1.1.1.1
    Instances.N.Dimensions.0.Name id4hdfsdatanode Dimension name of the EMR instance ID String-type dimension name, such as id4hdfsdatanode
    Instances.N.Dimensions.0.Value id4hdfsdatanode Specific EMR instance ID Specific EMR instance ID, such as emr-mm8bs222
    Instances.N.Dimensions.1.Name host4hdfsdatanode Dimension name of the node IP in the EMR instance String-type dimension name, such as host4hdfsdatanode
    Instances.N.Dimensions.1.Value host4hdfsdatanode Specific node IP in the EMR instance Specific node IP, such as 1.1.1.1
    Instances.N.Dimensions.0.Name id4hdfsjournalnode Dimension name of the EMR instance ID String-type dimension name, such as id4hdfsjournalnode
    Instances.N.Dimensions.0.Value id4hdfsjournalnode Specific EMR instance ID Specific EMR instance ID, such as emr-mm8bs222
    Instances.N.Dimensions.1.Name host4hdfsjournalnode Dimension name of the node IP in the EMR instance String-type dimension name, such as host4hdfsjournalnode
    Instances.N.Dimensions.1.Value host4hdfsjournalnode Specific node IP in the EMR instance Specific node IP, such as 1.1.1.1
    Instances.N.Dimensions.0.Name id4hdfszkfailovercontroller Dimension name of the EMR instance ID String-type dimension name, such as id4hdfszkfailovercontroller
    Instances.N.Dimensions.0.Value id4hdfszkfailovercontroller Specific EMR instance ID Specific EMR instance ID, such as emr-mm8bs222
    Instances.N.Dimensions.1.Name host4hdfszkfailovercontroller Dimension name of the node IP in the EMR instance String-type dimension name, such as host4hdfszkfailovercontroller
    Instances.N.Dimensions.1.Value host4hdfszkfailovercontroller Specific node IP in the EMR instance Specific node IP, such as 1.1.1.1

    Input Parameters

    EMR (HDFS) supports querying monitoring data based on the following six combinations of dimensions. The values for the input parameters are as follows:

    1. To query the metric monitoring data of HDFS - OverviewAggregation, use the following input parameters:
    &Namespace=QCE/TXMR_HDFS
    &Instances.N.Dimensions.0.Name=id4hdfsoverview
    &Instances.N.Dimensions.0.Value=EMR instance ID

    2. To query the metric monitoring data of HDFS - Overview, use the following input parameters:
    &Namespace=QCE/TXMR_HDFS
    &Instances.N.Dimensions.0.Name=id4hdfsoverview
    &Instances.N.Dimensions.0.Value=EMR instance ID
    &Instances.N.Dimensions.1.Name=host4hdfsoverview
    &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance

    3. To query the metric monitoring data of HDFS - NameNode, use the following input parameters:
    &Namespace=QCE/TXMR_HDFS
    &Instances.N.Dimensions.0.Name=id4hdfsnamenode
    &Instances.N.Dimensions.0.Value=EMR instance ID
    &Instances.N.Dimensions.1.Name=host4hdfsnamenode
    &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance

    4. To query the metric monitoring data of HDFS - DataNode, use the following input parameters:
    &Namespace=QCE/TXMR_HDFS
    &Instances.N.Dimensions.0.Name=id4hdfsdatanode
    &Instances.N.Dimensions.0.Value=EMR instance ID
    &Instances.N.Dimensions.1.Name=host4hdfsdatanode
    &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance

    5. To query the metric monitoring data of HDFS - Journal Node, use the following input parameters:
    &Namespace=QCE/TXMR_HDFS
    &Instances.N.Dimensions.0.Name=id4hdfsjournalnode
    &Instances.N.Dimensions.0.Value=Specific EMR instance ID
    &Instances.N.Dimensions.1.Name=host4hdfsjournalnode
    &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance

    6. To query the metric monitoring data of HDFS - ZKFC, use the following input parameters:
    &Namespace=QCE/TXMR_HDFS
    &Instances.N.Dimensions.0.Name=id4hdfszkfailovercontroller
    &Instances.N.Dimensions.0.Value=Specific EMR instance ID
    &Instances.N.Dimensions.1.Name=host4hdfszkfailovercontroller
    &Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help