## Namespace
Namespace=QCE/TXMR_HDFS
EMR (HDFS) provides the following metrics: HDFS - Overview, HDFS - OverviewAggregation, HDFS - NameNode, HDFS - DataNode, HDFS - Journal Node, and HDFS - ZKFC.
For more information on the parameters in each dimension, please see Overview of the Parameters in Each Dimension.
Parameter | Metric Name | Unit | Description | Dimension |
---|---|---|---|---|
EmrHdfsOverview HdfsNnBlockCapacityTotal |
Cluster storage capacity_CapacityTotal | GB | Total cluster storage capacity | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockCapacityUsed |
Cluster storage capacity_CapacityUsed | GB | Used cluster storage capacity | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockCapacityRemaining |
Cluster storage capacity_CapacityRemaining | GB | Remaining cluster storage capacity | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockCapacity UsedNonDFS |
Cluster storage capacity_CapacityUsedNonDFS | GB | Non-HDFS used cluster capacity | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockTotalLoad |
Cluster load_TotalLoad | Count | Number of current connections | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockFilesTotal |
Total number of cluster files_FilesTotal | Count | Total number of files | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockBlockstotal |
Number of blocks_BlocksTotal | Count | Total number of blocks | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockPending ReplicationBlocks |
Number of blocks_PendingReplicationBlocks | Count | Number of blocks waiting to be backed up | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockUnder ReplicatedBlocks |
Number of blocks_UnderReplicatedBlocks | Count | Number of blocks with insufficient replicas | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockBlocksCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of corrupted blocks | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockScheduled ReplicationBlocks |
Number of blocks_ScheduledReplicationBlocks | Count | Number of blocks arranged for backup | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockPending DeletionBlocks |
Number of blocks_PendingDeletionBlocks | Count | Number of blocks waiting to be deleted | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of excessive blocks | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockPostponed MisreplicatedBlocks |
Number of blocks_PostponedMisreplicatedBlocks | Count | Number of blocks with exceptions that were postponed to be processed | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockBlockCapacity |
Capacity of blocks_BlockCapacity | Count | Capacity of blocks | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockNumLiveDataNodes |
Cluster DataNodes_NumLiveDataNodes | Count | Number of live DataNodes | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockNumDeadDataNodes |
Cluster DataNodes_NumDeadDataNodes | Count | Number of DataNodes marked as dead | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockNum DecomLiveDataNodes |
Cluster DataNodes_NumDecomLiveDataNodes | Count | Number of decommissioned live nodes | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockNum DecomDeadDataNodes |
Cluster DataNodes_NumDecomDeadDataNodes | Count | Number of decommissioned dead nodes | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockNum DecommissioningDataNodes |
Cluster DataNodes_NumDecommissioningDataNodes | Count | Number of decommissioning nodes | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockNum StaleDataNodes |
Cluster DataNodes_NumStaleDataNodes | Count | Number of current DataNodes marked as expired due to heartbeat delay | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockSnapshots |
Snapshots_Snapshots | Count | Number of snapshots | host4hdfsoverview, id4hdfsoverview |
EmrHdfsOverview HdfsNnBlockVolumeFailuresTotal |
Disk failures_VolumeFailuresTotal | Count | Total number of failures on all DataNodes | host4hdfsoverview, id4hdfsoverview |
Parameter | Metric Name | Unit | Description | Dimension |
---|---|---|---|---|
EmrHdfsOverview Aggregation HdfsNnBlockCapacityTotal |
Cluster storage capacity_CapacityTotal | GB | Total cluster storage capacity | id4hdfsoverview |
EmrHdfsOverview Aggregation HdfsNnBlockCapacityUsed |
Cluster storage capacity_CapacityUsed | GB | Used cluster storage capacity | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockCapacityRemaining |
Cluster storage capacity_CapacityRemaining | GB | Remaining cluster storage capacity | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockCapacity UsedNonDFS |
Cluster storage capacity_CapacityUsedNonDFS | GB | Non-HDFS used cluster capacity | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockTotalLoad |
Cluster load_TotalLoad | Count | Number of current connections | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockFilesTotal |
Total number of cluster files_FilesTotal | Count | Total number of files | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockBlockstotal |
Number of blocks_BlocksTotal | Count | Total number of blocks | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockPending ReplicationBlocks |
Number of blocks_PendingReplicationBlocks | Count | Number of blocks waiting to be backed up | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockUnder ReplicatedBlocks |
Number of blocks_UnderReplicatedBlocks | Count | Number of blocks with insufficient replicas | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockBlocksCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of corrupted blocks | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockScheduled ReplicationBlocks |
Number of blocks_ScheduledReplicationBlocks | Count | Number of blocks arranged for backup | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockPending DeletionBlocks |
Number of blocks_PendingDeletionBlocks | Count | Number of blocks waiting to be deleted | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of excessive blocks | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockPostponed MisreplicatedBlocks |
Number of blocks_PostponedMisreplicatedBlocks | Count | Number of blocks with exceptions that were postponed to be processed | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockBlockCapacity |
Capacity of blocks_BlockCapacity | Count | Capacity of blocks | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockNumLiveDataNodes |
Cluster DataNodes_NumLiveDataNodes | Count | Number of live DataNodes | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockNumDeadDataNodes |
Cluster DataNodes_NumDeadDataNodes | Count | Number of DataNodes marked as dead | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockNum DecomLiveDataNodes |
Cluster DataNodes_NumDecomLiveDataNodes | Count | Number of decommissioned live nodes | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockNum DecomDeadDataNodes |
Cluster DataNodes_NumDecomDeadDataNodes | Count | Number of decommissioned dead nodes | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockNum DecommissioningDataNodes |
Cluster DataNodes_NumDecommissioningDataNodes | Count | Number of decommissioning nodes | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockNum StaleDataNodes |
Cluster DataNodes_NumStaleDataNodes | Count | Number of current DataNodes marked as expired due to heartbeat delay | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockSnapshots |
Snapshots_Snapshots | Count | Number of snapshots | id4hdfsoverview |
EmrHdfsOverview AggregationHdfsNn BlockVolumeFailuresTotal |
Disk failures_VolumeFailuresTotal | Count | Total number of failures on all DataNodes | id4hdfsoverview |
Parameter | Metric Name | Unit | Description | Dimension |
---|---|---|---|---|
HdfsNnPort4007RxtxReceivedbytes | Data traffic_ReceivedBytes | Bytes/s | Data receiving rate | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007RxtxSentbytes | Data traffic_SentBytes | Bytes/s | Data sending rate | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007Qps Rpcqueuetimenumops |
QPS_RpcQueueTimeNumOps | Calls/s | RPC call rate | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007RtRpc queuetimeavgtime |
Request processing delay _RpcQueueTimeAvgTime |
ms | Average RPC delay | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007AuthRpc authenticationfailures |
Authentication and authorization _RpcAuthenticationFailure |
Count | Number of RPC authentication failures | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007AuthRpc authenticationsuccesses |
Authentication and authorization _RpcAuthenticationSuccesses |
Count | Number of RPC authentication successes | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007AuthRpc authorizationfailures |
Authentication and authorization _RpcAuthorizationFailures |
Count | Number of RPC authorization failures | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007AuthRpc authorizationsuccesses |
Authentication and authorization _RpcAuthorizationSuccesses |
Count | Number of RPC authorization successes | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007Connections Numopenconnections |
Number of current connections _NumOpenConnections |
Count | Number of current connections | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPort4007Queue LenCallqueuelength |
RPC processing queue length _CallQueueLength |
Count | Length of current RPC processing queue | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmMemMemnonheapusedm | JVM memory _MemNonHeapUsedM |
MB | Size of the NonHeapMemory currently used by JVM | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmMemMemnon heapcommittedm |
JVM memory _MemNonHeapCommittedM |
MB | JVM memory | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmMemMemheapusedm | JVM memory _MemHeapUsedM |
MB | Size of the HeapMemory currently used by JVM | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmMem Memheapcommittedm |
JVM memory _MemHeapCommittedM |
MB | Size of the HeapMemory committed by JVM | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmMemMemheapmaxm | JVM memory_MemHeapMaxM | MB | Size of the HeapMemory configured by JVM | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmMemMemmaxm | JVM memory_MemMaxM | MB | Maximum memory size that can be used by JVM during runtime | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnBlockReportRt Blockreportavgtime |
Block reporting delay _BlockReportAvgTime |
Blocks/s | Average delay in processing DataNode blocks per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilGcCountFgc | GC count_FGC | Operations/s | Full GC count | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilGcCountYgc | GC count_YGC | 2 operations/s | Young GC count | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilGcTimeYgct | GC time_YGCT | ms | Time consumed by Young GC | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilGcTimeFgct | GC time_FGCT | ms | Time consumed by Full GC | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilGcTimeGct | GC time_GCT | ms | Time used to collect garbage | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmJavaThreadsThreadsnew | Number of JVM threads _ThreadsNew |
Count | Number of threads in the new state | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmJavaThreads Threadsrunnable |
Number of JVM threads _ThreadsRunnable |
Count | Number of threads in the runnable state | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmJavaThreads Threadsblocked |
Number of JVM threads _ThreadsBlocked |
Count | Number of threads in the blocked state | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmJavaThreads Threadswaiting |
Number of JVM threads _ThreadsWaiting |
Count | Number of threads in the WAITING state | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmJavaThreads Threadstimedwaiting |
Number of JVM threads _ThreadsTimedWaiting |
Count | Number of threads in the TIMED WAITING state | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmJavaThreads Threadsterminated |
Number of JVM threads _ThreadsTerminated |
Count | Number of threads in the Terminated state | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmLogTotalLogfatal | Number of JVM logs_LogFatal | Count | Number of Fatal logs | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmLogTotalLogerror | Number of JVM logs_LogError | Count | Number of Error logs | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmLogTotalLogwarn | Number of JVM logs_LogWarn | Count | Number of Warn logs | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnJvmLogTotalLoginfo | Number of JVM logs_LogInfo | Count | Number of Info logs | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnStaleStorages CountNumstalestorages |
Number of storages marked as expired _NumStaleStorages |
Count | Number of current DataNodes marked as expired due to heartbeat delay | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnPendingDatanodeMessage CountPendingdatanode messagecount |
Number of pending block operation messages on the standby NameNode _PendingDataNode MessageCount |
Requests/s | Number of DataNode requests queued on the standby NameNode | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnBlocksMissingNum berofmissingblocks |
Total number of missing blocks _NumberOfMissingBlocks |
Count | Number of missing data blocks | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnBlocksMissingNumberof missingblockswithreplication factorOne |
Total number of missing blocks_NumberOf MissingBlocksWithReplication FactorOne |
Count | Number of missing blocks (rf = 1) | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOpsAllowsnapshotops | Snapshot operations _AllowSnapshotOps |
Operations/s | Number of AllowSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOps Disallowsnapshotops |
Snapshot operations _DisallowSnapshotOps |
Operations/s | Number of DisallowSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOps Createsnapshotops |
Snapshot operations _CreateSnapshotOps |
Operations/s | Number of CreateSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOps Deletesnapshotops |
Snapshot operations _DeleteSnapshotOps |
Operations/s | Number of DeleteSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOps Listsnapshottabledirops |
Snapshot operations_ListSnapshottableDirOps | Operations/s | Number of ListSnapshottableDir operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOps Snapshotdiffreportops |
Snapshot operations_SnapshotDiffReportOps | Operations/s | Number of SnapshotDiffReportOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSnapshotOps Renamesnapshotops |
Snapshot operations_RenameSnapshotOps | Operations/s | Number of RenameSnapshotOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsCreatefileops | File operations_CreateFileOps | Operations/s | Number of CreateFile operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsGetlistingops | File operations_GetListingOps | Operations/s | Number of GetListing operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsTotalfileops | File operations_TotalFileOps | Operations/s | Number of TotalFileOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsDeletefileops | File operations_DeleteFileOps | Operations/s | Number of DeleteFile operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsFileinfoops | File operations_FileInfoOps | Operations/s | Number of FileInfo operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsGetadditional datanodeops |
File operations _GetAdditionalDatanodeOps |
Operations/s | Number of GetAdditionalDatanode operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsCreatesymlinkops | File operations_CreateSymlinkOps | Operations/s | Number of CreateSymlink operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsGetlinktargetops | File operations_GetLinkTargetOps | Operations/s | Number of GetLinkTarget operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnFilesOpsFilesingetlistingops | File operations _FilesInGetListingOps |
Operations/s | Number of FilesInGetListing operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnTransactionOps Transactionsnumops |
Transaction operations _TransactionsNumOps |
Operations/s | Number of Journal transaction operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnTransactionOps Transactionsbatchedinsync |
Transaction operations _TransactionsBatchedInSync |
Operations/s | Number of Journal transaction operations processed in batches per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnImageOpsGeteditnumops | Image operations_GetEditNumOps | Operations/s | Number of GetEditNumOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnImageOpsGetimagenumops | Image operations_GetImageNumOps | Operations/s | Number of GetImageNumOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnImageOpsPutimagenumops | Image operations_PutImageNumOps | Operations/s | Number of PutImageNumOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSyncsOpsSyncsnumops | Sync operations_SyncsNumOps | Operations/s | Number of Journal syncs operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnBlocksOpsBlock receivedanddeletedops |
Data block operations _BlockOpsQueued |
Operations/s | Number of BlockReceivedAndDeletedOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnBlocksOpsBlockopsqueued | Data block operations _BlockOpsQueued |
Operations/s | Delay in processing DataNode block reporting operations | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnCacheReportOps Cachereportnumops |
Cache reporting _CacheReportNumOps |
Operations/s | Number of CacheReport operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnBlockReportOps Blockreportnumops |
Block reporting _BlockReportNumOps |
Operations/s | Number of DataNode block reporting operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnSyncsRtSyncsavgtime | Sync operation delay_SyncsAvgTime | ms | Average delay in processing Journal syncs operations | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnCacheReportRt Cachereportavgtime |
Cache reporting delay_CacheReportAvgTime | ms | Average delay in cache reporting operations | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnImageRtGeteditavgtime | Image operation delay _GetEditAvgTime |
ms | Average delay in reading the Edit file | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnImageRtGetimageavgtime | Image operation delay _GetImageAvgTime |
ms | Average delay in reading the image file | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnImageRtPutimageavgtime | Image operation delay _PutImageAvgTime |
ms | Average delay in writing the image file | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnTransactionRt Transactionsavgtime |
Transaction operation delay _TransactionsAvgTime |
ms | Average delay in processing Journal transaction operations | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnStartTimeStarttime | Start time_StartTime | ms | Process start time | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnStateState | Primary/secondary status_State | - | NN status | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnThreadCountPeakthreadcount | Number of threads_PeakThreadCount | Count | Peak number of threads | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnThreadCountThreadcount | Number of threads_ThreadCount | Count | Number of threads | host4hdfsnamenode, id4hdfsnamenode |
HdfsNnThreadCount Daemonthreadcount |
Number of threads_DaemonThreadCount | Count | Number of background threads | host4hdfsnamenode, id4hdfsnamenode |
Parameter | Metric Name | Unit | Description | Dimension |
---|---|---|---|---|
HdfsDnXceiverXceivercount | Number of Xceivers_XceiverCount | Xceivers | Number of Xceivers | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBytesByteswrittenmb | Data read/write rate_BytesReadMB | Bytes/s | DN byte write rate | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBytesBytesreadmb | Data read/write rate_BytesReadMB | Bytes/s | DN byte read rate | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBytesRemotebytesreadmb | Data read/write rate _RemoteBytesReadMB |
Bytes/s | Rate of bytes read by the remote client | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBytesRemoteby teswrittenmb |
Data read/write rate _RemoteBytesWrittenMB |
Bytes/s | Rate of bytes written by the remote client | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnClientWritesfrom remoteclient |
Number of client connections _WritesFromRemoteClient |
Count | QPS of write operations from the remote client | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnClientWritesfromlocalclient | Number of client connections _WritesFromLocalClient |
Count | OPS of write operations from the local client | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnClientReadsfrom remoteclient |
Number of client connections _ReadsFromRemoteClient |
Count | QPS of read operations from the remote client | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnClientReadsfromlocalclient | Number of client connections _ReadsFromLocalClient |
Count | QPS of read operations from the local client | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksVerifiedFailures Blockverificationfailures |
Block check failure _BlockVerificationFailures |
Failures/s | Number of block check failures | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnVolumeFailures Volumefailures |
Disk failures_VolumeFailures | Failures/s | Number of disk failures | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnNetworkErrors Datanodenetworkerrors |
Network errors _DatanodeNetworkErrors |
Errors/s | Total number of network errors | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnHbRtHeartbeatsavgtime | Heartbeat delay_HeartbeatsAvgTime | ms | Average heartbeat API time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnHbOpsHeartbeatsnumops | Heartbeat QPS_HeartbeatsNumOps | Heartbeats/s | Heartbeat API QPS | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnDatapacketAvgtimeSend datapackettransfer nanosavgtime |
Packet transfer operations QPS_SendDataPacketTransfer NanosAvgTime |
ms | Average data packet sending time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsRead blockopnumops |
Data block operations _ReadBlockOpNumOps |
Operations/s | OPS of block reads from the DataNode | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsWrite blockopnumops |
Data block operations _WriteBlockOpNumOps |
Operations/s | OPS of block writes to the DataNode | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsBlock checksumopnumops |
Data block operations _BlockChecksumOpNumOps |
Operations/s | OPS of Checksum operations by the DataNode | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsCopy blockopnumops |
Data block operations _CopyBlockOpNumOps |
Operations/s | OPS of block copying operations | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsReplace blockopnumops |
Data block operations _ReplaceBlockOpNumOps |
Operations/s | OPS of Replace Block operations | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsBlock reportsnumops |
Data block operations _BlockReportsNumOps |
Operations/s | OPS of block reporting operations | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsIncremental blockreports numops |
Data block operations _IncrementalBlockReports NumOps |
Operations/s | OPS of incremental block reporting | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsCache reportsnumops |
Data block operations _CacheReportsNumOps |
Operations/s | OPS of cache reporting | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksOpsPacketack roundtriptimenanos numops |
Data block operations _PacketAckRoundTripTimeNanos NumOps |
Operations/s | Number of ACK ROUND TRIP operations processed per second | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnFsyncOpsFsync nanosnumops |
Fsync operations _FsyncNanosNumOps |
Operations/s | Number of Fsync operations | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnFlushOpsFlush nanosnumops |
Flush operations _FlushNanosNumOps |
Operations/s | Number of Flush operations processed per second | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtRead blockopavgtime |
Data block operation delay _ReadBlockOpAvgTime |
ms | Average block read time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtWrite blockopavgtime |
Data block operation delay _ReplaceBlockOpAvgTime |
ms | Average block write time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtBlock checksumopavgtime |
Data block operation delay_BlockChecksumOpAvgTime | ms | Average block check time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtCopy blockopavgtime |
Data block operation delay _CopyBlockOpAvgTime |
ms | Average block copy time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRt Replaceblockopavgtime |
Data block operation delay _Replaceblockopavgtime |
ms | Average Replace Block operation time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtBlock reportsavgtime |
Data block operation delay _BlockReportsAvgTime |
ms | Average block reporting time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtIncremental blockreportsavgtime |
Data block operation delay _IncrementalBlockReportsAvgTime |
ms | Average time of incremental block reporting | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtCache reportsavgtime |
Data block operation delay _CacheReportsAvgTime |
ms | Average time of cache reporting | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnBlocksRtPacketack roundtriptimenanos avgtime |
Data block operation delay _PacketAckRoundTripTimeNanos AvgTime |
ms | Average time of processing ACK ROUND TRIP | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnFlushRtFlushnanosavgtime | Flush delay_FlushNanosAvgTime | ms | Average Flush operation time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnFsyncRtFsyncnanosavgtime | Fsync delay_FsyncNanosAvgTime | ms | Average Fsync operation time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOp Ramdiskblockswrite |
RAMDISKBlocks_Ram DiskBlocksWrite |
Blocks/s | Total number of blocks written to the memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOp Ramdiskblockswritefallback |
RAMDISKBlocks_Ram DiskBlocksWriteFallback |
Blocks/s | Total number of blocks failed to be written to the memory (failover to the disk) | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOpRamdisk blocksdeletedbeforelazypersisted |
RAMDISKBlocks_RamDiskBlocks DeletedBeforeLazyPersisted |
Blocks/s | Total number of blocks deleted before the application is saved to the disk | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOp Ramdiskblocksreadhits |
RAMDISKBlocks_Ram DiskBlocksReadHits |
Blocks/s | Total number of reads from the blocks in the memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOp Ramdiskblocksevicted |
RAMDISKBlocks_Ram DiskBlocksEvicted |
Blocks/s | Total number of blocks cleared in the memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOpRamdisk blocksevictedwithoutread |
RAMDISKBlocks_RamDiskBlocks EvictedWithoutRead |
Blocks/s | Total number of blocks retrieved from the memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOp Ramdiskblockslazypersisted |
RAMDISKBlocks_RamDisk BlocksLazyPersisted |
Blocks/s | Number of disk writes by the lazy writer | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksOp Ramdiskbyteslazypersisted |
RAMDISKBlocks_Ram DiskBytesLazyPersisted |
Bytes/s | Total number of bytes written to the disk by the lazy writer | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRamBlocksBytes Ramdiskbyteswrite |
RAM Disk write speed_RamDiskBytesWrite | Bytes/s | Total number of bytes written to the memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmMem Memnonheapusedm |
JVM memory _MemNonHeapUsedM |
MB | Size of the NonHeapMemory currently used by JVM | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmMem Memnonheapcommittedm |
JVM memory _MemNonHeapCommittedM |
MB | Size of the NonHeapCommittedM configured by JVM | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmMemMemheapusedm | JVM memory _MemHeapUsedM |
MB | Size of the HeapMemory currently used by JVM | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmMem Memheapcommittedm |
JVM memory _MemHeapCommittedM |
MB | Size of the HeapMemory committed by JVM | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmMemMemheapmaxm | JVM memory _MemHeapMaxM |
MB | Size of the HeapMemory configured by JVM | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmMemMemmaxm | JVM memory _MemMaxM |
MB | Maximum memory size that can be used by JVM during runtime | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmJavaThreadsThreadsnew | Number of JVM threads _ThreadsNew |
Count | Number of threads in the new state | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmJavaThreads Threadsrunnable |
Number of JVM threads _ThreadsRunnable |
Count | Number of threads in the runnable state | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmJavaThreads Threadsblocked |
Number of JVM threads _ThreadsBlocked |
Count | Number of threads in the blocked state | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmJavaThreads Threadswaiting |
Number of JVM threads _ThreadsWaiting |
Count | Number of threads in the WAITING state | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmJavaThreads Threadstimedwaiting |
Number of JVM threads _ThreadsTimedWaiting |
Count | Number of threads in the TIMED WAITING state | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmJavaThreads Threadsterminated |
Number of JVM threads _ThreadsTerminated |
Count | Number of threads in the Terminated state | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmLogTotalLogfatal | Number of JVM logs_LogFatal | Count | Number of Fatal logs | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmLogTotalLogerror | Number of JVM logs_LogError | Count | Number of Error logs | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmLogTotalLogwarn | Number of JVM logs_LogWarn | Count | Number of Warn logs | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnJvmLogTotalLoginfo | Number of JVM logs_LogInfo | Count | Number of Info logs | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilGcCountFgc | GC count_FGC | Count | Full GC count | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilGcCountYgc | GC count_YGC | Count | Young GC count | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilGcTimeYgct | GC time_YGCT | s | Time consumed by Young GC | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilGcTimeFgct | GC time_FGCT | s | Time consumed by Full GC | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnGcUtilGcTimeGct | GC time_GCT | s | Time used to collect garbage | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004RxtxReceivedbytes | Data traffic_ReceivedBytes | Bytes/s | Data receiving rate | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004RxtxSentbytes | Data traffic_SentBytes | Bytes/s | Data sending rate | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004QpsRpc queuetimenumops |
QPS_RpcQueueTimeNumOps | Queries/s | RPC call rate | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004RtRpc queuetimeavgtime |
Request processing delay _RpcQueueTimeAvgTime |
ms | Average RPC delay | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004AuthRpc authenticationfailures |
Authentication and authorization _RpcAuthenticationFailures |
Failures/s | Number of RPC authentication failures | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004AuthRpc authenticationsuccesses |
Authentication and authorization _RpcAuthenticationSuccesses |
Successes/s | Number of RPC authentication successes | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004AuthRpc authorizationfailures |
Authentication and authorization _RpcAuthorizationFailures |
Failures/s | Number of RPC authorization failures | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004AuthRpc authorizationsuccesses |
Authentication and authorization _RpcAuthorizationSuccesses |
Successes/s | Number RPC authorization successes | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004Connections Numopenconnections |
Number of current connections _NumOpenConnections |
Count | Number of current connections | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnPort4004QueueLen Callqueuelength |
RPC processing queue length _CallQueueLength |
Count | Length of the current RPC processing queue | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnThreadTimeCurrent threadcputime |
CPU time _CurrentThreadCpuTime |
ms | CPU time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnThreadTimeCurrent threadusertime |
CPU time _CurrentThreadUserTime |
ms | User time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnStartTimeStarttime | Start time_StartTime | s | Process start time | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnThreadCount Peakthreadcount |
Number of threads_PeakThreadCount | Count | Peak number of threads | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnThreadCount Daemonthreadcount |
Number of threads_DaemonThreadCount | Count | Number of background threads | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRtWrit | Read/write delay_Write | MB/s | Disk write rate | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnRtRead | Read/write delay_Read | Queries/s | Read QPS | id4hdfsdatanode, host4hdfsdatanode |
HdfsDnDatapacketOps Datapacketops |
Packet transfer QPS_DataPacketOps | Queries/s | Packet transfer QPS | id4hdfsdatanode, host4hdfsdatanode |
Parameter | Metric Name | Unit | Description | Dimension |
---|---|---|---|---|
HdfsJnJvmMemMemnon heapusedm |
JVM memory_MemNonHeapUsedM | MB | Size of the NonHeapMemory currently used by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmMemMemnon heapcommittedm |
JVM memory_MemNonHeapCommittedM | MB | JVM memory | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmMemMem heapusedm |
JVM memory_MemHeapUsedM | MB | Size of the HeapMemory currently used by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmMemMem heapcommittedm |
JVM memory_MemHeapCommittedM | MB | Size of the HeapMemory committed by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmMemMem heapmaxm |
JVM memory_MemHeapMaxM | MB | Size of the HeapMemory configured by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmMemMemmaxm | JVM memory_MemMaxM | MB | Maximum memory size that can be used by JVM during runtime | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmJavaThreads Threadsnew |
Number of JVM threads_ThreadsNew | Count | Number of threads in the new state | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmJavaThreads Threadsrunnable |
Number of JVM threads_ThreadsRunnable | Count | Number of threads in the runnable state | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmJavaThreads Threadsblocked |
Number of JVM threads_ThreadsBlocked | Count | Number of threads in the blocked state | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmJavaThreads Threadswaiting |
Number of JVM threads_ThreadsWaiting | Count | Number of threads in the WAITING state | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmJavaThreads Threadstimedwaiting |
Number of JVM threads_ThreadsTimedWaiting | Count | Number of threads in the TIMED WAITING state | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmJavaThreads Threadsterminated |
Number of JVM threads_ThreadsTerminated | Count | Number of threads in the Terminated state | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmLogTotalLogfatal | Number of JVM logs_LogFatal | Count | Number of Fatal logs | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmLogTotalLogerror | Number of JVM logs_LogError | Count | Number of Error logs | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmLogTotalLogwarn | Number of JVM logs_LogWarn | Count | Number of Warn logs | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnJvmLogTotalLoginfo | Number of JVM logs_LogInfo | Count | Number of Info logs | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilGcCountFgc | GC count_FGC | Count | Full GC count | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilGcCountYgc | GC count_YGC | Count | Young GC count | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilGcTimeYgct | GC time_YGCT | s | Time consumed by Young GC | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilGcTimeFgct | GC time_FGCT | s | Time consumed by Full GC | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnGcUtilGcTimeGct | GC time_GC | s | Time used to collect garbage | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005Rxtx Receivedbytes |
Data traffic_ReceivedBytes | Bytes/s | Data receiving rate | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005Rxtx Receivedbytes |
Data traffic_ReceivedBytes | Bytes/s | Data sending rate | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005QpsRpc queuetimenumops |
QPS_RpcQueueTimeNumOps | Queries/s | RPC call rate | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005RtRpc queuetimeavgtime |
Request processing delay_RpcQueueTimeAvgTime | ms | Average RPC delay | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005AuthRpc authenticationfailures |
Authentication and authorization_RpcAuthenticationFailures | Failures/s | Number of RPC authentication failures | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005AuthRpc authorizationsuccesses |
Authentication and authorization_RpcAuthorizationSuccesses | Successes/s | Number of RPC authorization successes | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005AuthRpc authenticationsuccesses |
Authentication and authorization_RpcAuthenticationSuccesses | Successes/s | Number of RPC authentication successes | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005AuthRpc authorizationfailures |
Authentication and authorization_RpcAuthorizationFailures | Failures/s | Number of RPC authorization failures | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005Connections Numopenconnections |
Number of current connections_NumOpenConnections | Count | Number of current connections | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnPort4005QueueLen Callqueuelength |
RPC processing queue length_CallQueueLength | Count | Length of the current RPC processing queue | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnThreadTimeCurrent threadcputime |
CPU time_CurrentThreadCpuTime | ms | CPU time | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnThreadTimeCurrent threadusertime |
CPU time_CurrentThreadUserTime | ms | User time | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnStartTimeStarttime | Start time_StartTime | s | Process start time | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnThreadCount Threadcount |
Number of threads_ThreadCount | Count | Number of threads | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnThreadCount Peakthreadcount |
Number of threads_PeakThreadCount | Count | Peak number of threads | host4hdfsjournalnode, id4hdfsjournalnode |
HdfsJnThreadCountDaemon threadcount |
Number of threads_DaemonThreadCount | Count | Number of background threads | host4hdfsjournalnode, id4hdfsjournalnode |
Parameter | Metric Name | Unit | Description | Dimension |
---|---|---|---|---|
HdfsDfzkGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilGcCountFgc | GC count_FGC | Count | Full GC count | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilGcCountYgc | GC count_YGC | Count | Young GC count | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilGcTimeYgct | GC time_YGCT | s | Time consumed by Young GC | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilGcTimeFgct | GC time_FGCT | s | Time consumed by Full GC | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
HdfsDfzkGcUtilGcTimeGct | GC time_GCT | s | Time used to collect garbage | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
The statistical granularity (
period
) may vary by metric. The DescribeBaseMetrics API can be used to obtain theperiod
supported by each metric.
Parameter Name | Dimension Name | Dimension Description | Format |
---|---|---|---|
Instances.N.Dimensions.0.Name | id4hdfsoverview | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsoverview |
Instances.N.Dimensions.0.Value | id4hdfsoverview | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
Instances.N.Dimensions.1.Name | host4hdfsoverview | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsoverview |
Instances.N.Dimensions.1.Value | host4hdfsoverview | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
Instances.N.Dimensions.0.Name | id4hdfsnamenode | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsnamenode |
Instances.N.Dimensions.0.Value | id4hdfsnamenode | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
Instances.N.Dimensions.1.Name | host4hdfsnamenode | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsnamenode |
Instances.N.Dimensions.1.Value | host4hdfsnamenode | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
Instances.N.Dimensions.0.Name | id4hdfsdatanode | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsdatanode |
Instances.N.Dimensions.0.Value | id4hdfsdatanode | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
Instances.N.Dimensions.1.Name | host4hdfsdatanode | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsdatanode |
Instances.N.Dimensions.1.Value | host4hdfsdatanode | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
Instances.N.Dimensions.0.Name | id4hdfsjournalnode | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsjournalnode |
Instances.N.Dimensions.0.Value | id4hdfsjournalnode | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
Instances.N.Dimensions.1.Name | host4hdfsjournalnode | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsjournalnode |
Instances.N.Dimensions.1.Value | host4hdfsjournalnode | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
Instances.N.Dimensions.0.Name | id4hdfszkfailovercontroller | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfszkfailovercontroller |
Instances.N.Dimensions.0.Value | id4hdfszkfailovercontroller | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
Instances.N.Dimensions.1.Name | host4hdfszkfailovercontroller | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfszkfailovercontroller |
Instances.N.Dimensions.1.Value | host4hdfszkfailovercontroller | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
EMR (HDFS) supports querying monitoring data based on the following six combinations of dimensions. The values for the input parameters are as follows:
1. To query the metric monitoring data of HDFS - OverviewAggregation, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsoverview
&Instances.N.Dimensions.0.Value=EMR instance ID
2. To query the metric monitoring data of HDFS - Overview, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsoverview
&Instances.N.Dimensions.0.Value=EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsoverview
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
3. To query the metric monitoring data of HDFS - NameNode, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsnamenode
&Instances.N.Dimensions.0.Value=EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsnamenode
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
4. To query the metric monitoring data of HDFS - DataNode, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsdatanode
&Instances.N.Dimensions.0.Value=EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsdatanode
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
5. To query the metric monitoring data of HDFS - Journal Node, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsjournalnode
&Instances.N.Dimensions.0.Value=Specific EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsjournalnode
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
6. To query the metric monitoring data of HDFS - ZKFC, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfszkfailovercontroller
&Instances.N.Dimensions.0.Value=Specific EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfszkfailovercontroller
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
Was this page helpful?