- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Quick Start
- Operation Guide
- Cloud Monitoring Overview
- Monitoring View
- Dashboard
- Alarm Service
- CVM Agents
- Cloud Access Management (CAM)
- CM Connection to Grafana
- Best Practice
- Troubleshooting
- API Documentation
- History
- API Category
- Making API Requests
- Monitoring Data Query APIs
- Alarm APIs
- DescribeAlarmHistories
- CreateAlarmPolicy
- DeleteAlarmPolicy
- DescribeAlarmPolicies
- DescribeAlarmPolicy
- ModifyAlarmPolicyStatus
- SetDefaultAlarmPolicy
- BindingPolicyObject
- UnBindingPolicyObject
- UnBindingAllPolicyObject
- ModifyAlarmPolicyCondition
- ModifyAlarmPolicyNotice
- ModifyAlarmPolicyTasks
- DescribeMonitorTypes
- DescribeAllNamespaces
- DescribeAlarmMetrics
- DescribeAlarmEvents
- PutMonitorData
- DescribePolicyGroupInfo
- DescribeBindingPolicyObjectList
- DescribePolicyConditionList
- SendCustomAlarmMsg
- ModifyAlarmPolicyInfo
- DescribeConditionsTemplateList
- Legacy Alarm APIs
- Notification Template APIs
- Event Center APIs
- Data Types
- Error Codes
- FAQs
- Tencent Cloud Service Metrics
- CVM
- CBS
- TencentDB
- TencentDB for SQL Server Monitoring Metrics
- TencentDB for MySQL Monitoring Metrics
- TencentDB for Redis Monitoring Metrics
- TencentDB for MongoDB Monitoring Metrics
- TencentDB for PostgreSQL Monitoring Metrics
- TencentDB for CYNOSDB_MYSQL Monitoring Metrics
- TencentDB for TcaplusDB Monitoring Metrics
- TencentDB for MariaDB Monitoring Metrics
- TDSQL for MySQL Monitoring Metrics (Legacy)
- TDSQL for MySQL Monitoring Metrics
- SCF
- CKafka
- VPC
- NAT Gateway Monitoring Metrics
- VPN Gateway Monitoring Metrics
- VPN Tunnel Monitoring Metrics
- Direct Connect Gateway Monitoring Metrics
- CCN Monitoring Metrics
- Peering Connection Monitoring Metrics
- Bandwidth Packet Monitoring Metrics
- EIP Monitoring Metrics
- Anycast EIP Monitoring Metrics
- Network Detection Monitoring Metrics
- CLB
- COS
- CFS
- CPM
- ECM
- EMR
- CDN
- Direct Connect
- TKE
- GAAP
- CMQ
- API Gateway
- Elasticsearch
- WAF
- CLS
- Documentation Guide
- Glossary
- Release Notes and Announcements
- Product Introduction
- Purchase Guide
- Quick Start
- Operation Guide
- Cloud Monitoring Overview
- Monitoring View
- Dashboard
- Alarm Service
- CVM Agents
- Cloud Access Management (CAM)
- CM Connection to Grafana
- Best Practice
- Troubleshooting
- API Documentation
- History
- API Category
- Making API Requests
- Monitoring Data Query APIs
- Alarm APIs
- DescribeAlarmHistories
- CreateAlarmPolicy
- DeleteAlarmPolicy
- DescribeAlarmPolicies
- DescribeAlarmPolicy
- ModifyAlarmPolicyStatus
- SetDefaultAlarmPolicy
- BindingPolicyObject
- UnBindingPolicyObject
- UnBindingAllPolicyObject
- ModifyAlarmPolicyCondition
- ModifyAlarmPolicyNotice
- ModifyAlarmPolicyTasks
- DescribeMonitorTypes
- DescribeAllNamespaces
- DescribeAlarmMetrics
- DescribeAlarmEvents
- PutMonitorData
- DescribePolicyGroupInfo
- DescribeBindingPolicyObjectList
- DescribePolicyConditionList
- SendCustomAlarmMsg
- ModifyAlarmPolicyInfo
- DescribeConditionsTemplateList
- Legacy Alarm APIs
- Notification Template APIs
- Event Center APIs
- Data Types
- Error Codes
- FAQs
- Tencent Cloud Service Metrics
- CVM
- CBS
- TencentDB
- TencentDB for SQL Server Monitoring Metrics
- TencentDB for MySQL Monitoring Metrics
- TencentDB for Redis Monitoring Metrics
- TencentDB for MongoDB Monitoring Metrics
- TencentDB for PostgreSQL Monitoring Metrics
- TencentDB for CYNOSDB_MYSQL Monitoring Metrics
- TencentDB for TcaplusDB Monitoring Metrics
- TencentDB for MariaDB Monitoring Metrics
- TDSQL for MySQL Monitoring Metrics (Legacy)
- TDSQL for MySQL Monitoring Metrics
- SCF
- CKafka
- VPC
- NAT Gateway Monitoring Metrics
- VPN Gateway Monitoring Metrics
- VPN Tunnel Monitoring Metrics
- Direct Connect Gateway Monitoring Metrics
- CCN Monitoring Metrics
- Peering Connection Monitoring Metrics
- Bandwidth Packet Monitoring Metrics
- EIP Monitoring Metrics
- Anycast EIP Monitoring Metrics
- Network Detection Monitoring Metrics
- CLB
- COS
- CFS
- CPM
- ECM
- EMR
- CDN
- Direct Connect
- TKE
- GAAP
- CMQ
- API Gateway
- Elasticsearch
- WAF
- CLS
- Documentation Guide
- Glossary
## Namespace
Namespace=QCE/TXMR_HDFS
Monitoring Metrics
EMR (HDFS) provides the following metrics: HDFS - Overview, HDFS - OverviewAggregation, HDFS - NameNode, HDFS - DataNode, HDFS - Journal Node, and HDFS - ZKFC.
For more information on the parameters in each dimension, please see Overview of the Parameters in Each Dimension.
HDFS - Overview
| Parameter | Metric Name | Unit | Description | Dimension |
|---|---|---|---|---|
| EmrHdfsOverview HdfsNnBlockCapacityTotal |
Cluster storage capacity_CapacityTotal | GB | Total cluster storage capacity | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockCapacityUsed |
Cluster storage capacity_CapacityUsed | GB | Used cluster storage capacity | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockCapacityRemaining |
Cluster storage capacity_CapacityRemaining | GB | Remaining cluster storage capacity | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockCapacity UsedNonDFS |
Cluster storage capacity_CapacityUsedNonDFS | GB | Non-HDFS used cluster capacity | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockTotalLoad |
Cluster load_TotalLoad | Count | Number of current connections | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockFilesTotal |
Total number of cluster files_FilesTotal | Count | Total number of files | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockBlockstotal |
Number of blocks_BlocksTotal | Count | Total number of blocks | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockPending ReplicationBlocks |
Number of blocks_PendingReplicationBlocks | Count | Number of blocks waiting to be backed up | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockUnder ReplicatedBlocks |
Number of blocks_UnderReplicatedBlocks | Count | Number of blocks with insufficient replicas | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockBlocksCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of corrupted blocks | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockScheduled ReplicationBlocks |
Number of blocks_ScheduledReplicationBlocks | Count | Number of blocks arranged for backup | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockPending DeletionBlocks |
Number of blocks_PendingDeletionBlocks | Count | Number of blocks waiting to be deleted | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of excessive blocks | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockPostponed MisreplicatedBlocks |
Number of blocks_PostponedMisreplicatedBlocks | Count | Number of blocks with exceptions that were postponed to be processed | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockBlockCapacity |
Capacity of blocks_BlockCapacity | Count | Capacity of blocks | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockNumLiveDataNodes |
Cluster DataNodes_NumLiveDataNodes | Count | Number of live DataNodes | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockNumDeadDataNodes |
Cluster DataNodes_NumDeadDataNodes | Count | Number of DataNodes marked as dead | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockNum DecomLiveDataNodes |
Cluster DataNodes_NumDecomLiveDataNodes | Count | Number of decommissioned live nodes | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockNum DecomDeadDataNodes |
Cluster DataNodes_NumDecomDeadDataNodes | Count | Number of decommissioned dead nodes | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockNum DecommissioningDataNodes |
Cluster DataNodes_NumDecommissioningDataNodes | Count | Number of decommissioning nodes | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockNum StaleDataNodes |
Cluster DataNodes_NumStaleDataNodes | Count | Number of current DataNodes marked as expired due to heartbeat delay | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockSnapshots |
Snapshots_Snapshots | Count | Number of snapshots | host4hdfsoverview, id4hdfsoverview |
| EmrHdfsOverview HdfsNnBlockVolumeFailuresTotal |
Disk failures_VolumeFailuresTotal | Count | Total number of failures on all DataNodes | host4hdfsoverview, id4hdfsoverview |
HDFS - OverviewAggregation
| Parameter | Metric Name | Unit | Description | Dimension |
|---|---|---|---|---|
| EmrHdfsOverview Aggregation HdfsNnBlockCapacityTotal |
Cluster storage capacity_CapacityTotal | GB | Total cluster storage capacity | id4hdfsoverview |
| EmrHdfsOverview Aggregation HdfsNnBlockCapacityUsed |
Cluster storage capacity_CapacityUsed | GB | Used cluster storage capacity | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockCapacityRemaining |
Cluster storage capacity_CapacityRemaining | GB | Remaining cluster storage capacity | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockCapacity UsedNonDFS |
Cluster storage capacity_CapacityUsedNonDFS | GB | Non-HDFS used cluster capacity | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockTotalLoad |
Cluster load_TotalLoad | Count | Number of current connections | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockFilesTotal |
Total number of cluster files_FilesTotal | Count | Total number of files | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockBlockstotal |
Number of blocks_BlocksTotal | Count | Total number of blocks | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockPending ReplicationBlocks |
Number of blocks_PendingReplicationBlocks | Count | Number of blocks waiting to be backed up | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockUnder ReplicatedBlocks |
Number of blocks_UnderReplicatedBlocks | Count | Number of blocks with insufficient replicas | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockBlocksCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of corrupted blocks | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockScheduled ReplicationBlocks |
Number of blocks_ScheduledReplicationBlocks | Count | Number of blocks arranged for backup | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockPending DeletionBlocks |
Number of blocks_PendingDeletionBlocks | Count | Number of blocks waiting to be deleted | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockCorruptblocks |
Number of blocks_CorruptBlocks | Count | Number of excessive blocks | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockPostponed MisreplicatedBlocks |
Number of blocks_PostponedMisreplicatedBlocks | Count | Number of blocks with exceptions that were postponed to be processed | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockBlockCapacity |
Capacity of blocks_BlockCapacity | Count | Capacity of blocks | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockNumLiveDataNodes |
Cluster DataNodes_NumLiveDataNodes | Count | Number of live DataNodes | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockNumDeadDataNodes |
Cluster DataNodes_NumDeadDataNodes | Count | Number of DataNodes marked as dead | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockNum DecomLiveDataNodes |
Cluster DataNodes_NumDecomLiveDataNodes | Count | Number of decommissioned live nodes | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockNum DecomDeadDataNodes |
Cluster DataNodes_NumDecomDeadDataNodes | Count | Number of decommissioned dead nodes | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockNum DecommissioningDataNodes |
Cluster DataNodes_NumDecommissioningDataNodes | Count | Number of decommissioning nodes | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockNum StaleDataNodes |
Cluster DataNodes_NumStaleDataNodes | Count | Number of current DataNodes marked as expired due to heartbeat delay | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockSnapshots |
Snapshots_Snapshots | Count | Number of snapshots | id4hdfsoverview |
| EmrHdfsOverview AggregationHdfsNn BlockVolumeFailuresTotal |
Disk failures_VolumeFailuresTotal | Count | Total number of failures on all DataNodes | id4hdfsoverview |
HDFS - NameNode
| Parameter | Metric Name | Unit | Description | Dimension |
|---|---|---|---|---|
| HdfsNnPort4007RxtxReceivedbytes | Data traffic_ReceivedBytes | Bytes/s | Data receiving rate | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007RxtxSentbytes | Data traffic_SentBytes | Bytes/s | Data sending rate | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007Qps Rpcqueuetimenumops |
QPS_RpcQueueTimeNumOps | Calls/s | RPC call rate | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007RtRpc queuetimeavgtime |
Request processing delay _RpcQueueTimeAvgTime |
ms | Average RPC delay | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007AuthRpc authenticationfailures |
Authentication and authorization _RpcAuthenticationFailure |
Count | Number of RPC authentication failures | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007AuthRpc authenticationsuccesses |
Authentication and authorization _RpcAuthenticationSuccesses |
Count | Number of RPC authentication successes | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007AuthRpc authorizationfailures |
Authentication and authorization _RpcAuthorizationFailures |
Count | Number of RPC authorization failures | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007AuthRpc authorizationsuccesses |
Authentication and authorization _RpcAuthorizationSuccesses |
Count | Number of RPC authorization successes | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007Connections Numopenconnections |
Number of current connections _NumOpenConnections |
Count | Number of current connections | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPort4007Queue LenCallqueuelength |
RPC processing queue length _CallQueueLength |
Count | Length of current RPC processing queue | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmMemMemnonheapusedm | JVM memory _MemNonHeapUsedM |
MB | Size of the NonHeapMemory currently used by JVM | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmMemMemnon heapcommittedm |
JVM memory _MemNonHeapCommittedM |
MB | JVM memory | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmMemMemheapusedm | JVM memory _MemHeapUsedM |
MB | Size of the HeapMemory currently used by JVM | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmMem Memheapcommittedm |
JVM memory _MemHeapCommittedM |
MB | Size of the HeapMemory committed by JVM | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmMemMemheapmaxm | JVM memory_MemHeapMaxM | MB | Size of the HeapMemory configured by JVM | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmMemMemmaxm | JVM memory_MemMaxM | MB | Maximum memory size that can be used by JVM during runtime | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnBlockReportRt Blockreportavgtime |
Block reporting delay _BlockReportAvgTime |
Blocks/s | Average delay in processing DataNode blocks per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilGcCountFgc | GC count_FGC | Operations/s | Full GC count | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilGcCountYgc | GC count_YGC | 2 operations/s | Young GC count | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilGcTimeYgct | GC time_YGCT | ms | Time consumed by Young GC | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilGcTimeFgct | GC time_FGCT | ms | Time consumed by Full GC | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilGcTimeGct | GC time_GCT | ms | Time used to collect garbage | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmJavaThreadsThreadsnew | Number of JVM threads _ThreadsNew |
Count | Number of threads in the new state | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmJavaThreads Threadsrunnable |
Number of JVM threads _ThreadsRunnable |
Count | Number of threads in the runnable state | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmJavaThreads Threadsblocked |
Number of JVM threads _ThreadsBlocked |
Count | Number of threads in the blocked state | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmJavaThreads Threadswaiting |
Number of JVM threads _ThreadsWaiting |
Count | Number of threads in the WAITING state | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmJavaThreads Threadstimedwaiting |
Number of JVM threads _ThreadsTimedWaiting |
Count | Number of threads in the TIMED WAITING state | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmJavaThreads Threadsterminated |
Number of JVM threads _ThreadsTerminated |
Count | Number of threads in the Terminated state | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmLogTotalLogfatal | Number of JVM logs_LogFatal | Count | Number of Fatal logs | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmLogTotalLogerror | Number of JVM logs_LogError | Count | Number of Error logs | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmLogTotalLogwarn | Number of JVM logs_LogWarn | Count | Number of Warn logs | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnJvmLogTotalLoginfo | Number of JVM logs_LogInfo | Count | Number of Info logs | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnStaleStorages CountNumstalestorages |
Number of storages marked as expired _NumStaleStorages |
Count | Number of current DataNodes marked as expired due to heartbeat delay | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnPendingDatanodeMessage CountPendingdatanode messagecount |
Number of pending block operation messages on the standby NameNode _PendingDataNode MessageCount |
Requests/s | Number of DataNode requests queued on the standby NameNode | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnBlocksMissingNum berofmissingblocks |
Total number of missing blocks _NumberOfMissingBlocks |
Count | Number of missing data blocks | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnBlocksMissingNumberof missingblockswithreplication factorOne |
Total number of missing blocks_NumberOf MissingBlocksWithReplication FactorOne |
Count | Number of missing blocks (rf = 1) | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOpsAllowsnapshotops | Snapshot operations _AllowSnapshotOps |
Operations/s | Number of AllowSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOps Disallowsnapshotops |
Snapshot operations _DisallowSnapshotOps |
Operations/s | Number of DisallowSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOps Createsnapshotops |
Snapshot operations _CreateSnapshotOps |
Operations/s | Number of CreateSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOps Deletesnapshotops |
Snapshot operations _DeleteSnapshotOps |
Operations/s | Number of DeleteSnapshot operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOps Listsnapshottabledirops |
Snapshot operations_ListSnapshottableDirOps | Operations/s | Number of ListSnapshottableDir operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOps Snapshotdiffreportops |
Snapshot operations_SnapshotDiffReportOps | Operations/s | Number of SnapshotDiffReportOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSnapshotOps Renamesnapshotops |
Snapshot operations_RenameSnapshotOps | Operations/s | Number of RenameSnapshotOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsCreatefileops | File operations_CreateFileOps | Operations/s | Number of CreateFile operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsGetlistingops | File operations_GetListingOps | Operations/s | Number of GetListing operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsTotalfileops | File operations_TotalFileOps | Operations/s | Number of TotalFileOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsDeletefileops | File operations_DeleteFileOps | Operations/s | Number of DeleteFile operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsFileinfoops | File operations_FileInfoOps | Operations/s | Number of FileInfo operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsGetadditional datanodeops |
File operations _GetAdditionalDatanodeOps |
Operations/s | Number of GetAdditionalDatanode operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsCreatesymlinkops | File operations_CreateSymlinkOps | Operations/s | Number of CreateSymlink operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsGetlinktargetops | File operations_GetLinkTargetOps | Operations/s | Number of GetLinkTarget operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnFilesOpsFilesingetlistingops | File operations _FilesInGetListingOps |
Operations/s | Number of FilesInGetListing operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnTransactionOps Transactionsnumops |
Transaction operations _TransactionsNumOps |
Operations/s | Number of Journal transaction operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnTransactionOps Transactionsbatchedinsync |
Transaction operations _TransactionsBatchedInSync |
Operations/s | Number of Journal transaction operations processed in batches per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnImageOpsGeteditnumops | Image operations_GetEditNumOps | Operations/s | Number of GetEditNumOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnImageOpsGetimagenumops | Image operations_GetImageNumOps | Operations/s | Number of GetImageNumOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnImageOpsPutimagenumops | Image operations_PutImageNumOps | Operations/s | Number of PutImageNumOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSyncsOpsSyncsnumops | Sync operations_SyncsNumOps | Operations/s | Number of Journal syncs operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnBlocksOpsBlock receivedanddeletedops |
Data block operations _BlockOpsQueued |
Operations/s | Number of BlockReceivedAndDeletedOps operations executed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnBlocksOpsBlockopsqueued | Data block operations _BlockOpsQueued |
Operations/s | Delay in processing DataNode block reporting operations | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnCacheReportOps Cachereportnumops |
Cache reporting _CacheReportNumOps |
Operations/s | Number of CacheReport operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnBlockReportOps Blockreportnumops |
Block reporting _BlockReportNumOps |
Operations/s | Number of DataNode block reporting operations processed per second | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnSyncsRtSyncsavgtime | Sync operation delay_SyncsAvgTime | ms | Average delay in processing Journal syncs operations | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnCacheReportRt Cachereportavgtime |
Cache reporting delay_CacheReportAvgTime | ms | Average delay in cache reporting operations | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnImageRtGeteditavgtime | Image operation delay _GetEditAvgTime |
ms | Average delay in reading the Edit file | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnImageRtGetimageavgtime | Image operation delay _GetImageAvgTime |
ms | Average delay in reading the image file | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnImageRtPutimageavgtime | Image operation delay _PutImageAvgTime |
ms | Average delay in writing the image file | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnTransactionRt Transactionsavgtime |
Transaction operation delay _TransactionsAvgTime |
ms | Average delay in processing Journal transaction operations | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnStartTimeStarttime | Start time_StartTime | ms | Process start time | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnStateState | Primary/secondary status_State | - | NN status | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnThreadCountPeakthreadcount | Number of threads_PeakThreadCount | Count | Peak number of threads | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnThreadCountThreadcount | Number of threads_ThreadCount | Count | Number of threads | host4hdfsnamenode, id4hdfsnamenode |
| HdfsNnThreadCount Daemonthreadcount |
Number of threads_DaemonThreadCount | Count | Number of background threads | host4hdfsnamenode, id4hdfsnamenode |
HDFS - DataNode
| Parameter | Metric Name | Unit | Description | Dimension |
|---|---|---|---|---|
| HdfsDnXceiverXceivercount | Number of Xceivers_XceiverCount | Xceivers | Number of Xceivers | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBytesByteswrittenmb | Data read/write rate_BytesReadMB | Bytes/s | DN byte write rate | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBytesBytesreadmb | Data read/write rate_BytesReadMB | Bytes/s | DN byte read rate | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBytesRemotebytesreadmb | Data read/write rate _RemoteBytesReadMB |
Bytes/s | Rate of bytes read by the remote client | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBytesRemoteby teswrittenmb |
Data read/write rate _RemoteBytesWrittenMB |
Bytes/s | Rate of bytes written by the remote client | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnClientWritesfrom remoteclient |
Number of client connections _WritesFromRemoteClient |
Count | QPS of write operations from the remote client | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnClientWritesfromlocalclient | Number of client connections _WritesFromLocalClient |
Count | OPS of write operations from the local client | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnClientReadsfrom remoteclient |
Number of client connections _ReadsFromRemoteClient |
Count | QPS of read operations from the remote client | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnClientReadsfromlocalclient | Number of client connections _ReadsFromLocalClient |
Count | QPS of read operations from the local client | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksVerifiedFailures Blockverificationfailures |
Block check failure _BlockVerificationFailures |
Failures/s | Number of block check failures | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnVolumeFailures Volumefailures |
Disk failures_VolumeFailures | Failures/s | Number of disk failures | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnNetworkErrors Datanodenetworkerrors |
Network errors _DatanodeNetworkErrors |
Errors/s | Total number of network errors | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnHbRtHeartbeatsavgtime | Heartbeat delay_HeartbeatsAvgTime | ms | Average heartbeat API time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnHbOpsHeartbeatsnumops | Heartbeat QPS_HeartbeatsNumOps | Heartbeats/s | Heartbeat API QPS | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnDatapacketAvgtimeSend datapackettransfer nanosavgtime |
Packet transfer operations QPS_SendDataPacketTransfer NanosAvgTime |
ms | Average data packet sending time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsRead blockopnumops |
Data block operations _ReadBlockOpNumOps |
Operations/s | OPS of block reads from the DataNode | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsWrite blockopnumops |
Data block operations _WriteBlockOpNumOps |
Operations/s | OPS of block writes to the DataNode | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsBlock checksumopnumops |
Data block operations _BlockChecksumOpNumOps |
Operations/s | OPS of Checksum operations by the DataNode | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsCopy blockopnumops |
Data block operations _CopyBlockOpNumOps |
Operations/s | OPS of block copying operations | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsReplace blockopnumops |
Data block operations _ReplaceBlockOpNumOps |
Operations/s | OPS of Replace Block operations | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsBlock reportsnumops |
Data block operations _BlockReportsNumOps |
Operations/s | OPS of block reporting operations | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsIncremental blockreports numops |
Data block operations _IncrementalBlockReports NumOps |
Operations/s | OPS of incremental block reporting | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsCache reportsnumops |
Data block operations _CacheReportsNumOps |
Operations/s | OPS of cache reporting | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksOpsPacketack roundtriptimenanos numops |
Data block operations _PacketAckRoundTripTimeNanos NumOps |
Operations/s | Number of ACK ROUND TRIP operations processed per second | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnFsyncOpsFsync nanosnumops |
Fsync operations _FsyncNanosNumOps |
Operations/s | Number of Fsync operations | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnFlushOpsFlush nanosnumops |
Flush operations _FlushNanosNumOps |
Operations/s | Number of Flush operations processed per second | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtRead blockopavgtime |
Data block operation delay _ReadBlockOpAvgTime |
ms | Average block read time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtWrite blockopavgtime |
Data block operation delay _ReplaceBlockOpAvgTime |
ms | Average block write time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtBlock checksumopavgtime |
Data block operation delay_BlockChecksumOpAvgTime | ms | Average block check time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtCopy blockopavgtime |
Data block operation delay _CopyBlockOpAvgTime |
ms | Average block copy time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRt Replaceblockopavgtime |
Data block operation delay _Replaceblockopavgtime |
ms | Average Replace Block operation time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtBlock reportsavgtime |
Data block operation delay _BlockReportsAvgTime |
ms | Average block reporting time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtIncremental blockreportsavgtime |
Data block operation delay _IncrementalBlockReportsAvgTime |
ms | Average time of incremental block reporting | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtCache reportsavgtime |
Data block operation delay _CacheReportsAvgTime |
ms | Average time of cache reporting | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnBlocksRtPacketack roundtriptimenanos avgtime |
Data block operation delay _PacketAckRoundTripTimeNanos AvgTime |
ms | Average time of processing ACK ROUND TRIP | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnFlushRtFlushnanosavgtime | Flush delay_FlushNanosAvgTime | ms | Average Flush operation time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnFsyncRtFsyncnanosavgtime | Fsync delay_FsyncNanosAvgTime | ms | Average Fsync operation time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOp Ramdiskblockswrite |
RAMDISKBlocks_Ram DiskBlocksWrite |
Blocks/s | Total number of blocks written to the memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOp Ramdiskblockswritefallback |
RAMDISKBlocks_Ram DiskBlocksWriteFallback |
Blocks/s | Total number of blocks failed to be written to the memory (failover to the disk) | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOpRamdisk blocksdeletedbeforelazypersisted |
RAMDISKBlocks_RamDiskBlocks DeletedBeforeLazyPersisted |
Blocks/s | Total number of blocks deleted before the application is saved to the disk | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOp Ramdiskblocksreadhits |
RAMDISKBlocks_Ram DiskBlocksReadHits |
Blocks/s | Total number of reads from the blocks in the memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOp Ramdiskblocksevicted |
RAMDISKBlocks_Ram DiskBlocksEvicted |
Blocks/s | Total number of blocks cleared in the memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOpRamdisk blocksevictedwithoutread |
RAMDISKBlocks_RamDiskBlocks EvictedWithoutRead |
Blocks/s | Total number of blocks retrieved from the memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOp Ramdiskblockslazypersisted |
RAMDISKBlocks_RamDisk BlocksLazyPersisted |
Blocks/s | Number of disk writes by the lazy writer | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksOp Ramdiskbyteslazypersisted |
RAMDISKBlocks_Ram DiskBytesLazyPersisted |
Bytes/s | Total number of bytes written to the disk by the lazy writer | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRamBlocksBytes Ramdiskbyteswrite |
RAM Disk write speed_RamDiskBytesWrite | Bytes/s | Total number of bytes written to the memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmMem Memnonheapusedm |
JVM memory _MemNonHeapUsedM |
MB | Size of the NonHeapMemory currently used by JVM | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmMem Memnonheapcommittedm |
JVM memory _MemNonHeapCommittedM |
MB | Size of the NonHeapCommittedM configured by JVM | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmMemMemheapusedm | JVM memory _MemHeapUsedM |
MB | Size of the HeapMemory currently used by JVM | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmMem Memheapcommittedm |
JVM memory _MemHeapCommittedM |
MB | Size of the HeapMemory committed by JVM | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmMemMemheapmaxm | JVM memory _MemHeapMaxM |
MB | Size of the HeapMemory configured by JVM | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmMemMemmaxm | JVM memory _MemMaxM |
MB | Maximum memory size that can be used by JVM during runtime | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmJavaThreadsThreadsnew | Number of JVM threads _ThreadsNew |
Count | Number of threads in the new state | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmJavaThreads Threadsrunnable |
Number of JVM threads _ThreadsRunnable |
Count | Number of threads in the runnable state | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmJavaThreads Threadsblocked |
Number of JVM threads _ThreadsBlocked |
Count | Number of threads in the blocked state | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmJavaThreads Threadswaiting |
Number of JVM threads _ThreadsWaiting |
Count | Number of threads in the WAITING state | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmJavaThreads Threadstimedwaiting |
Number of JVM threads _ThreadsTimedWaiting |
Count | Number of threads in the TIMED WAITING state | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmJavaThreads Threadsterminated |
Number of JVM threads _ThreadsTerminated |
Count | Number of threads in the Terminated state | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmLogTotalLogfatal | Number of JVM logs_LogFatal | Count | Number of Fatal logs | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmLogTotalLogerror | Number of JVM logs_LogError | Count | Number of Error logs | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmLogTotalLogwarn | Number of JVM logs_LogWarn | Count | Number of Warn logs | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnJvmLogTotalLoginfo | Number of JVM logs_LogInfo | Count | Number of Info logs | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilGcCountFgc | GC count_FGC | Count | Full GC count | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilGcCountYgc | GC count_YGC | Count | Young GC count | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilGcTimeYgct | GC time_YGCT | s | Time consumed by Young GC | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilGcTimeFgct | GC time_FGCT | s | Time consumed by Full GC | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnGcUtilGcTimeGct | GC time_GCT | s | Time used to collect garbage | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004RxtxReceivedbytes | Data traffic_ReceivedBytes | Bytes/s | Data receiving rate | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004RxtxSentbytes | Data traffic_SentBytes | Bytes/s | Data sending rate | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004QpsRpc queuetimenumops |
QPS_RpcQueueTimeNumOps | Queries/s | RPC call rate | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004RtRpc queuetimeavgtime |
Request processing delay _RpcQueueTimeAvgTime |
ms | Average RPC delay | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004AuthRpc authenticationfailures |
Authentication and authorization _RpcAuthenticationFailures |
Failures/s | Number of RPC authentication failures | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004AuthRpc authenticationsuccesses |
Authentication and authorization _RpcAuthenticationSuccesses |
Successes/s | Number of RPC authentication successes | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004AuthRpc authorizationfailures |
Authentication and authorization _RpcAuthorizationFailures |
Failures/s | Number of RPC authorization failures | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004AuthRpc authorizationsuccesses |
Authentication and authorization _RpcAuthorizationSuccesses |
Successes/s | Number RPC authorization successes | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004Connections Numopenconnections |
Number of current connections _NumOpenConnections |
Count | Number of current connections | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnPort4004QueueLen Callqueuelength |
RPC processing queue length _CallQueueLength |
Count | Length of the current RPC processing queue | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnThreadTimeCurrent threadcputime |
CPU time _CurrentThreadCpuTime |
ms | CPU time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnThreadTimeCurrent threadusertime |
CPU time _CurrentThreadUserTime |
ms | User time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnStartTimeStarttime | Start time_StartTime | s | Process start time | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnThreadCount Peakthreadcount |
Number of threads_PeakThreadCount | Count | Peak number of threads | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnThreadCount Daemonthreadcount |
Number of threads_DaemonThreadCount | Count | Number of background threads | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRtWrit | Read/write delay_Write | MB/s | Disk write rate | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnRtRead | Read/write delay_Read | Queries/s | Read QPS | id4hdfsdatanode, host4hdfsdatanode |
| HdfsDnDatapacketOps Datapacketops |
Packet transfer QPS_DataPacketOps | Queries/s | Packet transfer QPS | id4hdfsdatanode, host4hdfsdatanode |
HDFS - Journal Node
| Parameter | Metric Name | Unit | Description | Dimension |
|---|---|---|---|---|
| HdfsJnJvmMemMemnon heapusedm |
JVM memory_MemNonHeapUsedM | MB | Size of the NonHeapMemory currently used by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmMemMemnon heapcommittedm |
JVM memory_MemNonHeapCommittedM | MB | JVM memory | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmMemMem heapusedm |
JVM memory_MemHeapUsedM | MB | Size of the HeapMemory currently used by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmMemMem heapcommittedm |
JVM memory_MemHeapCommittedM | MB | Size of the HeapMemory committed by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmMemMem heapmaxm |
JVM memory_MemHeapMaxM | MB | Size of the HeapMemory configured by JVM | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmMemMemmaxm | JVM memory_MemMaxM | MB | Maximum memory size that can be used by JVM during runtime | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmJavaThreads Threadsnew |
Number of JVM threads_ThreadsNew | Count | Number of threads in the new state | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmJavaThreads Threadsrunnable |
Number of JVM threads_ThreadsRunnable | Count | Number of threads in the runnable state | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmJavaThreads Threadsblocked |
Number of JVM threads_ThreadsBlocked | Count | Number of threads in the blocked state | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmJavaThreads Threadswaiting |
Number of JVM threads_ThreadsWaiting | Count | Number of threads in the WAITING state | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmJavaThreads Threadstimedwaiting |
Number of JVM threads_ThreadsTimedWaiting | Count | Number of threads in the TIMED WAITING state | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmJavaThreads Threadsterminated |
Number of JVM threads_ThreadsTerminated | Count | Number of threads in the Terminated state | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmLogTotalLogfatal | Number of JVM logs_LogFatal | Count | Number of Fatal logs | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmLogTotalLogerror | Number of JVM logs_LogError | Count | Number of Error logs | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmLogTotalLogwarn | Number of JVM logs_LogWarn | Count | Number of Warn logs | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnJvmLogTotalLoginfo | Number of JVM logs_LogInfo | Count | Number of Info logs | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilGcCountFgc | GC count_FGC | Count | Full GC count | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilGcCountYgc | GC count_YGC | Count | Young GC count | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilGcTimeYgct | GC time_YGCT | s | Time consumed by Young GC | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilGcTimeFgct | GC time_FGCT | s | Time consumed by Full GC | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnGcUtilGcTimeGct | GC time_GC | s | Time used to collect garbage | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005Rxtx Receivedbytes |
Data traffic_ReceivedBytes | Bytes/s | Data receiving rate | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005Rxtx Receivedbytes |
Data traffic_ReceivedBytes | Bytes/s | Data sending rate | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005QpsRpc queuetimenumops |
QPS_RpcQueueTimeNumOps | Queries/s | RPC call rate | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005RtRpc queuetimeavgtime |
Request processing delay_RpcQueueTimeAvgTime | ms | Average RPC delay | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005AuthRpc authenticationfailures |
Authentication and authorization_RpcAuthenticationFailures | Failures/s | Number of RPC authentication failures | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005AuthRpc authorizationsuccesses |
Authentication and authorization_RpcAuthorizationSuccesses | Successes/s | Number of RPC authorization successes | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005AuthRpc authenticationsuccesses |
Authentication and authorization_RpcAuthenticationSuccesses | Successes/s | Number of RPC authentication successes | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005AuthRpc authorizationfailures |
Authentication and authorization_RpcAuthorizationFailures | Failures/s | Number of RPC authorization failures | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005Connections Numopenconnections |
Number of current connections_NumOpenConnections | Count | Number of current connections | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnPort4005QueueLen Callqueuelength |
RPC processing queue length_CallQueueLength | Count | Length of the current RPC processing queue | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnThreadTimeCurrent threadcputime |
CPU time_CurrentThreadCpuTime | ms | CPU time | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnThreadTimeCurrent threadusertime |
CPU time_CurrentThreadUserTime | ms | User time | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnStartTimeStarttime | Start time_StartTime | s | Process start time | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnThreadCount Threadcount |
Number of threads_ThreadCount | Count | Number of threads | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnThreadCount Peakthreadcount |
Number of threads_PeakThreadCount | Count | Peak number of threads | host4hdfsjournalnode, id4hdfsjournalnode |
| HdfsJnThreadCountDaemon threadcount |
Number of threads_DaemonThreadCount | Count | Number of background threads | host4hdfsjournalnode, id4hdfsjournalnode |
HDFS - ZKFC
| Parameter | Metric Name | Unit | Description | Dimension |
|---|---|---|---|---|
| HdfsDfzkGcUtilMemoryS0 | Memory space percentage_S0 | % | Percentage of used Survivor 0 memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilMemoryS1 | Memory space percentage_S1 | % | Percentage of used Survivor 1 memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilMemoryE | Memory space percentage_E | % | Percentage of used Eden memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilMemoryO | Memory space percentage_O | % | Percentage of used Old memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilMemoryM | Memory space percentage_M | % | Percentage of used Metaspace memory | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilMemoryCcs | Memory space percentage_CCS | % | Percentage of memory used by compressed class space | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilGcCountFgc | GC count_FGC | Count | Full GC count | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilGcCountYgc | GC count_YGC | Count | Young GC count | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilGcTimeYgct | GC time_YGCT | s | Time consumed by Young GC | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilGcTimeFgct | GC time_FGCT | s | Time consumed by Full GC | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
| HdfsDfzkGcUtilGcTimeGct | GC time_GCT | s | Time used to collect garbage | host4hdfszkfailovercontroller, id4hdfszkfailovercontroller |
The statistical granularity (
period) may vary by metric. The DescribeBaseMetrics API can be used to obtain theperiodsupported by each metric.
Overview of the Parameters in Each Dimension
| Parameter Name | Dimension Name | Dimension Description | Format |
|---|---|---|---|
| Instances.N.Dimensions.0.Name | id4hdfsoverview | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsoverview |
| Instances.N.Dimensions.0.Value | id4hdfsoverview | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
| Instances.N.Dimensions.1.Name | host4hdfsoverview | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsoverview |
| Instances.N.Dimensions.1.Value | host4hdfsoverview | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
| Instances.N.Dimensions.0.Name | id4hdfsnamenode | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsnamenode |
| Instances.N.Dimensions.0.Value | id4hdfsnamenode | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
| Instances.N.Dimensions.1.Name | host4hdfsnamenode | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsnamenode |
| Instances.N.Dimensions.1.Value | host4hdfsnamenode | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
| Instances.N.Dimensions.0.Name | id4hdfsdatanode | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsdatanode |
| Instances.N.Dimensions.0.Value | id4hdfsdatanode | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
| Instances.N.Dimensions.1.Name | host4hdfsdatanode | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsdatanode |
| Instances.N.Dimensions.1.Value | host4hdfsdatanode | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
| Instances.N.Dimensions.0.Name | id4hdfsjournalnode | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfsjournalnode |
| Instances.N.Dimensions.0.Value | id4hdfsjournalnode | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
| Instances.N.Dimensions.1.Name | host4hdfsjournalnode | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfsjournalnode |
| Instances.N.Dimensions.1.Value | host4hdfsjournalnode | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
| Instances.N.Dimensions.0.Name | id4hdfszkfailovercontroller | Dimension name of the EMR instance ID | String-type dimension name, such as id4hdfszkfailovercontroller |
| Instances.N.Dimensions.0.Value | id4hdfszkfailovercontroller | Specific EMR instance ID | Specific EMR instance ID, such as emr-mm8bs222 |
| Instances.N.Dimensions.1.Name | host4hdfszkfailovercontroller | Dimension name of the node IP in the EMR instance | String-type dimension name, such as host4hdfszkfailovercontroller |
| Instances.N.Dimensions.1.Value | host4hdfszkfailovercontroller | Specific node IP in the EMR instance | Specific node IP, such as 1.1.1.1 |
Input Parameters
EMR (HDFS) supports querying monitoring data based on the following six combinations of dimensions. The values for the input parameters are as follows:
1. To query the metric monitoring data of HDFS - OverviewAggregation, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsoverview
&Instances.N.Dimensions.0.Value=EMR instance ID
2. To query the metric monitoring data of HDFS - Overview, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsoverview
&Instances.N.Dimensions.0.Value=EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsoverview
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
3. To query the metric monitoring data of HDFS - NameNode, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsnamenode
&Instances.N.Dimensions.0.Value=EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsnamenode
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
4. To query the metric monitoring data of HDFS - DataNode, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsdatanode
&Instances.N.Dimensions.0.Value=EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsdatanode
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
5. To query the metric monitoring data of HDFS - Journal Node, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfsjournalnode
&Instances.N.Dimensions.0.Value=Specific EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfsjournalnode
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
6. To query the metric monitoring data of HDFS - ZKFC, use the following input parameters:
&Namespace=QCE/TXMR_HDFS
&Instances.N.Dimensions.0.Name=id4hdfszkfailovercontroller
&Instances.N.Dimensions.0.Value=Specific EMR instance ID
&Instances.N.Dimensions.1.Name=host4hdfszkfailovercontroller
&Instances.N.Dimensions.1.Value=Specific node IP in the EMR instance
Yes
No
Was this page helpful?