tencent cloud

Feedback

Log Search

Last updated: 2022-05-16 12:45:11

    Overview

    The log search feature collects component operation logs. Then, you can search for core service logs of the current cluster and node system logs by keyword to quickly view key service logs without logging in to nodes.

    Directions

    1. Log in to the EMR console and click the ID/Name of the target cluster in the cluster list to enter the cluster details page.
    2. On the cluster details page, select Cluster Monitoring > Log Search to filter and view log content by current cluster, log file, node IP, and time range.
      Or, on the cluster details page, select Cluster Service > target component block > Service Status, click a Node IP in Role List to enter the page where the node monitoring metrics are displayed, and then click Role Log to enter the Log Search page.

      Click a Node IP to enter the Node Status page, or click Log Source to enter the page where the node monitoring metrics are displayed.
    3. During troubleshooting, you usually need to pay attention to contextual logs of keywords. On the Log Search page, click View Context to enter the Log Context page.
    Note:

    • Currently, only logs in the past 15 days can be searched for.
    • If log collection has not been enabled on your cluster, you can contact your Tencent Cloud rep for assistance.
    Component Role Log Description
    HDFS NameNode /data/emr/hdfs/logs/hadoop-hadoop-namenode.log NameNode runtime log
    ZKFC /data/emr/hdfs/logs/hadoop-hadoop-zkfc.log ZKFC runtime log
    DataNode /data/emr/hdfs/logs/hadoop-hadoop-datanode.log DataNode runtime log
    JournalNode /data/emr/hdfs/logs/hadoop-hadoop-journalnode.log JournalNode runtime log
    DFSRouter /data/emr/hdfs/logs/hadoop-hadoop-dfsrouter.log DFRouter runtime log
    YARN ResourceManager /data/emr/yarn/logs/yarn-hadoop-resourcemanager.log ResourceManager runtime log
    NodeManager /data/emr/yarn/logs/yarn-hadoop-nodemanager.log NodeManager runtime log
    JobHistoryServer /data/emr/yarn/logs/mapred-hadoop-historyserver.log JobHistoryServer runtime log
    HBase HMaster /data/emr/hbase/logs/hbase-hadoop-master.log HMaster runtime log
    ThriftServer /data/emr/hbase/logs/hbase-hadoop-thrift.log ThriftServer runtime log
    RegionServer /data/emr/hbase/logs/hbase-hadoop-regionserver.log RegionServer runtime log
    ClickHouse ClickHouse-server /data/clickhouse/clickhouse-server/logs/clickhouse-server.log ClickHouse-server runtime log
    Druid Broker /data/emr/druid/var/log/druid/broker.log Broker runtime log
    Coordinator /data/emr/druid/var/log/druid/coordinator.log Coordinator runtime log
    Router /data/emr/druid/var/log/druid/router.log Router runtime log
    Overload /data/emr/druid/var/log/druid/overload.log Overload runtime log
    Historical /data/emr/druid/var/log/druid/historical.log Historical runtime log
    MiddleManager /data/emr/druid/var/log/druid/middleManager.log MiddleManager runtime log
    ZooKeeper ZooKeeper /data/emr/zookeeper/logs/zookeeper-root-server.log ZooKeeper runtime log
    Hive HiveServer2 /data/emr/hive/logs/hadoop-hive HiveServer2 runtime log
    Kudu KuduMaster /data/emr/kudu/logs/kudu-master.WARNING KuduMaster runtime log
    KuduServer /data/emr/kudu/logs/kudu-tserver.WARNING KuduServer runtime log
    Alluxio AlluxioMaster /data/emr/alluxio/logs/master.log AlluxioMaster runtime log
    AlluxioWorker /data/emr/alluxio/logs/worker.log AlluxioWorker runtime log
    Ranger EmbeddedServer /data/emr/ranger/logs/ranger-admin.log EmbeddedServer runtime log
    CosRanger CosRangerServer /usr/local/service/cosranger/log/info.log COSRanger runtime log
    Impala Catalogd /data/emr/impala/logs/catalogd.INFO Cataloged runtime log
    Statestored /data/emr/impala/logs/statestored.INFO Statestored runtime log
    Impalad /data/emr/impala/logs/impalad.INFO Impalad runtime log
    Spark HistoryServer /data/emr/spark/logs/spark-hadoop.log HistoryServer runtime log
    Kylin Kylin /data/emr/kylin/logs/kylin.log Kylin runtime log
    Zeppelin ZeppelinServer /data/emr/zepplin/logs/zeppelin-hadoop.log ZeppelinServer runtime log
    Knox Gateway /data/emr/knox/logs/gateway.log Gateway runtime log
    Doris BrokerBootstrap /data/emr/doris/broker/log/apache_hdfs_broker.log BrokerBootstrap runtime log
    PaloFe /data/emr/doris/fe/log/fe.log PaloFe runtime log
    PaloBe /data/emr/doris/be/log/be.INFO PaloBe runtime log
    Kafka Kafka /user/local/service/kafka/logs/server.log Kafka runtime log

    Query Rule for Minimum Log Level

    Minimum Log Level Queryable Log Levels
    INFO INFO, WARN, ERROR, FATAL
    WARN WARN, ERROR, FATAL
    ERROR ERROR, FATAL
    FATAL FATAL
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support