Running Kafka Client (Optional)

Last updated: 2021-07-14 15:54:38

    Overview

    This document explains how to start using Kafka APIs after you purchase the CKafka service. After setting up a CKafka environment on a CVM instance, you need to download and decompress the Kafka installation file and perform simple testing on Kafka APIs.

    Directions

    Step 1. Install a JDK

    1. Check Java installation

    Open a terminal window and run this command:

    java -version
    

    If the Java version number is output, Java installation is successful. If Java has not been installed, please download and install the JDK.

    2. Set up the Java environment

    Set the JAVA_HOME environment variable and point it to the Java installation directory on your machine.
    For example, if you use Java JDK 1.8.0_20, the outputs on different operating systems are as follows:

    OS Output
    Windows Set the environment variable JAVA_HOME to
    C:\Program Files\Java\jdkjdk1.8.0_20
    Linux export JAVA_HOME=/usr/local/java-current
    Mac OSX export JAVA_HOME=/Library/Java/Home

    Add the Java compiler path to the system path:

    OS Output
    Windows Add ;C:\Program Files\Java\jdk1.8.0_20\bin to the end of the system variable PATH.
    Linux export PATH=$PATH:$JAVA_HOME/bin/
    Mac OSX not required

    Use the java -version command to check your Java installation.

    Step 2. Download the Kafka installation file

    Download and decompress the Kafka installation file.
    Currently, CKafka is fully compatible with Kafka 0.9, 0.10, 1.1, and 2.4. We recommend that you download one of these versions. This document uses kafka_2.12-1.1.1 as an example.

    Step 3. Test Kafka APIs

    Go to the ./bin directory and produce and consume a message with CLI commands.

    1. Open a terminal window to start the consumer.

      bash kafka-console-consumer.sh --bootstrap-server XXXX:port --topic XXXX --consumer.config ../config/consumer.properties
      
      Note:

      • Replace XXXX:port with the domain name and port for VPC access, which can be obtained in the Access Mode section on the Instance Details page in the console.
      • topic: replace XXXX with the topic name, which can be obtained on the Topic Management page in the console.
    2. Open another terminal window to start the producer.

      bash kafka-console-producer.sh --broker-list XXXX:port --topic XXXX --producer.config ../config/producer.properties
      
      Note:

      • Replace XXXX:port with the domain name and port for VPC access, which can be obtained in the Access Mode section on the Instance Details page in the console.
      • topic: replace XXXX with the topic name, which can be obtained on the Topic Management page in the console.

      Enter the content of the message, press Enter, and you can see that the consumer received the message almost at the same time.

      Producing a message:

      Consuming a message:

    3. In the message querying page of the CKafka console, query the message sent.

      The details of the message are as follows: