Last updated: 2018-09-12 10:56:24PDF
1. View the CKafka instance and create a topic
After your application is approved, the CKafka instance is displayed in the CKafka console. Click the instance to view its details, including region, private network vip and port.
In the topic management page, you can create a topic, and specify the number of partitions and replicas for the topic.
Note: The current topic name cannot be changed after input. In addition, when you have specified the number of partitions, you can only add partitions. Number of replicas cannot be changed once being specified.
After the topic and partitions are created, you can perform the production and consumption operations on this instance via the Kafka client on CVM.
2. Download the Kafka toolkit locally
2.1 Install the JDK environment
This document describes how to set up a CKafka environment on your CVM. First, purchase a CVM on the purchase page and then log in to the CVM. The configuration of the test machine in this example is as follows:
Operating system: CentOS 6.8 64-bit
CPU: 1 core
Memory: 2 GB
Public network bandwidth: 1 Mbps
Next, install JDK on the CVM.
(1). Download JDK, which can be obtained with the wget command. You can also download a different version from the official website.
It is recommended to use a version later than JDK 1.7. The version used in this example is JDK 1.7.0_79.
(2). Move it to a fixed folder and decompress it:
mkdir /usr/local/jdk mv jdk-7u79-linux-x64.tar.gz /usr/local/jdk/ cd /usr/local/jdk/ tar -xzvf jdk-7u79-linux-x64.tar.gz
(3). Configure the environment variables.
Add the configuration of the following environment variables to the end of the file.
export JAVA_HOME=/usr/local/jdk/jdk1.7.0_79(JDK's decompressed directory) export JRE_HOME=/usr/local/jdk/jdk1.7.0_79/jre export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$JRE_HOME/lib
Save and exit wq, and then use the command 'source / etc / profile' to make the file take effect immediately.
Verify whether the environment has been installed using the following command (javac command also applies), and then check whether the version numbers are consistent.
The following figure indicates the JDK installation is complete.
2.2 Download the Kafka toolkit
Download and decompress the Kafka installer package
CKafka is 100% compatible with Kafka 0.9. You're recommended to download this version of Kafka installer package from the official website http://kafka.apache.org/.
wget "http://mirrors.hust.edu.cn/apache/kafka/0.9.0.1/kafka_2.11-0.9.0.1.tgz" tar -xzvf kafka_2.11-0.9.0.1.tgz mv kafka_2.11-0.9.0.1 /opt/
The Kafka is ready for use immediately after download and decompression, without the need of configuring other environments.
You can test whether this machine is connected to the CKafka instance with telnet command.
telnet ip 9092
2.3 Simple tests for Kafka APIs
./kafka-console-producer.sh --broker-list xxx.xxx.xxx.xxx:9092 --topic topicName This is a message This is another message
The IP in the broker-list is the vip in the CKafka instance, and topicName is the topic name in the CKafka instance.
Receive messages (Zookeeper cluster is hidden in CKafka by default)
./kafka-console-consumer.sh --bootstrap-server xxx.xxx.xxx.xxx:9092 --from-beginning --new-consumer --topic topicName This is a message This is another message
In the above commands, no consumer group is specified for consumption, so the system generates a group at random for consumption. In this way, it is very likely to reach the limit of group. Therefore, it is recommended to receive messages by specifying a group.
First, configure the specified group name in the consumer.properties, as shown below.
After the configuration, run the command that specifies the consumer group as shown below:
./kafka-console-consumer.sh --bootstrap-server xxx.xxx.xxx.xxx:9092 --from-beginning --new-consumer --topic topicName --consumer.config ../config/consumer.properties
Check the CKafka monitor.
3. Other features
3.1 Enable the whitelist
CKafka supports enabling IP whitelist for a topic to ensure the data security.
You can enable the IP whitelist in both "New Topic" and "Edit Topic" pages.
3.2 Set message retention time
CKafka supports setting the message retention time (in minutes). The minimum is 1 minute, and the maximum is 30 days.