tencent cloud

Feedback

Topic Management

Last updated: 2022-05-17 15:55:56
This document is currently invalid. Please refer to the documentation page of the product.

    Overview

    Topic is a category name where messages are stored and published. With CKafka, producers write messages to topics, and consumers read messages from topics. To enable horizontal scaling, a topic is divided into multiple partitions. This allows you to horizontally scale your resources by adding more partitions in case of performance bottlenecks.

    This document describes how to manage the topics under an existing instance in the CKafka console.

    Directions

    Creating topic

    1. Log in to the CKafka console.
    2. On the Instance List page, click the ID/Name of the target instance to enter the instance details page.
    3. On the instance details page, click Topic Management at the top and click Create.
    4. In the Create Topic window, set the number of partitions and replicas and other parameters.
      • Name: The topic name. It cannot be changed once entered and can contain only letters, digits, underscores, hyphens, and periods.
      • Partition Count: It is a concept in physical partition, where one topic can contain one or more partitions. CKafka uses partition as an allocation unit.
      • Replica Count: The number of partition replicas is used to ensure the high availability of the partition. To ensure data reliability, creating a single-replica topic is not supported. Two replicas are enabled by default.
        Replicas are also counted into the number of partitions. For example, if you create 1 topic with 6 partitions, and 2 replicas for each partition, then you have a total of 12 partitions (1 x 6 x 2).
      • Tag: Set a resource tag. For more information, see Tag Overview.
      • Preset ACL Policy: Select the preset ACL policy. For more information on ACL policy, see Configuring ACL Policy.
    5. Click Submit.

    Viewing topic details

    1. Log in to the CKafka console.
    2. Click Instance List on the left sidebar and click the ID/Name of the target instance to enter the instance details page.
    3. On the instance details page, click Topic Management to view the topic information and enter the topic list page.
    4. On the topic list page, click the right triangle icon on the left of the topic name to view the topic details.
    Item Description
    Partition Name Partition name
    Leader The leader processes all read/write requests in the partition, and the follower passively and periodically copies the data on the leader
    Replica Replica list
    ISR Replicas with synced messages
    Start Offset The last position of message consumption
    End Offset The last position of message write
    Messages Number of stored messages
    Unsynced Replicas Number of unsynced replicas. You can filter partitions with unsynced replicas

    Sending messages

    1. On the Instance List page, click the ID/Name of the target instance to enter the instance details page.
    2. On the instance details page, select Topic Management and click Send Message in the Operation column.
      • Message Content: Enter the content of the message to be sent, which is required.
      • Message Key: Enter the sending key, which is optional.
      • Send to Specified Partition: This parameter supports sending messages to the specified partition, which is disabled by default.
    3. Click OK to send the message. In the Sent the message successfully pop-up window, click Message Query to view the message just sent.

    Viewing producer connection

    1. On the Instance List page, click the ID/Name of the target instance to enter the instance details page.
    2. On the instance details page, select Topic Management and click Producer Connection in the Operation column to view the list of producers connected to the topic.

    Deleting topic

    Note:

    • Deleting a topic will delete the messages stored in the topic too. Proceed with caution.
    • Topic deletion is an async operation. After you finish the steps required to delete a topic, it takes 1 minute for the configuration to take effect with ZooKeeper. During this period, if you try to create a topic with the same name as the deleted one, the system will return the error code [4000]10011. Wait and try again later.
    1. On the Instance List page, click the ID/Name of the target instance to enter the instance details page.
    2. On the instance details page, select Topic Management and click Delete in the Operation column.
    3. In the window that pops up, click OK to delete the topic.

    Configuring advanced topic parameters

    1. On the Instance List page, click the ID/Name of the target instance to enter the instance details page.
    2. On the instance details page, select Topic Management.
    3. In the Operation column, click Edit > Show advanced configuration and set the following parameters:

    The parameters are described as follows:

    Parameter Default Value Valid Values Description
    cleanup.policy delete delete/compact Log can be deleted by retention time, or can be compacted by key (the compact mode is required for kafka connect).
    min.insync.replicas 1 - When "producer" sets "request.required.acks" to 1, "min.insync.replicas" will specify the minimum number of replicas.
    unclean.leader.election.enable true true/false This parameter specifies whether a replica not in ISR can be set as a leader.
    segment.ms - 1–90 days Segment shard rolling duration in ms. Minimum value: 86,400,000 ms.
    retention.ms Message retention period of the instance 60000 ms–90 days Message retention period at the topic level.
    retention.bytes Message retention size of the instance 1–1024 GB Message retention size at the topic level. If both the message retention period and size are set for a topic, the threshold first reached by actually retained messages will prevail.
    max.message.bytes - 1 KB–12 MB Maximum message size at the topic level. If this parameter is left empty, it will be 1 MB by default.

    Setting topic traffic throttling rule

    You can throttle the topic traffic to prevent the excessive traffic of one topic from affecting other topics.

    1. On the Instance List page, click the ID/Name of the target instance to enter the instance details page.
    2. On the instance details page, select Topic Management.
    3. In the Operation column, click Edit > Traffic Throttling and set the threshold.
      • Maximum Topic Production Traffic: This value excludes replica traffic and ranges from 1 MB/s to the maximum bandwidth purchased for the instance / number of replicas of the topic.
      • Maximum Topic Consumption Traffic: This value ranges from 1 MB/s to the maximum bandwidth purchased for the instance.
        Note:

        • The underlying layer throttles the traffic for brokers, and the actual traffic throttling value (equal to an integer multiple of the number of brokers) may be slightly different from the set value.
        • For more information on the soft traffic throttling mechanism, see Traffic Throttling.
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support