A distributed, high-throughput and highly scalable messaging system
Tencent Cloud Kafka (CKafka) is a distributed, high-throughput and highly scalable messaging system that is fully compatible with open-source Apache Kafka API v0.9 and v0.10. Based on the publish/subscribe model, CKafka enables async interaction between the message producer and consumer by decoupling the messages and thereby eliminating wait time. CKafka supports data compression and offline and real-time data processing, making it ideal for collecting compressed logs and aggregating monitoring data.
CKafka is fully compatible with Apache Kafka v0.9 and v0.10, completely eliminating the costs associated with cloudification. In addition, it works well with open-source upstream and downstream components and seamlessly supports Kafka Streams, Kafka Connect and KSQL.
CKafka can interconnect with more than 13 Tencent Cloud products such as EMR, COS, CIS, SCS, SCF and CLS, enabling fast one-click deployment.
CKafka boasts high clustering performance and outperforms open-source solutions in terms of productivity. Furthermore, its distributed deployment solution guarantees cluster stability.
CKafka clusters are horizontally scalable and instances can be seamlessly upgraded. The underlying system automatically scales elastically according to the business scope without affecting the user experience.
Tenants are isolated at the network level so that the network access of instances is naturally isolated among different accounts. CAM authentication for management streams and SASL permission control for data streams are supported for strict access control.
CKafka provides a complete set of OPS services empowered by the Tencent Cloud platform, including multi-dimensional monitoring and alarm services such as tenant isolation, access control, message retention query and consumer details.
CKafka effectively decouples the relationship between the message producer and consumer, allowing you to independently scale or modify the production-consumption processing procedure as long as they follow the same interface constraints.
As a result, CKafka can reliably replace traditional messaging middleware. Aside from its ability to decouple producer-consumer relationships and cache unprocessed messages, CKafka features higher throughput, a stronger partition replication mechanism and better fault tolerance.
CKafka can work seamlessly with EMR to build a complete log analysis system. Logs are first collected by the agent deployed on the client, and the data is aggregated to CKafka. The data is then computed and consumed multiple times by the backend big data suite such as Spark, and the original logs are cleaned, stored or graphically displayed.
CKafka can be used together with Stream Compute Service (SCS) for real-time/offline data processing and exception detection to meet the needs of different scenarios:
Real-time data is analyzed and displayed and exceptions are detected to quickly locate system issues.
Historical consumption data is stored and analyzed offline for secondary data processing and trend report generation.
CKafka supports two billing modes: pay-as-you-go and monthly subscription. For more information, see Billing Overview.