Overview

Last updated: 2020-02-26 20:15:25

PDF

Introduction to CMQ message queuing

CMQ (Cloud Message Queue) is a distributed message queuing system based on Tencent Tencent's message engine. CMQ ensures strong consistency of messages through Tencent Tencent's distributed Raft algorithm, and 3 copies of message synchronization are placed on disk to ensure high reliability of messages. It provides message queuing, publish subscription, message rewind, delayed messages, sequential messages, message tracks and other services. It has the advantages of high reliability, high availability, high performance, dynamic scalability and so on. With an iterative development history of more than 7 years, CMQ provides asynchronous messaging service support for Tencent's internal large-scale businesses, including Wechat, webank, QQ Show, Mobile QQ and so on.

CMQ has been officially commercially available and provides highly available messages to Tencent Cloud in many regions around the world. Tencent Cloud services's hardware facilities are implemented in accordance with the high standards of Tencent Client's IDC. Using multi-Data center deployment in a single region, even if the entire Data center is not available, you can still provide message publish services for applications. At the same time, there are deployments in Shenzhen Financial Zone and Shanghai Financial Zone to provide financial-level data highly reliable message queuing service.

CMQ currently provides access to HTTP (S) and TCP Protocol. Provide SDK access in rich languages such as PHP, Java, Python, etc.

Ingestion mode HTTP (S) access TCP access
Scene oriented Provides synchronous access based on HTTP (S) Protocol, and supports simple and convenient access to Restful API and multilingual SDK. Provide TCP synchronous / asynchronous access mode, support multilingual SDK, to improve the efficiency of production side and consumer side, and provide higher performance message queue service.
Bright spot ability The news is unlimited heap, the horizontal expansion of financial-level news is highly reliable, and the news is down in real time. TCP asynchronously non-blocking way to send and receive messages to improve efficiency. The news is unlimited heap, the horizontal expansion of financial-level news is highly reliable, and the news is down in real time.

?

  • The CMQ TCP access method is under internal testing. Ticket is welcome to contact the after-sales engineer. There will be a special person to provide Enable internal testing service for you.
    1. It supports the customized deployment. You are most welcome to contact our after-sales engineers with your ticket and your personal engineer will be here to enable the internal trial service for you.

Overview of Application Scenarios

CMQ is recommended for application scenarios in which asynchronous communication is required. We take the following examples for illustration.

In applications where reliable delivery of a message needs to be ensured, even if the receiver is unavailable for reasons such as power outage, crash or CPU overload when the message is being sent from the sender, receiver can still receive the message once becoming available. This function is not available in traditional message queues since it only stores messages in memory. Tencent Cloud CMQ enables messages to be stored persistently until the message is acquired by the receiver successfully.

A message queue should run normally even if the number of messages stored in the queue and visits keep increasing. The traditional message queue stores messages in local memory. Since the processing capability of stand-alone computers and memory capacity are limited, the queue is not scalable. In contrast, Tencent Cloud CMQ, featuring distributed architecture, can ensure the simple scalability of the queue and the scalability is completely visible to CMQ users.

Two services need to communicate with each other in the case that the networks cannot be interconnected or the routing information (such as IP and port) of applications is not known. For example, if two services on Tencent Cloud need to communicate while they do not know each other's address, the communication can be achieved in such a way that one party sends messages to and the other party receives the messages from the queue agreed upon by both parties.

Communication between system components or applications is frequent so that the components or applications need to maintain their network connections, and there are multiple communication contents. In such a scenario, traditional architecture will complicate the system design. For example, when a central processing service needs to assign tasks to multiple task processing services (similar to the master-worker mode), the master needs to maintain connections with all the workers and determine whether the workers have started to process the tasks so as to decide whether to reassign tasks. At the same time, the workers need to report task processing results to the master. Maintaining such a system at the same level will complicate the design, thus increasing the implementation difficulty and maintenance cost. As shown below, Tencent Cloud CMQ can reduce the coupling between the two sides, making the system simpler and more efficient.

The coupling between system components or applications is tight and needs to be reduced, especially when the controllability over its components is relatively weak. For example, when a company's business CGI receives user submissions, it will store some data in its own system and forward the processed data to other business applications (such as data analysis system, data storage system, etc.). The traditional solution is to establish a connection between services through the socket. In this way, if the receiver's IP or port changes, or the receiver changes, the data sender needs to modify the relevant information accordingly. But with Tencent Cloud CMQ, the sender and the receiver do not know each other's information, thus greatly reducing the coupling between them.