Online Picture Processing

Last updated: 2020-02-26 20:08:29


Meitu set up an online photo processing service in Tencent Cloud, which allows users to upload photos and specify the actions to be performed on them, such as cropping, red-eye processing, tooth whitening, recoloring, contrast adjustment, generating thumbnails and so on. After the user uploads the picture, submit it to job, then wait for the picture to be processed, and then download the processed picture. Different operations take different processing time, ranging from a few seconds to a few minutes, and users may upload several pictures at a time, dozens or even hundreds of pictures at a time, so the total processing time is related to the number of pictures uploaded, the size of the pictures, and the actions chosen by the user.

Tencent Cloud CMQ meets the above requirements: The user's images are stored in Tencent Cloud storage (such as CBS/COS); each operation request is saved as a message (image index, consisting of the image name + the operation type requested by the user + the index key of the image storage location) into the Request Queue.

The image processing service on the CVM receives the message (image index) from the Request Queue. The image processing server downloads the images from the cloud, edits them, sends the processing results to the Response Queue, and stores the edited images in the cloud storage.

Tencent Cloud CMQ demonstrates high scalability and reliability under the following circumstances:

  • If the image processing service is temporarily unavailable due to bug or other reasons. However, the system uses CMQ to make the error transparent to users. On the one hand, users can continue to upload photos, web server can continue to send messages to Request Queue, and messages will be saved in the queue until the picture processing service is available; on the other hand. The image processing service does not have to remember the messages that were processed before the crash, and the messages processed at the time of the crash can also be reprocessed. Because the characteristics of receiving messages (including receiving sequential queue messages and receiving Concurrence queue messages) provided by CMQ ensure that the message remains in the queue after it is received until Explicit, the recipient of the message, deletes it. This feature ensures the decoupling of image processing service and picture upload service.

If a single image processing service cannot meet the needs of users (or users have to wait for a long time to get the processing results after uploading the images), CMQ plus multiple image processing services will be able to meet the growing user access demands.

1) A CMQ queue can be accessed by multiple servers to send, receive and delete concurrent queued messages at the same time;
2) A message can only be received by one server at a time and will be temporarily locked. The recipient of the message can specify the time when the message is locked, and delete the message after processing it. If the recipient fails to process the message, another service can get the message after the lock of the message expires.

These two features ensure that the number of processing servers can be dynamically increased or decreased as the load changes.