Basic knowledge of message queuing

Scenarios used by message queues: asynchronism, decoupling and peak clipping.

Asynchronous: For example, in an approval scenario, multiple steps need to be completed in one process. For example, after approval/rejection, messages need to be sent to inform, handle transactions and notify downstream systems. If these steps are synchronous, the whole call chain will be too long and take too long, which will affect the overall performance. Therefore, message notification, proxy and steps downstream of notification can be processed asynchronously.

Decoupling: If multithreading is used for asynchronism, it will lead to coupling with downstream systems. For each system, an interface call will be added and the system will be republished. The message queue is used to publish messages to the message queue, and the downstream system directly monitors the message-aware approval progress of the approval process to achieve the purpose of decoupling.

Peak clipping: mainly to deal with the impact of sudden flow on the system, resulting in system overload. Message queue can control the consumption speed and consume according to the processing capacity of the system, such as seconds kill system.

Jms: Java message service

Jms is the abbreviation of java message service, and jms clients can use jms service for asynchronous message transmission. ActiveMQ is implemented based on JMS specification.

Jms supports two message models:

Five different message body formats of JMS:

AMQP: Advanced Message Queuing Protocol.

Contrast:

The selection needs comprehensive consideration: reliability, performance, throughput, message sequence, whether messages will be lost, etc.

Repeated consumption: If consumers consume the same message repeatedly, it will lead to abnormal business. In order to solve this problem, we need to ensure that the interfaces are idempotent to eliminate the problems caused by repeated consumption. Idempotent means that the same operation is executed many times, with the same effect as the first execution.

For strong verification scenarios, we can add a journal reconciliation table to ensure idempotency, that is, the journal reconciliation table is used to record consumption messages. If the message appears again, first compare whether there is consumed data and return it directly.

For the weak inspection scenario, we can compare and judge the cache.

Sequential consumption: in the case of sequential consumption of multiple messages from multiple consumers in the same scene, such as adding, modifying and deleting operations, it is necessary to ensure sequential consumption. Or the start, pass and end messages of the approval process need to be strictly sorted. The queue mechanism in RocketMQ theme can guarantee FIFO, which is realized by sending messages to the same queue synchronously. Here, only orderly delivery is guaranteed, and orderly consumption is guaranteed by consumers themselves.

Distributed transaction: A transaction is the guarantee for a series of operations to succeed or roll back. A distributed transaction is a system in which a series of operations guarantee success or failure.

Common distributed transaction solutions: