Rascal has the following two concepts for retries and redeliveries:
retry/recovery: this happens when a message handler did throw an error that
got caught and a retry is done based on a configured recovery strategy.
This scenario is fully managed by Rascal code.
redeliveries: this concept is when a message was received from RabbitMQ and
an error could not be caught and the application crashes. The message
counter implementation tries to solve this issue.
When the application crashes Rascal is not able to start a retry mechanism,
add some message headers, or track something elsewhere.
RabbitMQ marks a message that was delivered but then the channel closed
without the consumer sending a nack or ack with a boolean "redelivery" flag.
The message is then made available again - meaning it is at the head of the
queue (not sent to the back).
When the application is restarted, rascal will thus get the message again.
Only on this second attempt this redeliveries counter will be called from
rascal. The counter will then track for any further attempt to process the
same message an incremented redelivery count in a PostgreSQL database table.
After the maximum amount of retries is reached, rascal moves the message to
the configured dead letter queue and the app should be able to run again.
Crashing applications should normally not occur due to message handling as
Rascal is using a very robust approach. But some messages may be too big and
cause out-of-memory exceptions or other errors. Those messages are called
poisonous messages.
To test that the message redeliveries counter works add the following code to
any message handlers onMessage function to trigger an application crash
by throwing an exception that is not caught.
When the message handler is triggered by an incoming message the application
will crash. After restarting the application five times, rascal moves the
message to the dead letter queue.
The following settings are managed by the message-bus library and should not
be altered from customizable code. Each Rascal subscription configuration
defines the redeliveries counter with the name 'mosaic':
PostgreSQL messaging counter for Rascal redeliveries
Rascal retry and redelivery concept
Rascal has the following two concepts for retries and redeliveries:
Redeliveries
When the application crashes Rascal is not able to start a retry mechanism, add some message headers, or track something elsewhere. RabbitMQ marks a message that was delivered but then the channel closed without the consumer sending a nack or ack with a boolean "redelivery" flag. The message is then made available again - meaning it is at the head of the queue (not sent to the back). When the application is restarted, rascal will thus get the message again. Only on this second attempt this redeliveries counter will be called from rascal. The counter will then track for any further attempt to process the same message an incremented redelivery count in a PostgreSQL database table. After the maximum amount of retries is reached, rascal moves the message to the configured dead letter queue and the app should be able to run again.
Crashing applications should normally not occur due to message handling as Rascal is using a very robust approach. But some messages may be too big and cause out-of-memory exceptions or other errors. Those messages are called poisonous messages.
To test that the message redeliveries counter works add the following code to any message handlers
onMessage
function to trigger an application crash by throwing an exception that is not caught.When the message handler is triggered by an incoming message the application will crash. After restarting the application five times, rascal moves the message to the dead letter queue.
Rascal configuration
The following settings are managed by the message-bus library and should not be altered from customizable code. Each Rascal subscription configuration defines the redeliveries counter with the name 'mosaic':
And the rascal configuration contains the following section in order to use the PostgreSQL based counter:
When creating a message-bus BrokerProxy, the counter must be added as part of the components parameter with
postgresCounter
as counter name: