Outbox Pattern

Outbox Pattern
Whenever we use more than one storage during single action, one storage may fail.
If that will be the case state will be correct only partially or we will lose the information completely.
This may happen for example, when we are storing data changes and then publish messages to Message Broker.
For critical parts of the system we want to have assurance that no message will be lost.
To solve this Ecotone implements Outbox pattern like solution.

In order to make sure your messages are stored together with your changes we need to set up Dbal Module.
Dbal Message Channel
By sending asynchronous messages via database, we are storing them together with data changes. This thanks to default transactions for Command Handlers, commits them together.
#[ServiceContext]
public function databaseChannel()
{
    return DbalBackedMessageChannelBuilder::create("async");
}
Asynchronous Event Handler
#[Asynchronous("async")]
#[EventHandler(endpointId:"notifyAboutNeworder")]
public function notifyAboutNewOrder(OrderWasPlaced $event) : void
{
    // notify about new order
}

After this all your messages will be go through your database as a message channel. 
Scaling the solution
One of the challenges of implementing Outbox pattern is way to scale it. When we start consume a lot of messages, we may need to run more consumers in order to handle the load.
Publishing deduplication
In case of Ecotone, you may safely scale your Messages Consumers that are consuming from your Dbal Message Channel. Each message will be reserved for the time of being published, thanks to that no duplicates will be sent when we scale.
Handling via different Message Broker
However we may actually want to avoid scaling our Message Consumers, as this may put more load on the database. 
For this situation Ecotone allows to make use so called Combined Message Channels.
In that case we would run Database Channel only for the outbox and for actual Message Handler execution a different one.
This is powerful concept, as we may safely produce messages with outbox and yet be able to handle and scale via RabbitMQ SQS Redis etc. 
#[Asynchronous(["database_channel", "rabbit_channel"])]
#[EventHandler(endpointId: 'orderWasPlaced')]
public function handle(OrderWasPlaced $event): void
{
    /** Do something */
}
database_channel is Dbal Message Channel
rabbit_channel is our RabbitMQ Message Channel
Then we run one or few Message Consumers for outbox and we scale Message Consumers for rabbit.
Combined Message Channels with reference
If we want more convient way as we would like to apply combined message channels on multiple Message Handlers, we may create an reference.
#[ServiceContext]
public function combinedMessageChannel(): CombinedMessageChannel
{
    return CombinedMessageChannel::create(
        'outbox_sqs', //Reference name
        ['database_channel', 'amazon_sqs_channel'], // list of combined message channels
    );
}
And then we use reference for our Message Handlers.
#[Asynchronous(["outbox_sqs"])]
#[EventHandler(endpointId: 'orderWasPlaced')]
public function handle(OrderWasPlaced $event): void
{
    /** Do something */
}

Last modified 1mo ago