Migrations / Storage

Database migrations and storage management in Ecotone PHP

Ecotone provides comprehensive migration commands to manage your storage infrastructure, giving you full control over when and how storage resources are created or removed.

Automatic Initialization (Default)

By default, Ecotone automatically creates required database tables and message channels on demand (first use). This means you can start using Ecotone right away without needing to set up any infrastructure yourself - tables are created when you first store a document, queues are declared when you first send a message, and so on.

This approach works great in most scenarios, especially during development and in applications where initialization state can be cached between requests.

When to Use Migration Commands

In some scenarios, you may want to take over the initialization process however:

• Performance-sensitive HTTP applications - If we send messages to a broker with each HTTP request, then as each request is clear state, automatic initialization will continuesly re-try to create the resource
• Strict deployment processes - Some applications follow a process where all migrations must be completed before starting the application
• Visibility and control - You want to see what storage resources exist and their initialization state
• Resource cleanup - You need to delete storage resources when features are no longer needed
• Integration with existing tools - You want to generate SQL statements for use with Doctrine Migrations, Laravel Migrations, or similar tools

For these cases, Ecotone provides migration commands to take over the process.

Overview

Ecotone offers two types of migration commands:

1. Database Migrations - Manage database tables for features like deduplication, dead letter, document store, database message queues, event sourcing, and projections
2. Channel Migrations - Manage message channel infrastructure (queues, exchanges, topics) for brokers like RabbitMQ, SQS

Disabling Automatic Initialization

Before using migration commands, disable automatic initialization. This way we disable automatic creation of tables and channels.

Database Tables

final readonly class EcotoneConfiguration
{
    #[ServiceContext]
    public function dbalConfiguration(): DbalConfiguration
    {
        return DbalConfiguration::createWithDefaults()
            ->withAutomaticTableInitialization(false);
    }
}

Message Channels

final readonly class EcotoneConfiguration
{
    #[ServiceContext]
    public function channelConfiguration(): ChannelInitializationConfiguration
    {
        return ChannelInitializationConfiguration::createWithDefaults()
            ->withAutomaticChannelInitialization(false);
    }
}

Database Migration Commands

Database migration commands require the DBAL module to be installed.

Viewing Status

Symfony

# Show all features and their status
bin/console ecotone:migration:database:setup

# Show only used features (default)
bin/console ecotone:migration:database:setup --onlyUsed=true

# Show all available features
bin/console ecotone:migration:database:setup --onlyUsed=false

# Show status for specific features
bin/console ecotone:migration:database:setup --feature=dead_letter --feature=deduplication

Laravel

Lite

Output displays three columns: Feature name, whether it's used, and whether it's initialized:

+----------------+------+-------------+
| Feature        | Used | Initialized |
+----------------+------+-------------+
| dead_letter    | Yes  | No          |
| deduplication  | Yes  | Yes         |
| document_store | No   | No          |
+----------------+------+-------------+

Initializing Tables

Symfony

# Initialize all used features
bin/console ecotone:migration:database:setup --initialize=true

# Initialize specific features only
bin/console ecotone:migration:database:setup --feature=dead_letter --initialize=true

# Initialize all available features (including unused)
bin/console ecotone:migration:database:setup --initialize=true --onlyUsed=false

Laravel

Lite

Getting SQL Statements

If you want to integrate with existing database migration tools (like Doctrine Migrations or Laravel Migrations), you can get the raw SQL statements:

Symfony

# Get SQL for all used features
bin/console ecotone:migration:database:setup --sql=true

# Get SQL for specific features
bin/console ecotone:migration:database:setup --feature=dead_letter --sql=true

Laravel

Lite

Deleting Tables

Symfony

# Preview what would be deleted (dry run)
bin/console ecotone:migration:database:delete

# Delete all used features
bin/console ecotone:migration:database:delete --force=true

# Delete specific features
bin/console ecotone:migration:database:delete --feature=dead_letter --force=true

Laravel

Lite

The --force flag is required to actually delete tables. Without it, the command only shows what would be deleted.

Available Database Features

Feature	Description
dead_letter	Failed message storage for error recovery
deduplication	Message deduplication tracking
document_store	Document store persistence
message_queue	DBAL-backed message queue
event_streams	Event sourcing stream storage
projection_state	Projection state tracking
projections_v1	Legacy projection tables

Channel Migration Commands

Message channels for brokers like RabbitMQ, SQS, and Kafka can be managed via migration commands.

Not all message channels require migration commands:

• Redis - Does not require any resources to be created beforehand. Channel creation is fully automatic as it simply inserts values under given keys.
• DBAL (Database) - Message channels backed by database are handled through the database migration commands using the message_queue feature. There is no need to use channel migration for DBAL channels.

Channel migration commands are primarily useful for RabbitMQ (queues, exchanges) and SQS (queues, topics) where infrastructure must be declared before use. Kafka does not have supoort for migrations at this moment.

Viewing Status

Symfony

# Show all channels and their status
bin/console ecotone:migration:channel:setup

# Show status for specific channels
bin/console ecotone:migration:channel:setup --channel=orders --channel=notifications

Laravel

Lite

Output displays two columns: Channel name and whether it's initialized:

+---------------+-------------+
| Channel       | Initialized |
+---------------+-------------+
| orders        | No          |
| notifications | Yes         |
+---------------+-------------+

Initializing Channels

Symfony

# Initialize all channels
bin/console ecotone:migration:channel:setup --initialize=true

# Initialize specific channels
bin/console ecotone:migration:channel:setup --channel=orders --initialize=true

Laravel

Lite

Deleting Channels

Symfony

# Preview what would be deleted (dry run)
bin/console ecotone:migration:channel:delete

# Delete all channels
bin/console ecotone:migration:channel:delete --force=true

# Delete specific channels
bin/console ecotone:migration:channel:delete --channel=orders --force=true

Laravel

Lite

Only channels that support managed initialization will appear in migration commands. In-memory channels and channels without explicit infrastructure requirements are not managed by migrations.

Command Reference

Database Commands

Command	Options	Description
ecotone:migration:database:setup	--features, --initialize, --sql, --onlyUsed	View status, initialize tables, or get SQL
ecotone:migration:database:delete	--features, --force, --onlyUsed	Delete database tables

Channel Commands

Command	Options	Description
ecotone:migration:channel:setup	--channel, --initialize	View status or initialize channels
ecotone:migration:channel:delete	--channel, --force	Delete message channels

Testing

For testing purposes, you can use the same migration commands with Ecotone Lite:

use Ecotone\Lite\EcotoneLite;
use Ecotone\Dbal\Configuration\DbalConfiguration;
use Ecotone\Messaging\Channel\Manager\ChannelInitializationConfiguration;

$ecotone = EcotoneLite::bootstrapFlowTesting(
    containerOrAvailableServices: [
        DbalConnectionFactory::class => $connectionFactory,
    ],
    configuration: ServiceConfiguration::createWithDefaults()
        ->withExtensionObjects([
            DbalConfiguration::createWithDefaults()
                ->withAutomaticTableInitialization(false),
            ChannelInitializationConfiguration::createWithDefaults()
                ->withAutomaticChannelInitialization(false),
        ]),
);

// Initialize storage in test setup
$ecotone->runConsoleCommand('ecotone:migration:database:setup', ['initialize' => true]);
$ecotone->runConsoleCommand('ecotone:migration:channel:setup', ['initialize' => true]);

// Run your tests...

// Clean up in teardown
$ecotone->runConsoleCommand('ecotone:migration:database:delete', ['force' => true]);
$ecotone->runConsoleCommand('ecotone:migration:channel:delete', ['force' => true]);