Luiz Schons – Messaging

Event-driven Architecture (EDA)

Sat, 03 Aug 2024 00:00:00 -0300

Software architecture involves many trade-offs. A poor choice can lead to rework and delays, so it is important to know the available options and understand when each one makes sense.

In this article, we will talk about Event-driven Architecture (EDA), a style whose main characteristic is communication between services through events.

What is EDA?

Event-driven Architecture promotes the production, detection, consumption of and reaction to events. An event can be any fact relevant to the system: a state change, a user action, an error or a received message.

Events are asynchronous messages generated by a producer and consumed by one or more consumers. The producer does not need to know the consumers, and the consumers do not need to know the producers. This makes it possible to build systems that are more decoupled, scalable and flexible.

In this article’s example, we will simulate the following scenario:

Hands-on example

The system is made up of four services.

`ecommerce-frontend`

Frontend of an e-commerce store that lets you add products to the cart.
Uses React to build the user interface.

`stock-service`

Manages product stock.
Uses PHP to communicate with RabbitMQ.

A simplified example of publishing a stock event in PHP:

$message = new AMQPMessage(json_encode([
 'order_id' => 1,
 'status' => 'updated',
]));

$channel->basic_publish($message, '', 'stock_updated');

The code above is just a teaching example, not working or production-safe code.

`payment-service`

Processes order payments.
Uses Python to communicate with RabbitMQ.

def process_payment(order):
 print(f"Processing payment for order: {order['order_id']}")
 return True

`rabbitmq`

Message broker that enables communication between the services.
Receives published events and routes them to interested consumers.

The full code is available in the project repository.

Communication flow

The user opens the e-commerce frontend and adds a product to the cart.
The frontend sends a request to the stock service to check out.
The stock service checks availability and publishes an event to RabbitMQ.
The payment service consumes the event and processes the payment.
The payment service publishes an event with the result of the processing.

Trade-offs

This flow is a simple example of how EDA can decouple services and enable asynchronous communication without producers and consumers having to know each other. Each service is responsible for one task and can be replaced or updated with less impact on the others.

On the other hand, event-driven architecture increases operational complexity. You have to deal with ordering, delivery, idempotency, traceability and debugging of distributed problems.

So before adopting EDA, consider whether the benefits outweigh that complexity for your project.

Conclusion

Event-driven Architecture is a powerful alternative for building decoupled, flexible and resilient systems. There is no perfect architecture: the best choice depends on the project’s requirements, constraints and operational maturity.

The full code is available on GitHub.

Hexagonal Architecture and Messaging with PHP

Thu, 04 Jul 2024 00:00:00 -0300

Hexagonal Architecture (Ports and Adapters)

Project link: https://github.com/sschonss/microservices-hexagonal

There are many ways to organize a project, and one of the best known is hexagonal architecture. Hexagonal architecture organizes a project so that it is independent of frameworks, databases, UI and so on. The goal is for the project to be independent of any technology, so that each one can be easily replaced.

In this project, hexagonal architecture was implemented with the PHP programming language and the Laravel framework. The idea is for the project to be independent of Laravel, so we use Laravel only to create routes and controllers, and the rest of the project does not depend on it.

Project structure

The project was split into 3 main layers:

Laravel: All of the project's infrastructure, such as routes, controllers, etc.
Adapters: Layer that holds the project's input and output adapters.
Core: Layer that holds the project's business rules. Inside the core, we will work with Domain Driven Design (DDD).

Adapters

Adapters are responsible for adapting the project's inputs and outputs. But what does that mean?

Imagine you have a project that needs to talk to a database. Without an adapter layer, you would have to use the database directly in your project. That means using the database classes directly in your code, which makes your project dependent on the database.

An ORM (Object Relational Mapping) is an example of an adapter. The ORM is a layer that adapts communication with the database, so the project does not depend on the database itself.

In our project, we have the following adapters:

EmailAdapter: Adapter responsible for sending emails.
RabbitMQAdapter: Adapter responsible for sending messages to RabbitMQ.

They contain the classes that adapt communication with the email service and with RabbitMQ.

All of this uses plain PHP, without Laravel, so the project stays independent and each piece can be easily replaced.

Core

The core is the layer that holds the project's business rules. Inside the core, we will work with Domain Driven Design (DDD).

What is DDD?

Domain Driven Design is a way of organizing a project so that it is split into domains. Each domain is a part of the project that holds business rules. This keeps the project organized and easy to maintain.

Inside the core, we have the following domains:

Email: Domain responsible for the application's emails in general.
Connections: Domain responsible for connections to external services, such as RabbitMQ, Redis, etc.

Inside each domain, we have the layers that hold the business rules for each feature.

Most importantly, the core depends on nothing. It is independent of any framework, database, etc. That way, we can easily replace Laravel with another framework, or MySQL with another database.

The core talks to external services through the adapters. That way, the core does not depend on any external service, only on the adapter.

The project

The project is a simple example of how hexagonal architecture can be implemented. It is a user registration system, where users can sign up and receive a welcome email.

The project was split into 3 services:

Main App: The main service, where users sign up. This is where the hexagonal architecture lives.
RabbitMQ: The messaging service, to which the Main App sends a message.
Email App: The email service, which reads messages from RabbitMQ and sends an email to the user.

This way, the Main App does not depend on RabbitMQ or on the email service. It just sends a message to RabbitMQ, and RabbitMQ delivers the message to the email service.

How to run the project

To run the project, you need Composer, Docker and Docker Compose installed on your machine.

After installing Docker and Docker Compose, just run the following command:

cd main-app composer install ./vendor/bin/sail up

After running the command, the project will be running on port 80.

We can now create a user through the POST /api/user endpoint, sending the following JSON:

{ "name": "John Doe", "email": "john@doe.com", "password": "123456" }

After the user is created, an event is published to RabbitMQ, but the email service is not running yet. To run it, just run the following command:

cd email-app docker-compose up --build --force-recreate

After running the command, the email service will be running, reading messages from RabbitMQ and sending an email to the user.

Accessing RabbitMQ

To access RabbitMQ, just open the following address:

http://localhost:15672

And log in with the following credentials:

Username: guest Password: guest

Conclusion

Hexagonal architecture is a way of organizing a project so that it is independent of any technology. In this project, it was implemented with PHP and the Laravel framework. The idea is for the project to be independent of Laravel, so we use Laravel only to create routes and controllers, and the rest of the project does not depend on it.

The project is a simple example of how hexagonal architecture can be implemented. It is a user registration system, where users can sign up and receive a welcome email.

We also implemented a messaging service with RabbitMQ and an email service. This way, the Main App does not depend on RabbitMQ or on the email service. It just sends a message to RabbitMQ, and RabbitMQ delivers it to the email service.

We saw how messaging helps decouple services, and how hexagonal architecture helps organize a project so that it is independent of any technology.

Microservices and Messaging with PHP, NodeJS, RabbitMQ and Docker

Sat, 29 Jun 2024 00:00:00 -0300

Microservices development is an architectural approach that structures an application as a set of small, independent services, each running in its own process and communicating through lightweight protocols such as HTTP, WebSockets or AMQP.

In this article, we will build a microservices application with PHP, RabbitMQ and Docker. RabbitMQ is messaging software that implements AMQP (Advanced Message Queuing Protocol), an asynchronous messaging protocol.

What is RabbitMQ?

Imagine the following scenario: you have a system that needs to send emails to users. Instead of sending the emails directly, you can send a message to a message queue, which will be consumed by a service that sends the emails. That is what RabbitMQ does: it receives messages from producers and delivers them to consumers.

This decouples producing messages from consuming them, which lets you scale each part of the system independently. RabbitMQ also makes sure messages are delivered in the right order and are not lost.

Application architecture

Our application will have two microservices: a producer and a consumer. The producer sends messages to a message queue, and the consumer reads the messages and prints them to the terminal.

Our first step is to create the docker-compose.yml file to define our application's services:

services:
 rabbitmq:
 image: "rabbitmq:3-management"
 container_name: "rabbitmq"
 ports:
 - "15672:15672"
 - "5672:5672"

 php-app:
 build: ./php-app
 depends_on:
 - rabbitmq

 node-app:
 build: ./node-app
 depends_on:
 - rabbitmq

In this file, we define three services: rabbitmq, php-app and node-app. The rabbitmq service is based on the rabbitmq:3-management image, which includes the RabbitMQ management UI. The php-app and node-app services are based on custom images we are going to build.

Building the producer with PHP

Now let's create the php-app directory and a Dockerfile inside it:

mkdir php-app && touch php-app/Dockerfile

In the Dockerfile, we define the base image and copy the application files:

FROM php:7.4-cli

RUN apt-get update && apt-get install -y librabbitmq-dev libssl-dev git unzip wget curl

RUN docker-php-ext-install sockets pdo pdo_mysql
RUN pecl install amqp && docker-php-ext-enable amqp

COPY . /usr/src/myapp
WORKDIR /usr/src/myapp

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN composer clear-cache

RUN composer install

CMD ["php", "./publisher.php"]

In this file, we use the php:7.4-cli base image and install the dependencies needed for RabbitMQ and Composer. Then we copy the application files into /usr/src/myapp, install the Composer dependencies and run the publisher.php script.

Now let's create the publisher.php file at the project root:

require_once __DIR__ . '/vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

$maxRetries = 5;
$retryDelay = 5;

for ($attempt = 0; $attempt try {
 $connection = new AMQPStreamConnection('rabbitmq', 5672, 'guest', 'guest');
 $channel = $connection->channel();
 break;
 } catch (Exception $e) {
 echo "Failed to connect to RabbitMQ. Retrying in $retryDelay seconds...\n";
 sleep($retryDelay);
 }
}

if (!isset($connection)) {
 echo "Not possible to connect to RabbitMQ. Exiting...\n The application will be restarted by Docker\n";
 exit(1);
}

$channel->queue_declare('hello', false, false, false, false);

$msg = new AMQPMessage('Hello, RabbitMQ! Now is ' . date('Y-m-d H:i:s'));
$channel->basic_publish($msg, '', 'hello');

echo " [x] Sent 'Hello, RabbitMQ!'\n";

$channel->close();
$connection->close();

In this file, we open a connection to RabbitMQ, declare a queue called hello and send a message to it. Then we close the connection. Notice that we try to connect to RabbitMQ several times, waiting 5 seconds between attempts.

This matters to make sure the application can connect to RabbitMQ even if it is not available right away. It is a common practice in distributed applications, where service availability can vary over time; look up the circuit breaker and retry pattern patterns to learn more.

To install the RabbitMQ dependencies with Composer, we need to create a composer.json file at the project root:

{ "require": { "php-amqplib/php-amqplib": "^3.1", "phpseclib/phpseclib": "^3.0" } }

Building the consumer with Node.js

Now let's create the node-app directory and a Dockerfile inside it:

mkdir node-app && touch node-app/Dockerfile

In the Dockerfile, we define the base image and copy the application files:

FROM node:14 
WORKDIR /usr/src/app 
COPY package\*.json ./ 
RUN npm install 
COPY . . 
CMD \["node", "subscriber.js"\]

In this file, we use the node:14 base image, copy the application files into /usr/src/app, install the Node.js dependencies and run the subscriber.js script.

Now let's create the subscriber.js file at the project root:

const amqp = require('amqplib');

async function connectWithRetry() {
 const maxRetries = 5;
 const retryDelay = 5000;

 for (let attempt = 1; attempt try {
 const connection = await amqp.connect('amqp://rabbitmq');
 return connection;
 } catch (err) {
 console.error(`Not possible to connect to RabbitMQ. Retrying in ${retryDelay}ms. Attempt ${attempt} of ${maxRetries}`);
 await new Promise(resolve => setTimeout(resolve, retryDelay));
 }
 }

 throw new Error(`Failed to connect to RabbitMQ after ${maxRetries} attempts`);
}

async function receiveMessages() {
 try {
 const connection = await connectWithRetry();
 const channel = await connection.createChannel();
 const queue = 'hello';

 await channel.assertQueue(queue, { durable: false });

 console.log(" [*] Waiting for messages in %s.", queue);

 channel.consume(queue, function (msg) {
 console.log(" [x] Received: %s", msg.content.toString());
 }, {
 noAck: true
 });

 } catch (err) {
 console.error(err.message);
 process.exit(1);
 }
}

receiveMessages();

In this file, we open a connection to RabbitMQ, declare a queue called hello and consume the messages from it.

Then we print the messages to the terminal. Just like in the producer, we try to connect to RabbitMQ several times, waiting 5 seconds between attempts. This makes sure the consumer can connect to RabbitMQ even if it is not available right away.

And to install the RabbitMQ dependencies, we need to create a package.json file at the project root:

{ "name": "node-app", "version": "1.0.0", "description": "", "main": "subscriber.js", "dependencies": { "amqplib": "^0.8.0" }, "author": "", "license": "ISC" }

Running the application

Now that we have the RabbitMQ, producer and consumer services, let's run the application with Docker Compose:

docker compose up --build

This will create the service containers and run the application. You will see the messages sent by the producer printed to the terminal by the consumer.

RabbitMQ will also be available on port 15672, where you can open the management UI and see the queues and messages.

With that, we have built a microservices application with PHP, RabbitMQ and Docker. This architectural approach lets you scale each part of the system independently and guarantees that messages are delivered in the right order.

I hope this article was useful and that you can apply these concepts in your projects. If you have any questions or suggestions, leave them in the comments.

Feel free to reach out with any questions.

Repository: Microservices and Messaging with PHP, RabbitMQ and Docker