Microservices - part 3 - Service Location and Load Balancing

When we split our application into multiple services we need a way to locate an instance of a service.

TL;DR;

A component, or components, which handle the

location of service instance,
load balance requests to a service instance.

this can be done by a few different ways, including:

Dns lookup
Service locater + load balancer
Queues (such as rabbit)

note: the first 2 require something to maintain the registry.

Service location

With the of hosting multiple instances of a service, located 1 or more machines. We need to be able to route massages to the correct endpoint.

for example, here is the Api-Gateway requiring to route messages to services.

How does the Api Gateway know where the services are located.

A possible (limited) solution would be to assign Host entires to each service, however this will not solve the issue of multiple instances and load balancing (check out the DNS solution).

Here are some solutions

DNS

Using the DNS is a simple solution, where all the instances of a service register with the DNS.

Now when when we require the location of a service, we do a DNS lookup and it returns the location of all (active) instances of the service (multiple A records).

It is up to the us (the calling client) to randomly pick an entry.

Load-balancer + Registry

Another way is to combine a couple of components such as a Load-Balancer + Registry.

In this case the service instances will register with a Registry (ZooKeeper, Ectd, or a DNS for example). This is where the list of active services will be maintained.

Now each request will be routed through a load-balancer (HA proxy, Nginx). In this case we can make use of the loadbalancer stratgies (round robin or more advance ones), effectively removing the responsibility of routing from the client.

Queue Broker

With queues such as rabbit, you will naturally get service location (as it naturally routes messages), also you can achieve load balancing (with rabbit this is done via the pre-fetch).

By setting the number of messages each consumer can actively handle, the broker will now load balance messages (instead of sending all of them to single consumer instance).

Amqp as the loadbalancer + registry service lookup.

Side Note

With the first 2 options you actively need to keep the registry up-to-date, with the last solution this is automatically taken care of.

Off the shelf

Given the side note it may be worth investing in a mange solution such as:

these offer a solution towards the first 2 scenarios mentioned here, as well as keeping the registry up-to-date.

and for the 3rd scenario, consider using

RabbitMq

Dave R.