The Gateway & Multi-Site Distribution

The challenges of high availability have greatly
increased the number of multi-site deployments
for enterprise .NET systems.

Another benefit of multi-site distributions is
faster response times for users as the
servers and data they access is closer.

RPC technologies quickly run into trouble in these
environments as they make machines in the same site
and those in remote sites look the same.

Messaging works out better than RPC in these cases but still many developers mistakenly represent physical site boundaries as logical boundaries resulting in problems here as well. NServiceBus prevents developers from going down the wrong path but may leave them wondering how is multi-site communication handled.

Disaster recovery and physical sites

In some cases, physical sites are replicas of each other. This is a common configuration for the purposes of disaster recovery and is largely influenced by technology, cost, and performance.

NServiceBus provides no special facilities for disaster recovery other than to enable developers to plug in their own specific technologies into it. This can be of the form of database replication of subscription information, configuring MSMQ to store its message data on a SAN, etc. The difference in price and performance of the various options is quite large and will not be covered here.

The use of NServiceBus in logically significant physical sites is described in the next section.

Logically significant physical sites

While each branch of a bank or retail store has significance in each domain, when looking at the behaviors of each site we see a great deal of similarity even to the point of identical functionality. This may not be true across all sites, especially when looking at sites which serve as regional centers or headquarters.

The logical services that make up the business solution can have components installed at multiple physical sites - some of the components can be the same, others may be different. It is likely that multiple logical services in the same site will collaborate with each other quite closely, and possibly less closely than with their own components at other sites.

For example, we'd expect the Sales service in a store to talk to the Pricing service in the same store for every transaction. On the other hand, the Pricing service at the headquarters would likely do at most a daily push of updated prices to the stores. Similarly we'd expect an End-Of-Day push of transactions from the Sales service at each store to the headquarters.

This approach is not only the most common but is recommended for use in situations where physical sites have logical significance - keep all inter-site communication within logical service boundaries.

Intra-Service Cross-Site Messaging

Notice that when sites have logical significance, the messages passed between them are different than the messages sent within the site. The act of publishing prices from the headquarters has logical significance. The manager of a store explicitly performs an "end of day" operation after all cash has been collected from the tills and counted.

Therefore, design separate classes for those messages passed between sites.

Cross-Site Data Transfer

Depending on your network technology, you can choose to set up a virtual private network (VPN) between your sites. This provides Windows networking visibility of queues in the target site from the sending site. At that point, you'll likely be using standard NServiceBus APIs to direct messages to their relevant targets, in the form of Bus.Send(toDestination, msg);

This model is recommended as it provides all the benefits of durable messaging between machines with an unreliable connection between them between sites the same as within a single site.

You can find a great deal of information on setting up and managing a Windows VPN here.

In the cases where you only have access to HTTP for connection between sites, you can deploy the NServiceBus Gateway process on each site and it will transmit messages from a queue in one site to a queue in another site, including the hash of the messages to ensure that the message was transmitted correctly. The following diagram shows how it works:

The sending process in site A sends a message to the gateway's input queue. The gateway then initiates an HTTP connection to it's configured target site. The gateway in site B accepts HTTP connections, takes the message transmitted, hashes it, and returns the hash to site A. If the hashes match, the gateway in site B transmits the message it received to a configured queue. If the hashes don't match, the gateway in site A retransmits.

Configuration and Code

When configuring the client endpoint, make sure that in the UnicastBusConfig's MessageEndpointMappings element that you have an entry indicating that the relevant message types go to the gateway's input queue.

In addition to that, the client needs to tell the gateway to which site the message should be sent. This is done by setting the 'HttpTo' header on the message being sent like this:

Bus.Send<YourRequest>(
    m =>
     {
         m.SetHttpToHeader(/* Http address of the remote site's gateway */);
         // set other properties of the message
     });

In order for the receipient of the message to reply back to the initiating side, it needs to tell the gateway on its end to which URL to reply. So, in the code of the message handler on the receiving side, when you reply, do it like this:

Bus.Reply<YourResponse>(
    r =>
     {
         r.SetHttpToHeader(message.GetHttpFromHeader());
         // set other properties of the response
     });

Securing The Gateway With SSL

In order to provide data encryption for messages transmitted between sites, configure SSL on the machines in each site where the gateway is running.

Follow the steps for configuring SSL found on this page and then make sure to configure the gateway to listen on the appropriate port, as well as to contact the remote gateway on the same port.