Pub/Sub Messaging in Cache: An Overview

This Publish/Subscribe (Pub/Sub) messaging paradigm provides an intermediary channel (called a topic) to exchange messages between multiple applications without any knowledge of the sender (publisher) and the receiver (subscriber). A publisher application sends messages to a subscriber application through topics.

Since all Pub/Sub models require a channel for communication, NCache acts as a medium for topics so that the publisher publishes the message to the topic. The subscribers receive the message via the topic as a notification. Using NCache as a medium for topics ensures loose coupling within the model and allows increased abstraction, providing additional benefits to the distributed topics.

Besides a distributed cache, NCache also provides a dedicated Pub/Sub Messaging cache.

Important

We recommend a dedicated Pub/Sub cache for the following reasons:

  • Eviction: If the cache contains messages and cache items, frequent eviction for cache items may also remove the messages before forwarding them to the subscriber.

  • State Transfer: State transfer adds a cost to each cache item operation. Since messages are frequently published, relayed, and expired, this recurrent activity can prove costly because it triggers state transfer.

Why Pub/Sub Messaging is Important

Real-time events require notification sharing among different applications in event-driven distributed architectures. Pub/Sub patterns enable publishers to share events with subscribers so that any desired processing takes place on the occurrence of an event of interest.

For example, a group of subscribers can be interested in information about order shipment details so that they can process it to track order delivery. Hence, they will subscribe to a topic that relays messages for order details. Once the publisher publishes a message to the topic, the subscribers will be notified and receive the message containing order details for further processing on their end.

Pub/Sub Components

The basic components of the NCache Publisher Subscriber Model are listed below and discussed in detail later:

  • Topic: A place where messages are published.

  • Publisher: An application (web application, desktop applications, microservices) that publishes messages to topic/topics

  • Subscriber: An application interested in receiving messages from topic/topics.

  • Subscription: An entity of interest created for a subscriber to receive intended messages.

  • Message: The actual data object the publisher sends and the subscribers receive via the topic.

Publish Subscribe in NCache

Topic

The publisher publishes messages on the topic. Subscribers subscribe to the topic to receive the message. This topic exists in a distributed form in NCache. Hence, its creation occurs on all the cluster nodes. It contains the message store, which stores the actual data objects the publisher publishes in a queue. It also internally maintains a list of all subscribers subscribing to it and a list of publishers.

The ITopic/Topic interface facilitates topic creation, topic getting, and topic deletion. You can also delete a topic asynchronously to avoid the wait for a topic to be deleted.

Note

The distributed nature of topics and underlying messages enhances scalability.

Once a message is published to the topic, it fires an event, which the topic relays to subscribers based upon the preference of message delivery option so that they can get the message as needed. The following figure illustrates the role of a topic as an intermediary channel for publishers and subscribers:

PubSub Topics in NCache

Note

In case of temporary subscriber disconnection and then auto re-connection, all information related to a topic such as subscriptions and failure event notifications is re-registered to the topic without any interruption on subscriber end.

Topic Priority

NCache introduces topic-level priority that helps you prioritize the lowest and high-priority topics based on their importance. You can create Topics with higher priority if they need to publish critical messages. So, those messages are delivered first. Similarly, you can create topics with a lower priority if they publish unimportant messages. The cache will evict these messages first.

Important

The priority of a topic can be specified at the time of topic creation only and cannot be modified afterward.

You can simply set the topic-level priority at the time of topic creation in order to prioritize the delivery of critical messages.

Topic Deletion Notification

All messages and related meta-info from the cache is deleted upon topic deletion. Hence, the subscriber and publisher receive notifications of this deletion because of the following reasons:

  • The subscriber might be waiting for incoming messages from the registered topic. Once the topic does not exist, the subscribers can then handle their execution accordingly through event notification and prevent an infinite waiting state.
  • The publisher can prevent sending messages to a non-existing topic and handle any pending messages and future execution accordingly.

To receive topic deletion notification, your application registers for the OnTopicDeleted event.

Publish Messages to a Topic

A publisher can publish messages to a topic by specifying topic name. Messages can be published asynchronously and in bulk to enhance the application performance. A partitioned cache distributes messages across the cluster. For message routing, every message has a unique id called MessageID at the publisher end, and the hash code of MessageID determines the cluster node to store the message.

Note

NCache allows publishing a bulk of messages in a single call to improve performance and memory usage.

NCache allows a publisher to use the following properties while publishing messages.

Important

Message delivery failure notification is only for the messages with expiration.

  • Topic deletion notification: A publisher can register for topic deletion notification to prevent undue publishing of corresponding messages once a topic no longer exists.

Subscribe to a Topic

A subscriber application can subscribe to topic messages by registering itself to a topic of interest through a subscription. In Pub/Sub, a subscription represents an interest of a subscriber(s) shown in a particular topic.

Note

In case a new node is added to the cache cluster and state transfer is triggered, all subscriptions along with the messages and cache data are replicated to the new node.

NCache allows to subscribe for a topic by providing topic name or a pattern. For further details, refer to Methods of Subscription. Moreover, you can also set message delivery mode as synchronous or asynchronous while creating a subscription.

NCache provides multiple types of Pub/Sub subscriptions which are discussed as follows.

Subscription and its Types

Subscriptions in NCache can be classified into following types:

Durable and NonDurable Subscriptions in NCache

Durable

Note

A durable subscription is a named subscription.

In a durable subscription, the cache ensures that the subscriber does not miss out on any message in case of disconnectivity due to application/machine shutdown, application restart or network failure. Hence, durable subscriptions don't suffer due to subscriber disconnection and re-connection.

Note

Durable subscriptions are not deleted unless the subscriber properly unsubscribes.

If a subscriber is disconnected, the messages intended for this subscriber are stored on a server until the subscriber rejoins or the messages are expired. Durable subscriptions are not deleted automatically on subscriber disconnection unless the subscriber has properly unsubscribed.

A durable subscription is further classified as:

  • Shared: A shared, durable subscription means that multiple subscribers share a named subscription. The messages assigned to a shared subscription are then load-balanced among the subscribers in a round-robin fashion. Even if any subscriber leaves the network, the messages continue their delivery to active subscribers. So, if a subscriber leaves gracefully or abruptly after an assignment, its assigned messages are reassigned to other active subscribers.

    Important

    A shared subscription is only supported by a Durable Subscription.

    In a shared subscription, the subscription will remain on the topic and cannot be unsubscribed until all subscribers have unsubscribed. This means as long as there is even a single active subscriber, the subscription will stay active.

  • Exclusive: An exclusive durable subscription means that there is only one active subscriber registered against a subscription at a time. If a subscriber gracefully unsubscribes, the exclusive subscription can be assigned to a new subscriber. If a subscriber leaves abruptly, the new subscription request is accepted after waiting for an idle time. The assigned messages always stay there even if there is no subscriber.

Non-Durable

In a non-durable subscription, the subscribers only receive messages intended for it for as long as it stays connected. If the subscriber leaves the network, it will receive no messages published during its disconnection period. A non-durable subscription is exclusive by default.

Important

Subscribers will lose messages if the subscriber restarts.

Also, non-durable subscriptions are automatically deleted when the subscriber leaves the network. This means that if that subscriber rejoins or establishes the connection again, it will be considered as a new subscription.

Expiration of the Durable Subscription

You can set expiration for a durable subscription. For instance, if there is no active subscriber for a significant time. Here subscriptions will expire after a specific period of inactivity. When a durable subscription is expired, messages assigned to this subscription are reassigned based on the DeliveryOption of these messages.

Note

Each time the subscriber polls or performs any other activity, the expiration time of subscription is reset.

Handling Inactive Subscriptions

Note

Only Non-Durable subscriptions are marked inactive and this does not apply to durable subscriptions.

In case there is no active subscriber against a subscription after waiting for a certain inactive period, the subscription is considered to be expired. If a subscriber rejoins after disconnection during the inactive period, it will be able to receive its assigned messages.

Important

Inactive subscriptions should be handled to avoid memory overload on server side.

After waiting for the inactive period, the subscription is expired and its assigned messages will be reassigned to other subscriptions. The reassignment of messages depends on the delivery option of messages.

  • If the message assigned to a subscription A had delivery option set to ALL and it is already assigned to another subscription B before the subscription A expires, the reassignment will not be required.

  • If the message assigned to a subscription had delivery option set to ANY and the subscription expires, the assigned messages will always be reassigned to any other subscription.

Message

A message contains the actual data object which is sent by the publisher and delivered to the subscribers through the topic. Once the publisher publishes the message to the topic, the registered subscribers are notified that a message pertaining to their interest has been published. In case of multiple messages, they are stored in a sequence inside the queue of a particular topic.

Note

The same message can be assigned to multiple topics. This is uniquely identified through an auto-generated ID.

Here, we further discuss message assignment to the subscription, message delivery and acknowledgement mechanism.

Message Assignment and Delivery to Subscription

Initially, the messages are unassigned on the server side. All subscriptions get messages according to the DeliveryOption. Once a subscription assignment occurs, the subscriber receives a notification. Then the subscriber implements a poll mechanism to bulk-fetch multiple messages - to reduce server-side overhead. After receiving the messages, the subscriber sends the acknowledgment, and the messages are considered delivered.

Note

Messages are stored in the topic if there is no subscriber to receive the messages. Messages are assigned and delivered to the first subscriber as soon as it subscribes to the topic.

Message Delivery Option

A publisher needs to specify the message delivery option to decide if a message will be delivered to a single subscriber or broadcasted to all subscribers at the time of publishing. It is important to note that, the definition of successful delivery depends on the specified delivery option.

The two delivery options and the corresponding successful delivery criteria are discussed as follows:

  • All: All registered subscribers receive messages. A message deletes when all the subscribers acknowledge it.

  • Any: Any single registered subscriber receives the message. If the assigned subscriber doesn't send acknowledgment, the messages get reassigned to the next subscriber. If they do send acknowledgment, the messages are considered successfully delivered.

Note

If a message is successfully delivered according to the specified delivery option, it is removed from the cache.

Message Storage and Distribution

Following are the important aspects of storage and distribution of messages in cache:

  • Messages are distributed among nodes based upon topologies.

  • For the Partition of Replica and Partitioned topologies, hash-based distribution is used.

  • For Replicated topology, messages are replicated to all nodes of the clustered cache. However, the coordinator node is responsible for message manipulation.

  • For Mirror topology, messages are published to the active node and then replicated to the passive node accordingly.

  • If the message store is near eviction, an event is logged indicating a full message store and the start of eviction.

  • Messages have an overhead on cache memory. Hence, message size must be accounted for while calculating the cache size.

Message Behavior

Here we discuss the expected behavior of messages in the following cases:

  • On clear cache: Messages are removed along with cache items once the cache is cleared.

  • On cache restart: Similar to clear cache, cache contents are cleared when the cache is restarted. This also includes all topics and the messages contained within them.

  • Eviction: If eviction is enabled on a Pub/Sub cache, the data will be evicted first and then messages will be evicted.

  • Encryption and compression: If encryption and compression is configured at cache level, it also applies to topic message payload.

  • State transfer: In case of state transfer, when messages move to another node in the cluster, the node where the message is finally stored is responsible for delivery.

Message Delivery Failure Notification

Important

Subscribers recieve a Delivery failure notification when a message having expiration expires before delivery to any of them.

In case a message does not get assigned or delivered to any of the subscriber, it is considered as failure. It may happen when the subscriber does not exist or is inactive due to network glitch. In such a scenario, a publisher can register message delivery failure notification. It is important to note that delivery failure notification is only for messages with expiration. Delivery of a message is considered to be failed when a message expires before delivery to any of the subscribers

Note

If there are multiple publishers against a topic, the failure notification will be delivered to any of the active publishers which has registered for the failure notification.

Message Expiration

Similar to cache items, a publisher can set expiration for messages. The messages will be expired from the cache as soon as the expiration interval has passed and uses the same cleaning interval mechanism.

The expiration of a message determines the time it persists in the cache. Regardless of delivery, if a message is not delivered, it will be removed after the expiration time. If a message is successfully delivered before expiration, it is removed from the cache without waiting for the expiration time.

Ordered Messages

NCache now supports ordered messages where the sequence of the messages is maintained on the client-side. A publisher can publish ordered messages by specifying a sequence name for a chunk of messages. Then the ordered messages are delivered to the subscribers in the exact same order they are published in. The sequence string should be the same for a chain of ordered messages. Using the sequence string, all the messages reside on the same node using the Location Affinity mechanism.

Note

Synchronous delivery mode can be used for ordered messages.

Following are the important characteristics of ordered messages:

  • Messages from a publisher with the same sequence reside on a single cache node.

  • If the DeliveryOption is set to Any, all the ordered messages of the same sequence are delivered to the same subscriber. In case the specific subscriber loses connection or becomes unavailable, a new subscriber is reassigned for this purpose. However, if the DeliveryOption is set to All, then all ordered messages of the same sequence are delivered to all the subscribers.

  • In case of state transfer, the ordered messages might lose their sequence and get published without keeping the order.

  • Ordered messages can only be published using the Sync mode in the Publish API. Bulk and Async API calls are not supported.

Monitoring

NCache provides you with the ability to monitor Pub/Sub Topic statistics and observe various performance counters in this regard. Pub/Sub topic activity and status can be monitored through the Windows PerfMon Counters and Windows PowerShell Cmdlet.

Reliability and High Availability

NCache implements the acknowledge mechanism for message delivery. Messages are kept in memory until they are delivered successfully based on the at least once delivery criteria. Hence, NCache ensures message delivery reliability for Pub/Sub Messaging in distributed architectures.

Moreover, fault tolerance of up to one node in partition-replica topology makes the Pub/Sub Messaging store highly available. If a node leaves the cluster for any reason, the replica has the message backup.

In This Section

Pub/Sub Topics
Explains how to create, get and delete topics for Pub/Sub model in NCache.

Publish Messages to a Topic
Provides sample code that creates a topic and publishes messages to it.

Subscribe to a Topic
Provides sample code to subscribe against a topic and receive messages of interest.

Pub/Sub Events
Explains Pub/Sub events to notify the publisher and subscriber about various events happening across the cache and applications.

Monitoring Pub/Sub Topics
Describes the ways Pub/Sub statistics can be monitored through NCache Monitor, PerfMon and PowerShell cmdlets.