Publisher Subscriber design patterns are an invaluable tool for building enterprise grade .NET/C# applications. Among them you will be very familiar with the Publish-Subscribe pattern also known as Pub/Sub. Just to refresh your memory, Pub/Sub is basically a messaging pattern where the senders of messages (publisher) do not have any knowledge about the intended recipients as in, which applications are the receivers or how many of them are there. Also, the publishing and listening application do not interact with each other directly but instead depend on a common medium, without knowledge of the producer or consumers presence.
Businesses today are developing high traffic ASP.NET web applications that serve tens of thousands of concurrent users. To handle this type of load, multiple application servers are deployed in a load balanced environment. In such a highly concurrent environment, multiple users often try to access and modify the same data and trigger a race condition. A race condition is when two or more users try to access and change the same shared data at the same time but end up doing it in the wrong order. This leads to high risk of loosing data integrity and consistency. This is where distributed lock mechanism comes in very handy to achieve data consistency.
ASP.NET web applications, .NET web services applications, and other .NET server applications need to handle extreme transaction loads without slowing down. And, although their application tier scales linearly, the data storage and database tier does not scale and therefore becomes a bottleneck. As a result, the entire application cannot scale.
Originally, simple in-memory distributed key-value stores like Memcached and later Redis were introduced on Unix/Linux platforms to help resolve this scalability problem. They quickly became quite popular mainly because they provided linear scalability just like the application tiers and removed the database bottleneck.
A Look Back and Our Future Vision
With enthusiasm and stars in our eyes, NCache was launched in July 2005. And wow have we all come a long way since then! We’ve grown up with the .NET community and its adoption of caching solutions. Back then, caching was pretty new for .NET. I remember we wrote articles, created slideware, went to shows, meetups and prospects everywhere educating developers and managers about the benefits of caching, how to architect it and what it can do for the enterprise. At the time, .NET itself was not used as much in high transaction, high traffic applications as it is today. But it took hold.
Update: Microsoft has extended the AppFabric support to April 11th 2017. The extension is only applicable for AppFabric 1.1 and not for 1.0. For more information check out their blog post.
April 2nd 2016 (April 11th 2017) is proving to be an important date for Alachisoft’s product NCache. This is the date when Microsoft ends AppFabric support – essentially withdrawing the product from the market. See details on Microsoft AppFabric 1.1 for Windows Server Ends Support 4/2/2016
AppFabric was Microsoft’s answer to .NET industry’s need for an in-memory distributed cache to provide performance and scalability boost for .NET server applications. But, first it was late to the party with AppFabric 1.0 released in June 2010 (five years after NCache). And, second, it struggled to provide the features and functionalities considered de rigueur for in-memory distributed caches. Eventually, Microsoft decided to pull the plug on a product that essentially provided less features than other .NET caching solutions like NCache.
Read AppFabric vs NCache comparison and see for yourself how much AppFabric lacked in features and capabilities.
Modern day .NET applications must be well equipped to cope with the scalability demands being put on them due to the explosion of ASP.NET web applications reaching millions of users, WCF web services applications handling Internet of Things (IoT) devices, and Big Data analytics applications processing huge amounts of data. Scalability here means to maintain the same high performance under extremely heavy loads that you see with no load.
The architectures of these .NET applications are very scalable because you can add more app servers as you need higher transaction loads. But, the problem is with data storage, especially relational databases like SQL Server and Oracle, and mainframe. Basically, data storage is unable to scale in the same manner as the application tier.
Some of the very basic needs for .NET applications to stay competitive in today’s market are to be extremely responsive and scalable. The bottleneck in the way of achieving these benchmarks is your relational database.
This is a two fold bottleneck, first the reads from disk are very inefficient and time consuming. Secondly, you cannot scale out the database tier by adding more database servers. Whereas if you have a .NET distributed cache, it provides fast data access because it is in-memory and it can also scale linearly the same way your application tier does.
The database tier based on traditional RDBMS has proven to be the biggest bottleneck in the way of achieving competitive response times for applications. This has forced the application vendors to look for alternatives that can provide improved performance. One such alternative is storing data in a distributed cache.
From the available cache technologies you need to pick one that answers most if not all of the major questions asked in this domain. Going forward I will be comparing two products in this arena, Couchbase and NCache – Open Source Distributed Cache.
Looking for Appfabric Alternatives?
Today’s applications need to scale and handle extreme data loads due to explosion of web technologies and Internet of Things. The biggest bottleneck in the way of achieving this is the data tier with relational databases. The need of the hour is to be able to make this data available to applications in a faster and more efficient way.
You can achieve this by using distributed caching of data that facilitates faster response times. There are a few big names in this domain with less than competitive products, e.g. Microsoft with their AppFabric caching product. In contrast to this NCache is an Open Source Distributed Cache that offers cutting edge technology and features.
Redis is an in-memory key value store developed in C/C++ with clients for various programming languages like .NET, Java and C. Redis has features to tackle critical issues but falls short in some fundamental aspects. NCache – a .NET distributed caching solution on the other hand when compared with Redis answers all these concerns effectively and also provides additional features that are absent in Redis making it an ideal replacement.
The demand for high speed data access along with its integrity and fault tolerance is ever increasing in the contemporary application arena. Traditional disk based RDBMS systems have failed to answer these concerns in a comprehensive manner. This paved way for in-memory data solutions that cover all the bases listed above.