I have written on a number of occasions on the way BLOBs are treated by the SharePoint/SQL Server combo and why it is important to separate the BLOBs out from the content metadata. One aspect of BLOB management is their storage which is effectively handled by externalizing them using StorageEdge. Another important aspect of this management is to minimize the access latency of these large chunks of unstructured data when accessed from SharePoint.
A typical SharePoint installation might grow as much as 10 TBs or more with the passage of time. There are a number of tasks that require movement of BLOBs in various directions. These range from the very trivial single-item access from within SharePoint to the much more complex such as BLOBs archiving, and linking legacy documents in SharePoint. Due to large size of unstructured BLOBs, access latencies hit the SharePoint performance badly. At times, even a trivial request such as for a single blob may result in multiple trips to the external storage. This becomes a matter of critical concern when a number of users access the storage media every time a frequently-used BLOB is required.
SharePoint blob caching is an important consideration when addressing the performance issues of blob access. SharePoint offers a very elementary form of blob caching which is disc based. It is restricted to only one WFE in the SharePoint infrastructure and does not go beyond its confines. But the reality tells us that BLOB access, with payload growing as high as terabytes, requires a much more sophisticated, distributed, in-memory, and fail-safe caching infrastructure.
StorageEdge provides precisely such a blob caching facility. It offers a unique combination of externalization and BLOB caching for effective blob management and access from a single interface. It makes use of NCache® technology to let you cache frequently used BLOBs in a distributed fashion with high availability, complete replication and failover. A distributed cache can give your SharePoint infrastructure a significant scalability boost because it keeps things distributed across multiple servers and still giving one logical view. But the cache actually lives on multiple servers and that’s what allows the cache to really scale.
NCache based caching in StorageEdge can be enabled using pretty inexpensive hardware. As a local cache on a WFE server, memory requirement is only 500 MB. For larger SharePoint install-base with BLOB size close to 10 TBs or more having 4-6 WFEs, caching can be plugged in using 02 inexpensive cache servers. In-memory cache is much faster than the disk based SharePoint BLOB cache that SharePoint provides.
Figure 1: Configuring SharePoint Blob Cache in StorageEdge
Here are some of the things you can do with SharePoint BLOB caching:
• Keep a subset of SharePoint BLOBs in client cache on WFE servers
• Specify max cache size and evict some BLOBs when cache is full
• Keep BLOBs fresh by expiring them in the cache
• Keep cache light by removing unreferenced BLOBs from cache
• Manage all SharePoint BLOB caching from a centralized web based admin app
• Monitor SharePoint BLOB cache activity in PerfMon