Limitations of RBS FileStream Provider

As you know, SharePoint stores all its documents in SQL Server database as BLOBs. But, since SharePoint is a document-heavy product, the SQL Server database very quickly becomes too large to manage and also slows down SharePoint. To handle this, Microsoft first introduced EBS in SharePoint 2007 and eventually embraced this concept in SQL Server 2008 RS as RBS FileStream Provider which allows you to store BLOBs in an external storage and free up the database.

RBS FileStream Provider in SharePoint makes use of this SQL Server RBS API to externalize BLOBs. However, this SQL Server FileStream provider only satisfies a very basic set of needs of BLOB management and lacks many important functions that are essential for a real-life SharePoint deployment. Here are some limitations:

Optimize SharePoint Performance and Storage - Free Download

Limitations of FileStream provider with RBS

FileStream Provider only allows for storing BLOBs on the same location as your SQL Server data files and therefore does not reduce I/O burden. In fact, it slows things down even further in some cases.

  • FileStream Provider only offers a basic level of garbage collection.
  • RBS FileStream Provider doesn’t support multiple storage tiers to help you optimize your storage cost (see my blog on SharePoint Archiving with multiple storage tiers).
  • No compression or encryption options are afforded.
  • RBS FileStream Provider is only supported in SharePoint 2010.
  • Much more…

So before using the RBS FileStream Provider, you should be very careful in analyzing the costs of its limitations.

On the other hand, StorageEdge provides a much superior alternative to the RBS FileStream Provider. Here are some important benefits of using StorageEdge FileStream Provider.

Benefits of Using StorageEdge RBS FileStream Providers

You can externalize BLOB content on any SAN, NAS and even Cloud Storage.

  • StorageEdge RBS FileStream Providers also allows you to apply filters to the externalized content so you don’t need to externalize everything.
  • Encryption and Compression are also supported.
  • It supports throttling of various activities reducing their impact on performance in peak times.
  • It also provides a rich administration interface from within SharePoint.
  • Organize the externalized content in personalized folder structure.
  • File names are available in externalized content. This feature is not supported in RBS FileStream Provider. You don’t know the actual name of the externalized file on disk as it is always a GUID.

Thus StorageEdge ensures that the burden of managing BLOBs is effectively off-loaded from the SQL Server.

RBS_Filestream Provider

 In a nutshell, externalizing BLOBs is much more than simply using a SharePoint RBS FileStream provider. The solution you choose must provide all the required features to let you handle complex situations of a real-life SharePoint deployment.

Download StorageEdge | StorageEdge Details

Posted in SharePoint RBS, SharePoint Storage | Tagged , , , | Leave a comment

Link Legacy Documents to SharePoint without Copying

SharePoint, as the name suggests, means a place to share something. True to its name, SharePoint is essentially a document-centric collaboration platform, supporting sharing of documents, reports, calendars, lists, announcements and other publishable material. However, despite all the niceties, SharePoint has its fair share of limitations too. An important one of them is the way it manages unstructured content.

The content database of SharePoint in MS SQL Server holds all the unstructured content in the form of BLOBs. As BLOBs increase, the performance of SQL Server degrades considerably as it not optimized to carry such a large payload of unstructured data. BLOBs can have a real impact on SharePoint’s overall performance. Importing an existing cache of legacy content in a SharePoint infrastructure may result in further compromising the performance. Moving of terra-bytes of legacy content to SharePoint may take months to complete. Your content database becomes huge and unmanageable, and hence performance degrades.

Optimize SharePoint Performance and Storage - Free Download

So, when porting this legacy content to SharePoint, the organizations are presented with a two-faceted problem, the import of legacy content into SharePoint and then its storage. The latter can be tackled by off-loading the BLOB content to multiple, inexpensive tiers of storage, using an enterprise-grade RBS or EBS provider, such as the one provided by StorageEdge.

For the former, there can be multiple approaches. One crude approach can be to bring the content into the SharePoint content database first, store it as BLOB and then externalize it using either EBS or RBS. But as discussed earlier, the SQL Server is not optimized to work with such huge BLOB payload. This two-step approach shall bring in serious performance issues.

A better alternative can be to somehow be able to reference it in SharePoint, without actually moving it to the content database, and treating it as externalized content. This approach offers immediate benefits. No transactional overhead occurs on the SQL Server as no actual BLOB movement occurs. The database stays manageable in size. The expensive database storage is not utilized; rather the content stays at the inexpensive storage. Only relevant indexes are updated and the content gets automatically externalized.

StorageEdge affords an intelligent approach while importing legacy content. StorageEdge provides such tools which makes this content import real quick. Instead of physically moving the content from the legacy storage to database, it lets the data stay on the storage media and updates the relevant references and indexes. It even allows your application to use these documents from SharePoint during the import process.

StorageEdge's SharePoint legacy document importer

In addition, StorageEdge allows for the creation of multiple tiers of Storage, allow policies for retention and movement of content between the tiers, and apply various filters to the externalized content. It allows for throttling of migration activity making it efficient, without impacting the performance of in peak times.

Moving of unstructured legacy data into SharePoint can be a hugely resource-consuming activity if not managed properly. However, when done right, the legacy content can easily be made accessible through SharePoint without degrading the overall performance.

Download StorageEdge | StorageEdge Details

Posted in BLOB Externalization, SharePoint RBS | Tagged , , , , , | Leave a comment

Bandwidth Throttling Prevents SharePoint Slowdown in BLOB Externalization

I’ve already written about the need to externalize BLOBs in SharePoint and how it reduces your storage cost, improves SharePoint performance, and makes SharePoint database management simpler and easier. So, I hope you’re already convinced about the value of that.

But, if like most people your SharePoint content database is large, you will face the challenge that BLOB migration will consume all your SharePoint bandwidth and slow it down for your users. And, at the same time, you cannot afford to allocate a lengthy downtime just for BLOB externalization because your users need SharePoint all the time.

Optimize SharePoint Performance and Storage - Free Download

So, what you really want to do is to externalize the BLOBs but without impacting your users negatively. StorageEdge handles this situation through a feature called Bandwidth Throttling that makes sure SharePoint performance remains good during BLOB externalization.

Bandwidth Throttling lets you specify when BLOB migration should be a full speed (full throttle), when it should be totally stopped, and when it should be at medium speed. You schedule different time slots and for each slot you specify a different migration speed.  With this, you can stop the bandwidth consumption during peak hours or lower it down in normal day activity. And, you can increase the speed or the bandwidth consumption when it is off time or at nights and during weekends.

You can use Bandwidth throttling also for reverting BLOBs back into SharePoint database, for doing garbage collection, or for transferring BLOBs from one storage-tier to another. You can also use it when converting from EBS to RBS. Without Bandwidth Throttling, the slowdown in SharePoint can be very annoying for your users.

Below is a snap of the admin GUI page in StorageEdge that allows you to configure all of this.


Example of Time Slots for Bandwidth Throttling:

  • From 4pm to 9 pm, it is heavily used. So no data is being migrated.
  • From 9pm to midnight, it is busy. So you’ll do all the particular set  migration
  • From midnight to 8am, you will do 100% migration
  • From 8am to 4pm, you will do 50% migration because all the usage is there but it’s not so heavy.

Bandwidth throttling helps you in such a way that your users are unaware of the BLOB migration happening.  And, having a good response time for SharePoint is important and there is no reason why BLOB migration should slowdown SharePoint.

Download StorageEdge | StorageEdge Details

Posted in BLOB Externalization, SharePoint RBS | Tagged , , , , | 2 Comments

Reduce Cost with SharePoint Archiving

SharePoint Archiving is an attraction for SharePoint users. SharePoint is a document centric product and the number of documents grows very rapidly in medium to large size company. Even after externalizing all the documents or BLOBs, the amount of storage needed is very large and keeping all those documents in one storage location often increases the storage cost because that storage is typically very expensive high end storage.

In reality, most of these documents are not commonly accessed, especially those that become older versions of a document. Keeping such large and unnecessary content on this expensive storage is not a wise idea so you should consider moving it to cheaper storage. Regarding this problem, StorageEdge provides a SharePoint archiving solution “multi-tier storage” that allows you to keep your most recent or the latest documents in the most expensive storage and archives older content out to less expensive storage. This will help reduce your organizational cost as the most expensive storage is not overwhelmed with millions of documents.

Optimize SharePoint Performance and Storage - Free Download

StorageEdge does SharePoint archiving of these documents based on two criteria:
• Age
• Versioning

You can specify content age that allows your content to stay in a tier before it gets migrated to the next tier. This age can be specified separately for each storage tier and you can have as many tiers as you want.

The other criterion is based on document versioning. Whenever you update a document, SharePoint does not update the original document but instead creates a new version and save that. It preserves the older versions and when you update a document multiple times, you will end up creating many versions of it. Usually, everybody works on the latest copy of the documents so it is practical to archive the older versions to the cheaper storage tier and keep the latest version on your expensive storage tier. The subsequent tiers could be simply cheaper storage either locally in your LAN or Cloud storage. For Cloud Storage, you may want to control your bandwidth and for that StorageEdge provides bandwidth throttling feature that handles this automatically. I’ll discuss throttling in more detail later in a separate blog.

SharePoint Archiving
SharePoint Archiving

So by having multi-tier storage, you are able to reduce your storage cost. And, surprisingly, it also improves SharePoint performance because the most active storage (meaning your Tier-1) no longer contains all those huge amount of documents that would have overwhelmed it.

Download StorageEdge | StorageEdge Details

Posted in Multi-Tier Storage, SharePoint Archiving, SharePoint Storage | Tagged , , , , | 4 Comments

How to Optimize SharePoint Storage

It’s no secret that SharePoint is an extremely popular product because it offers really useful features for organizations that are trying to share documents and enhance collaboration. But, as you probably know, SharePoint content is by nature very document-centric because it is the place where people store a lot of their documents.

The problem is that all these documents (called BLOBs in SharePoint lingo) are stored inside the SQL Server database. And, SQL Server (like all relational databases) wasn’t designed to store BLOBs. It was designed to store structured relational data.

Optimize SharePoint Performance and Storage - Free Download

Additionally, the number of documents can grow rapidly over time and especially for medium to large size organizations. SharePoint content database size can easily grow into 100’s of GB and even into TB range.

Due to these reasons, SQL Server very quickly gets overwhelmed by so much BLOB data. This makes database management very difficult and also slows down SharePoint.

So to address these issues, the best thing to do is for you to externalize all the BLOBs, meaning move them out of the database and into an external storage. This will reduce your database size by almost 95% because all the documents are moved out and you’re only left with meta-data, list data, and other similar data.

You can externalize BLOBs to file system, NAS, SAN, or Cloud Storage but most people end up choosing either file system or NAS/SAN storage. For externalizing BLOBs, you have the following options in SharePoint:

  1. EBS (SharePoint 2007 & 2010): EBS is used for storing BLOBs outside of SQL Server. But, you need a third-party EBS provider since no built-in provider comes with SharePoint. StorageEdge includes an EBS provider that you can use.
  2. RBS (SharePoint 2010): RBS can only be used with SharePoint 2010 and SQL Server 2008 R2. And unlike EBS, SharePoint 2010 comes with a built-in RBS provider for file stream. But I would strongly recommend using a third-party provider like StorageEdge. StorageEdge goes well beyond what built-in RBS does and takes care of the whole life-cycle of BLOB externalization.

So, once you’ve decided to externalize your BLOBs, you need to decide whether to use EBS or RBS. I recommend using EBS with StorageEdge as your third-party provider. EBS is better because it is much simpler to use than RBS. RBS has to be installed as part of SQL Server 2008 R2 and the process is therefore more complex.

Please note that StorageEdge supports both EBS and RBS so if your heart is set on RBS, you can still use StorageEdge. I am just giving you my personal opinion about which path is better.

With StorageEdge, you can schedule a timed-job in SharePoint to initiate the migration of BLOBs out of your SQL Server. Secondly, once EBS or RBS is in-place, any documents that are created or modified by users at runtime are now stored automatically in an external storage rather than SQL Server. Here is how SharePoint looks with externalized BLOBs.

So, if you have a SharePoint with large amounts of BLOB data, take a look at StorageEdge. Below are some useful links for it.

Download StorageEdge |  StorageEdge Details

Posted in SharePoint RBS, SharePoint Storage | Tagged , , , , | 3 Comments