HTTP 404 error when opening document on restored and RBS enabled database

Today I want to make aware of a recent discovered “Documentation Issue on TechNet Article””!!

From a recent case, a customer reported an issue with an RBS enabled database, that was backed up and restored from a production to a test server.

ISSUE description:

You have a Production SharePoint 2010 Server on RTM (Build: 14.0.4763.1000) and a SQL Server 2008 R2.
You have RBS enabled on the Prod. Server and database and it is working fine!

You perform a backup on Prod. Server via central admin (SharePoint build-in backup) which finished successfully (Only some minor warnings in logs, because of having RBS enabled and additional steps have to be considered)

On Test Server (same status as Prod. Server), you also have installed and enabled RBS as before by following TechNet article “Install and configure RBS (SharePoint Server 2010)”
You then restored the taken backup files and also no major errors found in logs.

While restore, RBS was detected and SharePoint asked for the new path for file stream location, database names etc.

After restore finished, you browse to the document library and find all documents displayed in library.

Once you click one document to open, you receive an “HTTP 404 File not found” Error.

Analysis:

It turned out, that on our TechNet article “Install and configure RBS (SharePoint Server 2010)” a wrong link was pointing to an RBS installer file version, that is not supported anymore!
In this article, in the “Install the RBS client library…” section, the download link for the RBS_x64.msi file pulled a pre-RTM version of the installer (10.50.1352.12).

So regardless, if this caused the issue or not, however, this file is not supported and must not be installed at all!

While this post was written (2011-10-13), the above mentioned “wrong” links have been disabled, but on the TechNet post “Install and configure RBS (SharePoint Foundation 2010)” there are also wrong leading links! Please do not download or install these files! The “content issue” is already reported and the mentioned TechNet articles are expected to be modified as soon as possible!!
So please stay tuned and revisit the links frequently to check for any updates!

*** Update 2012-07-04 ***
Meanwhile the TechNet article has been revised and the “wrong links” are corrected now!
However, it is still an ongoing discussion whether we do support only the FEATURE PACK for RBS or whether the “Install and configure RBS (SharePoint Server 2010)” still applies as well.

To be sure, just use the below “feature pack for RBS” and you should be fine 😉

*** end of Update 2012-07-04 ***

The correct file and only supported with SharePoint can be found here:
SQL 2008 R2 feature pack – http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=16978

Note!

From Overview of RBS (SharePoint Server 2010):

SharePoint Server 2010 supports the FILESTREAM provider that is included in the SQL Server Remote BLOB Store installation package from the Feature Pack for SQL Server 2008 R2

This version of RBS is available at Microsoft® SQL Server® 2008 R2 Feature Pack . Be aware that this is the only version of RBS that is supported by SharePoint Server 2010. Earlier versions are not supported.

Conclusion:

I tried to reproduce the scenario but was using the correct RBS-Installer file versions from the Microsoft® SQL Server® 2008 R2 Feature Pack. I also followed the steps as described in Install and configure RBS (SharePoint Server 2010) and with some sneaks over from Todd Klindt’s post and on the very first attempt to open a document from the fresh restored content and RBS stream, I got an error but not with “404 file not found”. In my case I got “The internet addess ‘http://servername/Shared Documents/doc.docx’ is not valid.”

So I did an IIS reset, but still same error, then I just reapplied the SQL commands from the initial setup advise,

use [WSS_Content_Blob_001]
if not exists (select * from sys.symmetric_keys where name = N‘##MS_DatabaseMasterKey##‘)
create master key encryption by password = N‘Admin Key Password !2#4‘

use [WSS_Content_Blob_001]
if not exists (select groupname from sysfilegroups where groupname=N‘RBSFilestreamProvider‘)
alter database [WSS_Content_Blob]
add filegroup RBSFilestreamProvider contains filestream

———————
then I checked via powershell, that my filestream provider was set to “enabled=true”

… and oh wonder! It worked like a charm! entering my restored docs was no problem and I also just could upload new ones without any issue.

So unfortunately this did not help my customer as the error still occurred.

Finally, knowing that customer had a wrong RBS installer version and that I wasn’t able to repro it (remarks: I used the correct file versions!), the restore via the SharePoint UI (central admin page) was quiet easy besides that I don’t know, why I had to rerun the queries on SQL at my restored database…. maybe I just should have rebootet the machine? not tried.

Anyway, I assume, once you have done everything correctly then restore should not the problem as SharePoint is aware of the RBS modification and from the backup file it respects also the streamed documents. So this might be the reason, why we don’t have a detailed “restore”-guide” because there is not really something to “write” Winking smile

Some last words about the restore operations:
I’ve seen some posts, telling that you must install the RBS installer files on SQL server –> Myth! The RBS –Installer is only required on ALL your SharePoint servers! On SQL server you only need to “enable the filestream option within the SQL server configuration manager (see shot below):

… and run the SQL commands:

EXEC sp_configure filestream_access_level, 2
RECONFIGURE

So that’s all! We’re done with the restore part
Winking smile

—————————————————————
BTW: Move-SPSite cmdlet extended with a new parameter!

Another important thing I stumbled over is the new added parameter [RbsProviderMapping], which was added in Microsoft SharePoint Server 2010 with Service Pack 1 (SP1) and Microsoft SharePoint Foundation 2010 with Service Pack 1 (SP1).
This parameter is used to move an RBS-enabled site collection from one RBS-enabled content database to another RBS-enabled content database without moving the underlying BLOB content. If the content database has more than one RBS provider associated with it, you must specify all providers. The same providers must be enabled on the target content database and the source content database.

—————————————————————

General considerations on “when to Blob or not to Blob”!

If you want to install and enable RBS with SharePoint and SQL Server 2008 R2, please first consider some major topics!

a) Do you really have a reasonable need for RBS?

b) Did you considered, that RBS requires a very solid and planned disaster recovery plan in place?

c) What do you expect from RBS?

Not in all cases it is the best just to start with RBS as it has a lot of other consequences to be considered, such as backup/restore and especially disaster recovery strategies! Also the large database size and limitations in several levels are very important! For performance, the break-even should be carefully weight as it might be even slower! (Requires disk sub-system performance of 0.25 IOPS per GB, 2 IOPS per GB is recommended for optimal performance)

Finally, RBS should be more an option for large scaled content databases where you have to store content for a long time but still needing rapid access to it, when it is required! (such as document archive scenarios where less than 5% of content is actively read from each month and less than 1% of content is actively written to).

You should also carefully verify against the RBS concepts of your planning’s the “SharePoint Server 2010 capacity management and Software boundaries and limits”

Here is a collection of all related and recommended links on this Topic:

About Remote Blob Stor (RBS)

SharePoint Content Database size limits and RBS clarifications