SharePoint 2010 ‘Page not found (404)’ page the way it should be
Best Practices, Development, Inconvenient SharePoint, SharePoint 2010, Usability, WCM
More and more frequently SharePoint is being used as the platform for Internet-facing websites. Not surprisingly the latest version of SharePoint is greatly improved making it event better solution for creating public websites. Although Microsoft did a great job in the Web Content Management area of SharePoint there are still a few shortcomings, one of which are custom ‘Page not found (404)’ pages. Find out how to do things right when moving your website to the SharePoint platform.
See no evil…
‘Page not found (404)’ pages are one of those things that no one wants to see but we all need it. No matter how hard you will try and how often you will verify your content for dead links, it’s just a matter of time before someone mistypes the URL. Although SharePoint won’t break immediately this is what you will see by default:
Far from perfect isn’t it? It isn’t even good to be honest. Especially if you’re considering to use SharePoint as the platform for your public website it’s definitely one of the things you have to do something about. So what are the options?
What do you really need?
Before we take a look at what SharePoint has to offer let’s have a quick glance at how a proper 404 page should be like.
First and the most important of all it should make clear to the user that the requested page doesn’t exist – no matter if it’s the user’s fault by mistyping the URL of it’s accessed via a dead link. And because it’s not the page that the user needs it should be minimalistic, simple and contain only the necessary elements.
The second important this is, that it should be clear to the user that he is still on the site. This means that the 404 page should have User Experience consistent with the rest of the site.
If you look from the user point of view, the 404 page is rarely the destination. As the name says, it’s an error page meant to communicate that something didn’t go as intended. That is exactly why a good 404 page should be more than only an error message and should help the user to find what he is looking for. If you’re using meaningful URL’s, which you should be using, you can use the missing URL to search for related information on your site and present those results to the user.
Seeing a 404 page is confusing for most users and you shouldn’t make it even more confusing. This is why it is important not to change the URL so that the user can see which URL he requested.
Finally, from the technical point of view, the 404 page should return the 404 HTTP status code. This will allow search engines and all other automated software to properly handle the page and decide whether they really want to process the undesired response or not.
So what are the options for good 404 pages in SharePoint?
PO(S)H
When looking at what SharePoint offers out of the box for custom ‘Page not found’ pages, there are not that many options. To be honest there is only one option which allows you to set a 404 page on the Web Application level. The worst part of it is, that SharePoint requires that page to be Plain Old (Semantic) HTML only. Additionally this page must be deployed to the LAYOUTS folder. Such approach means that not only the custom ‘Page not found’ page has limited capabilities but also that there is no easy way for content managers to update the contents of that page.
Once you get everything in place there is one more surprise for you. Let’s take a look at how the 404 page looks like when you mistype the file extension:
The HTML is being encoded!
So are there any alternatives to make a proper 404 page or is it something that we should learn live with?
Alternative solutions
One of the common workarounds for the standard capabilities is to create a custom ‘Page not found’ page which redirects to a regular SharePoint Publishing Page. Such page can be easily maintained by the content editing team and has a User Experience which is consistent with the rest of the site. The downside of this approach is that the URL in the browser is being switched with the URL of the error page what might be confusing to the user. Additionally automatic redirections are frequently seen as inaccessible what makes it not the most user friendly solution out there.
A slight variation of this approach is a custom HTTP Module which intercepts all 404 responses and redirects to the ‘Page not found’ page instead. While this approach deals with the issue of client-side redirect it still has some downsides. The 404 response code is replaced by a redirect status code which might be processed by search engines. Additionally the original URL is being replaced with the URL of the error page what might confuse the user.
There is however a better way…
A proper SharePoint 2010 404 page
Recently I’ve done some research on custom 404 pages in SharePoint 2010 and found out that all you need to have to create a proper 404 page are a custom HTTP Module, a Publishing Page and a custom Web Part. Here is how it works.
First you create a custom HTTP Module which will intercept all 404 responses:
public class PageNotFoundHttpModule : IHttpModule
{
private HttpApplication app;
private string pageNotFoundUrl = "/pages/page-not-found.aspx";
public void Dispose()
{
}
public void Init(HttpApplication context)
{
app = context;
app.PreSendRequestContent += new EventHandler(app_PreSendRequestContent);
}
void app_PreSendRequestContent(object sender, EventArgs e)
{
HttpResponse res = app.Response;
HttpRequest req = app.Request;
if (res.StatusCode == 404 &&
!req.Url.AbsolutePath.Equals(pageNotFoundUrl, StringComparison.InvariantCultureIgnoreCase))
{
app.Server.TransferRequest(pageNotFoundUrl);
}
}
}
This module checks if the response status code is 404 and the URL is different than the URL of the ‘Page not found’ page. If that is the case the request is being transferred to the ‘Page not found’ page. This is better than the standard redirect since it allows you to keep the original URL in the browser.
Important: Since you’re replacing the URL be careful what kind of functionality you provide on the error page. Rewriting URLs is not supported in SharePoint and might result in breaking some functionality so you should always test your solution before deploying this in production.
The next step is to create the ‘Page not found’ Publishing Page – nothing special here.
If you test your solution now you will see that you are almost there:
Although your custom ‘Page not found’ page is being displayed with the original URL (1), the response code is set to 200 OK (2) which is different than the 404 code we expected. This is caused by the request transfer that takes place in our HTTP Module. Unfortunately the response code cannot be set directly after calling the transfer since it’s being set later in the Page execution cycle. And this is exactly where our custom Web Part fits in:
public class PageNotFoundWebPart : WebPart
{
protected override void OnLoad(EventArgs e)
{
base.OnLoad(e);
Page.Response.StatusCode = (int)HttpStatusCode.NotFound;
}
}
By placing the Web Part as showed above the response code is being changed to 404 as expected:
No matter if the non-existing URL is caused by wrong page name or a faulty extension the HTTP Module will intercept the 404 response and will display your custom ‘Page not found’ page with the 404 response code – exactly as you would expect it to work.
January 25th, 2011 at 2:35 pm
Hey Waldek, thanks for sharing. I am going to try this out myself in a couple of minutes. ;-) Next, I am planning to add a web part that provides an alternative using Search.
January 26th, 2011 at 4:51 pm
@Servé: you're welcome. search based on 404 url is definitely a great improvement. The downside is that out of the box SharePoint doesn't really do URL-based search suggestions so you would have to do some URL processing to get desired results.
January 28th, 2011 at 3:47 pm
I was getting a bit frustrated as the page did not work. So I just rewrote the http module to output just "hello" (my favourite). Surprise: it works when you type in a valid URL but still shows a 404 page with an invalid URL. It has nothing to do with your module, SharePoint or IIS but with a setting in your browser.
This is what visitors need to do: turn off the "Show friendly HTTP error messages" in your IE internet options -> Advanced tab.
Else you will still see a "404" page rendered by the browser (as the browser is the last one in the chain who still responds to the http status code).
Also in Google Chrome you will need to change the setting to see the custom 404 page: Advanced options -> Use a webservice to lookup navigation errors (or something like that). Uncheck.
Maybe it is better to not return the 404 status code by not adding that last web part unless it is more important to you that Search engines will stop crawling dead links (eg on Internet facing sites)…
If you are a real FAST expert you might want to put something on your page that FAST recognizes. Eg extend the content processing pipeline.
January 30th, 2011 at 9:36 am
@Servé: this is pretty odd, since no matter the setting the 404 page using exactly this technique works on the mavention.nl website. If I'm correct there is a limit that an error page should be that large. If the page is under that size the browser will take over and will display its own 'friendly' HTTP error page.
May 6th, 2011 at 11:12 am
Hi Waldek,
thanks for sharing this and useful tip
but one question :
this code works pretty fine when you type in ivalid .aspx page urls but when you type in invalid web url (server relative) then shows default 404 error page , why?
for example : If I type in http://www.yoursite.com/Pages/invalidpage.aspx then code works fine and redirect users to your custom page not found page but
If I type in url like http://www.yoursite.com/invalidweb
then this shows 404 status page directly
is this something with SP IIS pipeline? or with SPModule , becuase in second casedebugger dont even come to custom HttpModule
May 6th, 2011 at 12:01 pm
Hi Waldek,
thanks for sharing this and useful tip
but one question :
this code works pretty fine when you type in ivalid .aspx page urls but when you type in invalid web url (server relative) then shows default 404 error page , why?
for example : If I type in http://www.yoursite.com/Pages/invalidpage.aspx then code works fine and redirect users to your custom page not found page but
If I type in url like http://www.yoursite.com/invalidweb
then this shows 404 status page directly
is this something with SP IIS pipeline? or with SPModule , becuase in second casedebugger dont even come to custom HttpModule
September 2nd, 2011 at 10:18 am
Hi,
It works for system page, but not for application page..
for example : If I type in http://www.yoursite.com/Pages/invalidpage.aspx then code works fine and redirect users to your custom page not found page but
If I type in url like http://www.yoursite.com/_layouts/appname/invalidweb
then this does not show custom 404 status page. Is there any fix for this
October 6th, 2011 at 5:22 pm
hi Waldek,
the HTTPModule seems like it might be a good solution for us – thanks for sharing! BTW – I ran into a similar problem as your encoded HTML page that you display above and after resolving the issue for myself by opening the file outside of Visual Studio to remove the Byte Order Marker (BOM), I can't help wondering if that may have been your issue as we. Here is a link to where I learned about the BOM issue: http://andreasglaser.net/post/2009/03/15/SharePoint-and-custom-404-Page-Not-Found-and-UTF-8-issue-with-Firefox.aspx
November 23rd, 2011 at 3:42 pm
Waldek, this worked great as is. I didn't experience any of the issues that the other commenters reported such as the module not working for a non existent web.
After deploying, I would probably also add a robots.txt entry for the page not found publishing page, and also exclude it from the SharePoint search results.
Thank you!
December 6th, 2011 at 10:38 pm
I found an alternative to the webpart to generate the 404. You can listen to this event in the HtppModule pipeline as well:
void app_PostRequestHandlerExecute(object sender, EventArgs e)
{
HttpResponse res = app.Response;
HttpRequest req = app.Request;
if (req.Url.AbsolutePath.Equals(pageNotFoundUrl, StringComparison.InvariantCultureIgnoreCase))
{
res.StatusCode = 404;
}
}
December 7th, 2011 at 8:51 am
Good one! Thank you for your feedback, Andrew!
March 28th, 2012 at 11:23 pm
Thanks Waldek, and thanks Andrew! Will be able to retract the web part, now.
April 11th, 2012 at 7:39 am
Thanks for this post, it works fine, however, if the site collection has the URL (example) http://server/sites/mysite/ then it doesn't work with "private string pageNotFoundUrl = "/pages/page-not-found.aspx";". How can I get the site collection URL so the string would be "/sites/mysite/pages/.." instead? Thanks in advance.
April 18th, 2012 at 5:16 pm
@Morgan: Have you tried seeing if the SPContext.Current.Site is available?
June 26th, 2012 at 11:25 am
Hi Waldek,
When I use the PostRequestHandlerExecute described by Andrew and then when I try to navigate from the 404 page I always stay on the 404 page. On my 404 I have a search box. When you click search a new request is done. The request is first going to 404 so the PostRequestHandlerExecute fires and sets status to 404. So is the only option use your webpart on the 404 page?
June 26th, 2012 at 11:33 am
It seems that the problem is in the app_PreSendRequestContent. When a postback occurs this event is firing and doing a transfer to the 404 page
June 26th, 2012 at 1:22 pm
The actual problem seems to be that the postback is done to the address of the page that wasn't found and caused the 404 to be displayed. This is the address that is in addressbar when 404 is displayed.
July 2nd, 2012 at 8:39 am
The problem is, that although you might not have noticed it, you're working with URL rewriting which isn't supported in SharePoint 2010. The exact reason that you are having is the difference between the physical URL of the page that contains the Web Part and the URL in the browser. My suggestion would be to avoid postbacks whatsoever and try to perform the search action either by creating a separate
July 3rd, 2012 at 3:05 pm
Hi Waldek,
When I run this module I get login prompt when opening site content & structure. SharePoint is loading some files from layouts folder that are not found. See this post for the files.
http://sharepoint.stackexchange.com/questions/28272/404-http-module-causes-an-authentication-popup-in-manage-content-structure-pag
I don't get why I get a login prompt. Is there another way to bypass this module other than check for layouts in url? I tried to check for IsAuthenticated but current user is always null
August 7th, 2012 at 4:00 pm
You spelled exist wrong in your 404.
August 8th, 2012 at 9:16 pm
LOL: you're absolutely right :)
September 3rd, 2012 at 2:29 pm
Nice post, but I have to especially thank Justin for supplying the fix to the easiest solution for a 404 page :-)
November 20th, 2012 at 8:17 am
Just curious but why not use httperror under system.webserver in the web config? No http module required and you can still use your page with a usercontrol or webpart to set the 404 http response code.
November 22nd, 2012 at 7:47 am
I'm not sure this would allow you to use a Publishing Page that is maintainable by content authors rather than the IT.
February 27th, 2013 at 6:09 pm
I'm guessing this would redirect all requests, including static files like images and scripts. Any suggestions on a good way to only redirect page requests?
February 27th, 2013 at 7:07 pm
One thing that you could do is to check the file extension and have the code fire only for extension-less URLs (webs) or URLs with .aspx.
March 20th, 2013 at 1:12 am
I'm new SharePoint. Where is the SharePoint Server 2010 web.config located to add an entry to section to point to my Custom Http Module?
March 20th, 2013 at 2:45 pm
web.config is located on the disk, very likely on a path similar to: C:\inetpub\wwwroot\wss\VirtualDirectories\mavention80 where the last directory is the name of your website.
March 20th, 2013 at 10:02 pm
Update to my earlier comments. I got it working! thanks for this post!
May 6th, 2013 at 2:05 am
Hi Waldek,I am new to HttpModule and SharePoint. How to create a HttpModule in SharePoint project in Visual studio?
where to put the PageNotFound.cs file to make it work? thanks!
May 6th, 2013 at 3:07 am
I just realize the httpmodule is seperate project…nothing to to with sharepoint, right? and but I need to catch the original url and redirect to new url on the sharepoint page. is it possible to catch the original url in the webpart?
May 6th, 2013 at 7:03 am
An HTTP Module is nothing more than a class in a SharePoint project, so it doesn't really matter where you put it. Since it's code it's going to be compiled into the assembly and there are no other artifacts for it packaged in the WSP.
May 6th, 2013 at 2:38 pm
sorry,confused… I just plan to dev a regular httpmodule project by visual studio web project, then install it in IIS sharepoint virtual directory. if as you said, how the IIS find the Module ?
May 7th, 2013 at 6:01 am
HTTP Modules must be registered in the web.config. Look at the Modules section in there for sample of the registration entries.
May 8th, 2013 at 2:03 am
Yes! I got the HttpModule works, I love you!
May 12th, 2013 at 2:43 pm
a serious problem, I got same issue as Prasham, it doesn't work on the links like http://sp1:11111/kk only works on links like http://sp1:11111/a1.aspx what can I do for this?
May 12th, 2013 at 2:51 pm
my fault, it works I didn't type in right link url, actually, I store url matching in sharepoint list, then redirect the old url to new one in the list. it works fine now.