About two years ago I created a Feature for SharePoint 2007 which allowed you to generate an XML Sitemap for your site. XML Sitemaps are a standard way to notify search engines about the content on your website. And while you might consider creating an XML Sitemap for your website manually for a small site, having to manage a few hundred pages makes it quickly a pretty tedious process. Being able to do that automatically would be a great improvement and so I’ve made the XML Sitemap Feature v1.0. While it did the job it was more of a proof-of-concept than a solution that you would use in real-life. It had too many limitations to be really useful in real-world scenarios. And so the v2.0 has been born.

Going back in time

The first version of the XML Sitemap Feature worked pretty simple. It shipped as a custom HTTP Handler and was generating the XML Sitemap on every request on-the-fly. Although it theoretically did the job, it had a few drawbacks.

First of all the HTTP Handler was generating the XML Sitemap on every request, putting some heavy load on the server, especially in case of larger sites. You could solve this issue by implementing some kind of caching.

Another limitation of the first version was the fact that the generated XML Sitemap was quite static. Every page in the whole Site Collection got the same Priority and Change Frequency setting what is far from what you would in the real life. One way to work around that would be to generate an XML Sitemap automatically, and then manually locate and edit the values for specific pages. While this might work once or twice, working on a website where lots of new content is being added regularly would make the whole process quite inconvenient.

Did you know that an XML Sitemap file has some limitations? For example it cannot contain more than 50.000 entries and it cannot be larger than 10 MB. The original XML Sitemap Feature didn’t contain any kind of notification system to let you know if your XML Sitemap was anywhere near those numbers, so once again you had a manual action on your list of maintaining the XML Sitemap.

A few months ago a new version of SharePoint has been released. And although it didn’t introduce any changes which might affect the working of the XML Sitemap it was definitely a great reason for a new release of the XML Sitemap Feature.

Generating and managing XML Sitemaps in SharePoint 2010 with Mavention XML Sitemap

Mavention XML Sitemap is a free SharePoint Solution that allows you to generate XML Sitemap(s) for your website. It consists of two Features: Mavention XML Sitemap Site Collection Feature which provisions resources required for generating an XML Sitemap and the Mavention Generate XML Sitemap Site Feature. Activating the Site Features adds the XML Sitemap link in the Site Settings > Site Administration section. Additionally it creates the XML Sitemap Additional Pages list (more about it later).

Mavention XML Sitemap is a Farm Solution. While I was looking at the possibilities to deliver it as a Sandboxed Solution, I realized that some of the pieces, like Long Operations (SPLongOperation) or Timer Jobs are not available in the Sandbox which may be leading to time outs while creating somewhat larger sitemaps, not to mention the consumption of the costly resources of the Sandbox.

New: Configuring Priority and Change Frequency per Content Type

As I mentioned before, one of the shortcomings of the first version of the previous version of the Sitemap XML was the fact that all entries had the same priority and change frequency values. In this version you can set different values based on the Page Content Type. So for example, a home page, which is likely to change often, could have the Change Frequency set to always, while a press release, which never changes to never.

New: Search Engines Notifications

There is little use of an up-to-date XML Sitemap if the search engines are not notified about its changes. And while of course you could do this manually, why not simplify the process and do it automatically right after a new XML Sitemap has been generated?

In this version of Mavention XML Sitemap you can send automatic notifications to Bing and Google which are probably the most important search engines on the Internet. Right after a new XML Sitemap has been created a notification is being sent to the search engines of your choice which can then pick up and process your new XML Sitemap.

New: Including additional pages in the XML Sitemap

One of the big shortcomings of the previous version of the XML Sitemap was the fact that you couldn’t add any additional pages to the generated XML Sitemap: it was either all or nothing. So in this version this feature was one of the first things on the list.

Additional pages are stored in the list called XML Sitemap Additional Pages. This list is being created on every Site which has the Mavention Generate XML Sitemap Feature activated. You can navigate to this list from the Mavention XML Sitemap settings page using the Additional Pages > Edit Additional Pages link.

The great thing about this functionality is, that it not only allows you to add additional entries to the generated XML Sitemap but it also allows you to override the default settings for the specific pages! If you for example would like to change settings for the home page, you would include the following entry:

Using the server-relative URL of the page, the engine of Mavention XML Sitemap solution can match this entry with the page and apply the alternative settings to it. This functionality allows you to tailor the generated XML Sitemap to your specific needs in an easy and manageable way!

New: General information report

An XML Sitemap as specified by the sitemaps.org protocol has some limitations. It can contain no more than 50.000 entries and the generated file size cannot be more than 10 MB. If you have a website with the amount of content that would cause exceeding any of those limits you should consider splitting your XML Sitemap across multiple files. But how do you know how many entries you have and how big the file is?

Another new functionality in the Mavention XML Sitemap is the General information section which contains some information about the most recently generated (current) XML Sitemap.

One great thing about the report is, that it not only shows you the statistics about the current XML Sitemap but also warns you if you are either about to or have already crossed any of the XML Sitemap limits!

For example if you have more than 49.000 entries the following warning will be displayed:

If you however cross the limit, the warning will turn into an error message:

The same applies to the XML Sitemap file size limit, where the warning is set to 9.5 MB and the error is displayed above 10 MB.

Additionally to the XML Sitemap specific information, the report shows you if notifying the search engines succeeded or not.

Changed: Generating the XML Sitemap as a static file

One thing that has changed is the approach I chose for generating the XML Sitemap. In the previous version it was being generated on-the-fly as requested by the search engine. As the new version adds more functionality to the process of generating an XML Sitemap making it therefore heavier I decided to write the contents of the generated XML Sitemap to a static file within the current Site. This file is by default called sitemap.xml but this name can be changed on the Mavention XML Sitemap settings page in the Sitemap.xml filename section.

Probably the last thing left to say is, that I hope you enjoy this new version and ping me if you have any feedback.

Download: Mavention XML Sitemap (19,2KB, WSP)

Technorati Tags: SharePoint 2010,WCM