A few days after we launched our new website I used my favorite search engine to do a search on one of my fellow-Mavens. I was really surprised to see that in just a few days the first result came from our brand new website! So what is it exactly that we did that gave us such great results in such a short time?

Search Engine Optimization is not a sauce

When we started defining requirements for our new website one of the things on the top of the list was getting high search results for our content. Our website is not only a showroom for our organization, our passion, our references and our products but it’s also a part of our marketing, so becoming findable was a high priority for us.

Optimizing websites for search engines consists in general of two parts. On one hand there is the technical part such as optimizing the HTML markup and the Content Management System and on the other hand there is writing good content. In this article I will focus on the technical part of the search engine optimization of our new website.

Since search engine optimization was an important requirement on our list we decided to implement it from the very beginning. Knowing that search engine optimization is not a sauce you pour over your website once it’s done we included it as a part of the solution architecture.

Crafting the markup

We started building the website by translating the graphical designs to HTML. Paul did here some fantastic work crafting the HTML templates. During the translation process he used two guidelines: simplicity and importance.

For every element of the page, Paul was trying to get the most simple and at the same meaningful markup possible. Minimizing the amount of HTML markup not only decreased the size of our pages but also made it easier for search engines to parse, understand and index our content.

At the same time we agreed that the most important content should be on top of the page. So instead using the visual design as a guideline, Paul reviewed all the pieces of the page and came up with a markup structure that would present the content based on its importance: starting with the body, and moving on to other elements such as additional information in the sidebar and navigation. This makes it not only easier for search engines to get the most relevant information first, but it also helps visually challenged visitors to easier explore the content on our website.

Once we had the HTML templates in place we proceeded with the implementation in SharePoint. Because we were using Mavention Flex Layout, we didn’t really have any challenges here and we had the templates implemented in SharePoint in a matter of hours. Because Mavention Flex Layout allows you to separate the visitors’ experience from the content editing experience, we were able to implement the HTML templates just as they were crafted by Paul and deliver optimized experience for both audiences.

When static becomes dynamic

Once we had HTML templates in place we started to replace static HTML blocks with dynamic content that would come up from SharePoint. Some of that content would come from the page itself, some would be provided by Web Parts and some would be dynamically generated based on some information available on the website. One important thing to notice here is, that whenever possible we tried to use components provided with the SharePoint platform to minimize the amount of custom development.

Hierarchical titles

Moving from the top to the bottom of the Master Page we started with generating proper titles for all our pages. Since titles have big influence on the search results we wanted to do it right. To support both search engines as well as visually challenged visitors, we decided to use our Hierarchical Title control. This allowed us not only to automatically include some additional information about the page in the title but also to make it easier to place the content in the right context when coming from different sites.

Heading structure

Moving further down the page markup we specified a proper heading structure for our pages. We decided on having only one h1 tag that would contain the title of each page. Headings in content would begin from h2. The contents of the h1 tag are very important from the search engine optimization point of view so you should always try to make it contain the most relevant summary of the content below it. And what’s better to use to describe the content of a page than its title?

Crafting URLs

One of the things that we spent quite some time doing right was designing and configuring URLs for our website. Although it might seems trivial, since it “works by default”, URLs are an important factor in optimizing websites for search engines so we knew we just had to do it right the first time.

Configuring canonical URLs

The first thing we did was to setup canonical URLs for our website. Canonical URLs is the term used in the SEO world to describe the one and only URL the content of a website should be available on. You see, serving the same content through different URLs might be noted as duplication by a search engine and might lead to decreasing the ranking of your pages, so ensuring that your content is accessible by one and only one URL is very important.

In case you didn’t know, our website is accessible via multiple domains: mavention.nl, mavention.com and mavention.eu. Additionally you can view our website by prepending each one of those domains with www. Only this gives us six different URLs to access one and the same content! The canonical for our website that we wanted to use was www.mavention.nl. So to do things right, we created a new IIS site with bindings for all the different domain names except for www.mavention.nl so that it can catch all requests to our site. Then, using the URL Rewrite module, we created a permanent redirect rule to www.mavention.nl:

Case-sensitive URLs

IIS – the webserver behind SharePoint – uses case-insensitive URLs. This means that no matter how you capitalize the URL, it will serve you the page that you are looking for. This is however pretty inconvenient consider the fact that URLs on the Web are actually case-sensitive!

Using different capitalization for the same pages may cause search engines to think that your content is duplicated. Although both pages will get indexed, it might lead to decreasing the content ranking since it will be divided across all the different “variants” of the page.

To avoid this issue on our website we agreed on using lowercase URLs across our website for everything: from sites and pages to assets such as images and CSS files. To enforce this rule we configured a redirection rule using URL Rewrite on our website. The great thing about it is, that a template for this rule is a part of the URL Rewrite module, so the only thing that you have to do is to add a new rule and choose Enforce lowercase URLs from the list of available templates.

After applying this rule all URLs that contain uppercase characters will be redirected using a permanent redirect to the lowercase version. The great thing about this is, that it applies only to the URL itself, so if you have querystring parameters that use uppercase characters, they will not be rewritten to lowercase.

It’s moved, really

When working with SharePoint you might have noticed that all redirects from the site to the Welcome Page are being done using temporary redirects (302). Theoretically this is good. If the Welcome Page would ever change a search engine wouldn’t index the contents of the old URL as  the default page for the site. Practically this isn’t good at all, because while it keep the search engine from associating the old URL with the site, it actually keeps it off of indexing the Welcome Page at all! Most search engine don’t follow temporary redirects, because – as the name says it – they are temporary and the “real” page might come back any minute.

To turn the temporary redirects into permanent ones we implemented the solution I described recently in my post about codeless configuration of the SharePoint redirects.

After we implemented this solution, however, we noticed SharePoint Search breaking. No matter if we set it to crawl our site as SharePoint site or through HTTP, the crawler couldn’t deal with the rules and in result didn’t crawl anything, returning lot’s of errors. Eventually we ended up using a different zone (and therefore a different IIS website which didn’t have the URL Rewrite rules configured) for crawling.

Just a little more help

Crafting good web content is not easy. You not only have to take care for it being easy to read but also to be found so that it will be read in the first place. Search engines use links to discover new content, so if you make your pages link to each other or you will get some links from other websites you should be good. This ideal picture is often difficult to achieve and you might end up with pages without any links to them. Does this mean that those pages will never end up in search engines’ index?

The great thing about search engines is that links are not the only thing that they use for discovering new content. You can use XML sitemaps to help search engines discover new content or content with fewer/no links to it. If your website consists of only a few pages you might consider creating an XML sitemap yourself, but if you have many pages and you update your website on regular it might be easier if you used a tool to generate an XML sitemap for you as well.

On our website we used the Mavention XML Sitemap for SharePoint 2010 tool that we made. Not only it allows you to quickly generate an XML sitemap but it also allows you to configure settings such as priority and change frequency. By automatically uploading the sitemap to Google and Bing right after it’s generated we ensure that a fresh XML sitemap is there ready to be processed by the search engine.

Summary

In this article I‘ve showed you things that we implemented in our website to optimize it for search engines. Although it makes things easier for us, it’s not a golden bullet and there is still a lot of work that we have to do each time we write content for our website.

What’s next

Developing Internet-facing websites on the SharePoint platform means that you not only have to understand how SharePoint works but also have to have some understanding about what the Web is and how things work there. In the next part of this How we did it series I will show you some things that we implemented to make our website a better citizen in the Web world. Stay tuned!

Technorati Tags: SharePoint 2010,Web Content Management,Mavention