Building an English version of our website was one of the most important design decisions for our new website. And although building and maintaining a multilingual website is an ongoing process, it is totally worth it, because so far we have had some great feedback. So how did we do it?

Mavention.nl yesterday

Nearly two years ago we launched our new website built on top of SharePoint 2010. It was a great site and we were pleased with the results. As the time passed by however we found out that it had one serious flaw.

In the last two years we noticed that nearly 60% of all of our visitors were originating from outside of the Netherlands. Additionally we noticed that we had a rather high bounce percentage. Although the majority of the blog posts published on our website were in English, the most important content about who we are and what we do was available in Dutch only. So when planning for our new website we knew that we had to do better. And so the idea of a multilingual experience was born.

Planning for multilingual

We wanted our website to do more than just presenting the same content in two languages. We wanted it to meet accessibility standards and to be optimized for search. Above all we wanted it to be easy and intuitive to use so before we started building the website we took a moment to get a better understanding of what we exactly wanted to achieve with regard to the multilingual experience.

Mixed content

From the very beginning of the planning stage for our new website we knew that we wanted to have the content available in Dutch and in English. We knew that we wouldn’t be able to have all of the content available in both languages. The majority of blog posts were in English already and having them translated to Dutch would add little value given the fact that they are technical and we are quite confident that all SharePoint professionals speak and read English. Other content published on the site presenting our company and services would become available in both English and Dutch.

Content translation

Translating content to other languages is a time-consuming process. Not only you have to take care for translating the words but you also need to ensure that the meaning is preserved. To save us time we decided on using a translation agency to translate the content for us: no matter if it’s from Dutch to English or the other way around.

Optimizing for search

Having the content of our website available in both Dutch and English we were hoping to be of better service to our non-Dutch-speaking visitors. We hoped for this to lower the bounce rate but also to increase the discoverability of our new website. For this last part in particular we decided that we would not only have to translate the content itself but also the URLs on which it’s published. URLs are one of the most important page properties that determine the ranking of a page in search results and we wanted to leverage this paradigm to increase the visibility of our website in search engines.

Switching between languages

Even though not all of the content on our website is technical, we are a company with 100% focus on SharePoint. With that there is a chance for us to use English SharePoint-related terms, which just don’t make sense to be translated on Dutch pages. Using English terms on Dutch pages makes it possible that an English-speaking person having searched for an English term would end up on the Dutch version of our website. To make it easy for our visitors to switch between the language we wanted to provide a seamless experience where they can switch between Dutch and English back and forth while staying on the same page.

Designing for multilingual

Having got a clearer understanding of what we were trying to achieve, we proceeded with thinking about how those requirements could be fulfilled using different SharePoint 2013 capabilities.

Publishing content on language-specific sites

SharePoint offers the variations capability for building multilingual websites. This capability has been available since SharePoint 2007 where Web Content Management workload has been included into the SharePoint platform. Although variations had some technical changes in the last few years, the concept remains unchanged in SharePoint 2013: you define a source language with its hierarchy and for every language (label) that you create that hierarchy is mirrored. This configuration results in a single Site Collection with a hierarchy of webs for every language. The new design opportunity that SharePoint 2013 introduces on top of this concept is the ability to map variations webs to separate Site Collections using Cross-Site Publishing (XSP). Take a look at the following schema:

On the left-hand side there is the authoring Site Collection with variation webs. The content is created and maintained here and this site is accessible to website authors only. Using Cross-Site Publishing content from the variation webs is then published on two publishing Site Collections on the right-hand side: mavention.nl and mavention.com. Using XSP for publishing multilingual content on separate sites would allow us to better optimize our website for search engines. First of all we can benefit of local search ranking by using Country-Code Top Level Domains (CCTLD), such as .nl for Dutch and .com for English. Additionally we could use fully translated URLs which would further increase the ranking of our content in search. Above all we would be able to manage the content in a single location as opposite to managing the content on the particular site that it’s published on.

Mixed content and accessibility – better together

One of the most important requirements for building our new website was to have it meet the accessibility requirements as defined for Dutch government websites. Although we are not a government organization ourselves we wanted to verify that you could build a fully-compliant website with SharePoint 2013. Amongst the accessibility requirements defined by the Dutch government guidelines there is one stating that one should always specify the natural language in which the content of a page is written. Having two separate sites – .nl for Dutch content and .com for English this requirement doesn’t seem like much and theoretically it could be solved by adding the lang attribute to the html tag:

<html lang="nl-NL">

This would be sufficient if only we didn’t have mixed content. As mentioned before, we planned to leave blog posts written in English untranslated and we would publish them in English on both mavention.nl and mavention.com. From the accessibility perspective denoting the whole page as written in Dutch would result in screen readers reading English text using Dutch pronunciation. So in order to overcome this challenge we decided to have a Site Column called Natural Language where we can specify in which language the particular page is written despite its variation location.

When rendering the page’s HTML we would add the lang attribute, with its value set to the Natural Language of the particular page, to the article tag surrounding the contents of the page, like:

<html lang="nl-NL">
...
<article lang="[NaturalLanguage]" id="content">
...
</article>
...
</html>

With this we could have mixed content on the page without any negative impact on accessibility or usability.

The page itself is not the only place where mixed content can be displayed. An overview page such as blogs or news or even search results can also contain a mix of English blog posts and Dutch pages so we would need to refer to the Natural Language property there as well.

Content translation

One of the new capabilities introduced in SharePoint 2013 are Translation Services. Using Translation Services you can either choose to have your content translated automatically or you can use Translation Packages to export the content from SharePoint, hand it off to the translation agency and once translated, import it back to SharePoint. Because we wanted to have our content translated by a translation agency we decided on using Manual Translation using XLIFF translation packages.

Optimizing for search

When publishing content using the new Cross-Site Publishing capability in SharePoint 2013 one of the configuration settings that you have to define is the field(s) that you want to use in the URL. From a product catalog perspective this can be a SKU or some other combination of fields uniquely identifying particular product. For our content we decided to have a Site Column called Slug which we would use to specify the page’s URL. In most scenarios it would be very similar to the page name, but then it wouldn’t contain the .aspx extension or any noise words allowing us to get the most out of SEO.

Switching between languages

So far we were designing for our new website using standard capabilities available with SharePoint 2013. Designing the language switch however turned out to be more complex than we initially thought it would be.

When designing for our website we decided on using variations for maintaining multilingual content and we also decided on using fully translated URLs to optimize our site for search. This could result in one and the same page being published on mavention.nl as http://www.mavention.nl/nieuws/mavention-nl-behaalt-waarmerk-drempelvrij and on mavention.com as http://www.mavention.com/news/mavention-nl-awarded-waarmerk-drempelvrij. As you can see those are two completely different URLs, so how do you switch between languages?

Normally, when using variations, this isn’t much of a challenge. URLs of all variations pages are stored in the PublishingPage.VariationPageUrls property. The only thing that you need to do, to provide a link to the same page in another variation, is to retrieve it from that property and render in on the page.

With Cross-Site Publishing things become a little more complicated. First of all the VariationPageUrls property is not a Site Column and therefore it cannot be picked up by SharePoint Search to become a Managed Property. And even if we wrote some custom code that would allow us to store the value of the VariationPageUrls property in a Site Column, it wouldn’t be of much use as it would point to the physical page located in the authoring site (ie. https://authoring.mavention.nl/en-us/pages/mavention-nl-awarded-waarmerk-drempelvrij.aspx) and not to the page published on the mavention.com website (http://www.mavention.com/news/mavention-nl-awarded-waarmerk-drempelvrij). One more thing that should be taken into account is that when using Cross-Site Publishing the hierarchy of the publishing site is determined by Managed Metadata. This means that even though mavention.nl and mavention.com are one and the same site but in a different language, there is no guarantee that a blog post or a press release is published on both sites in the same place in the hierarchy of the site. SharePoint 2013 offers us the flexibility to deviate from it on a per-page basis and in our solution for switching languages we should also provide such flexibility. So with all the things that we didn’t know about a page and couldn’t rely on, how did we get the URL of the same page in a different language?

Building the multilingual experience

The majority of the multilingual experience that you can see live at mavention.nl and mavention.com is built using standard SharePoint 2013 capabilities. We use variations for authoring and maintaining multilingual content in the authoring site. We translate the content using manual translations with translation packages, and once done we publish that content to mavention.nl and mavention.com using Cross-Site Publishing. While building the multilingual experience for our new website there were only two things that required some extra attention.

Mixed content and accessibility

When designing for publishing mixed content on our website we found out that we needed an additional Site Column to denote the natural language the particular page is written in. We would then use the information from that column and have it displayed on the page itself but also in content aggregations.

To include the natural language of the page in the page’s HTML we used the CatalogItemReuseWebPart:

<html lang="nl-NL">
...
<article lang="<Search:CatalogItemReuseWebPart runat='server'
                UseServerSideRenderFormat='True'
                UseSharedDataProvider='True'
                SelectedPropertiesJson='[&quot;NaturalLanguageOWSCHCS&quot;]'/>" id="content">
...
</article>
...
</html>

Although it’s not the prettiest piece of markup it allows us to have the language of the page be set globally on the html tag in the Master Page and then reset the language if necessary for a particular page. Because we’re using the CatalogItemReuseWebPart configured to use shared data provider, the NaturalLanguageOWSCHCS Managed Property is added to the existing search query and retrieved along with other properties displayed on the page without the need to perform another search query.

Another place where we make use of the natural language property are overview pages such as the home page (http://www.mavention.nl), overview of blog posts (http://www.mavention.nl/blog) or search results (http://www.mavention.nl/zoeken?k=sharepoint+2013). On all those pages we have blocks that represent different items and each of those can be either in Dutch or in English. Although we are using the Content Search Web Part for building aggregations we use server-side rendering instead of Display Templates with client-side rendering. Server-side rendering in SharePoint 2013 is based on XSLT and in order to have the natural language property rendered on a page we refer to it from within the XSL:

...
<xsl:template match='Row'>
    <xsl:variable name='NaturalLanguage' select='NaturalLanguageOWSCHCS'/>
    <div lang='{$NaturalLanguage}'>
...
</xsl:template>

With this every block representing an item has its own lang attribute set ensuring that should a screen reader read the contents of the page, it will automatically choose the suitable pronunciation.

Switching between languages

When designing for the switch between languages we stumbled upon a challenge. We noticed that there was no way for us to determine the URL of the current page in another language. After having explored a number of different options we thought that the only way in which you can really tell the URL of a page on a particular site is… to search for it!

If you look at the URL behind the language switch button on our website you will see something like:

http://www.mavention.com/_layouts/15/mv/variation.ashx?page=sharepoint-wcm-specialist

The variation.ashx is a custom HTTP Handler that we have built that searches for the particular page in the given Site Collection and redirects to it if found. But there is more to it than meets the eye.

When optimizing our website for search engines we decided on translating the Slug to the natural language of the page so that we can get the maximal benefit from search engines. Given that all of the contents of a page are translated, there is no way for us to determine which page we are looking for. After all, from the context of a page published using XSP, the only things at our disposal are Managed Properties that we can retrieve using the CatalogItemReuseWebPart. So in order to be able to match pages across variations we decided to add another Site Column called Variation Key.

The contents of the Variation Key Site Column are the same as the Slug of the page from the source variation. The only difference here is that the Variation Key property is not translated across languages so for every variation page that value is the same.

Another challenge that we had to overcome was the fact that we have not one but three types of pages on our website: we have the home page, overview (category) pages (such as blog and news) and item pages (eg. a particular blog post). From those three types of pages only item pages exist in the authoring site and have the Variation Key specified. So depending on the type of the page we would need a different way of identifying that particular page. To solve this we came up with an idea to stamp item pages with a property that we could then check for in the language switch link control. If it’s present it would mean that we are on an item page otherwise it would mean that we are on a category page. To stamp item pages we built a simple control that is present in all Page Layouts used by template pages for item pages and which sets a property in the current context:

public class CatalogItemPage : Control {
    protected override void OnInit(EventArgs e) {
        HttpContext.Current.Items[Common.IsCatalogItemPagePropertyName] = Boolean.TrueString;
    }

    protected override void Render(HtmlTextWriter writer) {
    }

    public static bool IsCatalogItemPage() {
        return (HttpContext.Current.Items[Common.IsCatalogItemPagePropertyName] as string) == Boolean.TrueString;
    }
}

When building the language switch link the only thing to do in order to know if we are on an item page or not is to call the IsCatalogItemPage method.

Before we could start searching for pages we had to build the language switch URL that visitors could use to switch between languages. We start from checking if the current page is an item page. If it is, we add a CatalogItemReuseWebPart to retrieve the value of the Variation Key property.

protected override void CreateChildControls() {
    if (CatalogItemPage.IsCatalogItemPage()) {
        Controls.Add(new CatalogItemReuseWebPart {
            UseServerSideRenderFormat = true,
            NumberOfItems = 1,
            UseSharedDataProvider = true,
            SelectedPropertiesJson = String.Format("['{0}']", VariationKeyPropertyName)
        });
    }
}

To avoid any performance penalty and to meet all accessibility guidelines we want the value of the Variation Key property to be retrieved together with all other Managed Properties on the page and rendered as a part of that page as opposite to using AJAX to retrieve it afterwards.

Finally, during the render stage, we choose the rendering method based on the page type. For item pages we intercept the rendering of the CatalogItemReuseWebPart to get the value of the Variation Key property and for overview pages we use the TaxonomyNavigationContext to get the current navigation term.

protected override void Render(HtmlTextWriter writer) {
    string variationKey = String.Empty;

    if (CatalogItemPage.IsCatalogItemPage()) {
        StringBuilder sb = new StringBuilder();
        using (StringWriter sw = new StringWriter(sb)) {
            using (HtmlTextWriter htw = new HtmlTextWriter(sw)) {
                base.Render(htw);
            }
        }

        variationKey = sb.ToString();
    }
    else if (TaxonomyNavigationContext.Current != null) {
        NavigationTerm navigationTerm = TaxonomyNavigationContext.Current.NavigationTerm;
        if (navigationTerm != null) {
            variationKey = navigationTerm.Id.ToString();
        }
    }

    // render the link
}

And with this the language switch link is ready to be used.

The logic behind searching for a page depends on what type of page we want to find. For item pages we execute a keyword query passing in the name of the Managed Property mapped to the Variation Key Site Column and the page variation key passed using the page query string parameter:

string page = context.Request.QueryString[Common.VariationPageQueryStringParameterName];

string url = null;

if (!String.IsNullOrEmpty(page)) {
    string variationKeyPropertyName = Utils.GetSettingFromProperty(Common.VariationKeyPropertyNamePropertyName) as string;

    if (!String.IsNullOrEmpty(variationKeyPropertyName)) {
        KeywordQuery query = new KeywordQuery(SPContext.Current.Site);

        Guid resultSourceId = Guid.Empty;
        if (Guid.TryParse(Utils.GetSettingFromProperty(Common.VariationLookupResultSourceIdPropertyName) as string, out resultSourceId)) {
            query.SourceId = resultSourceId;
        }

        query.QueryText = String.Format("{0}:{1}", variationKeyPropertyName, page);
        query.SelectProperties.Add("Path");
        ResultTableCollection tables = new SearchExecutor().ExecuteQuery(query);
        if (tables.Exists(KnownTableTypes.RelevantResults)) {
            ResultTable table = tables.Filter("TableType", KnownTableTypes.RelevantResults).Single<ResultTable>(relevantTable => relevantTable.QueryRuleId == Guid.Empty);
            if (table != null && table.ResultRows.Count == 1 && table.Table.Columns.Contains("Path")) {
                DataRow row = table.Table.Rows[0];
                url = table.Table.Rows[0]["Path"].ToString();
            }
        }
    }
}

Because we use this code with two different Site Collection and we also want this code to be reusable for other solutions, we have the name of the Variation Key Managed Property and the ID of the Result Source used for searching configurable.

For looking up category pages we use the NavigationTermSet instead, where we iterate through terms looking for the one of which the ID has been passed using the term query string parameter:

string term = context.Request.QueryString[Common.VariationTermQueryStringParameterName];

string url = null;

if (!String.IsNullOrEmpty(term)) {
    Guid termId = Guid.Empty;
    if (Guid.TryParse(term, out termId)) {
        try {
            NavigationTermSetView view = new NavigationTermSetView(SPContext.Current.Web, "GlobalNavigationTaxonomyProvider") {
                ExcludeTermsByProvider = false
            };
            NavigationTermSet navigationTermSet = TaxonomyNavigation.GetTermSetForWeb(SPContext.Current.Web, "GlobalNavigationTaxonomyProvider", true).GetWithNewView(view);
            NavigationTerm navigationTerm = navigationTermSet.Terms.Where(nt => nt.Id == termId).FirstOrDefault();

            if (navigationTerm != null) {
                url = navigationTerm.GetResolvedDisplayUrl(String.Empty);
            }
        }
        catch { }
    }
}

If we succeeded in finding the page we perform a permanent redirect to that page. This allows us to ensure that search engines are going to follow those links as well. If however the link points to a page that hasn’t been found (like when a page hasn’t been publishing on the English site yet), we redirect the visitor to the home page using a temporary redirect.

if (!String.IsNullOrEmpty(url)) {
    context.Response.RedirectPermanent(url, false);
    context.ApplicationInstance.CompleteRequest();
}
else {
    context.Response.Redirect("/", false);
    context.ApplicationInstance.CompleteRequest();
}

And with this seamlessly switching between languages is complete.

Summary

One of the most important decisions when planning for our new website was building an English version of our website. Using the new capabilities provided with SharePoint 2013 we were able to cover the majority of the requirements using the standard functionality. Seamless switching between languages is the only place where we had to build a custom solution for, but even that was relatively easy to build using the rich API that SharePoint 2013 offers.