Inconvenient Content Targeting with User Segments in Search REST API

The User Segment Terms property in a Search REST API request
Content Targeting with User Segments just works with Content Search Web Parts but some additional work is needed if you want to leverage it with the Search REST API.

Content Targeting with User Segments in SharePoint 2013

One of the great capabilities of SharePoint 2013 Search is Content Targeting with User Segments. Using Query Rules you can specify how the search results of the particular search query should be modified. This allows you to change how the results are sorted but you can also add new results to the original set of search results. In order to have those Query Rules executed, all you need to do is to pass Term IDs of the User Segments for the current user. Although how you define the applicable User Segments for the current user is totally up to you, you might think of some common ways such as tracking the pages visited in the current session, tracking referrals or using information from the profile of the current user if she is logged in on your website.

By leveraging Content Targeting with User Segments you can display more relevant content to the visitors of your website. As a result you should see an increase in the number of pages visited per session and the overall increase of conversion from your website.

Content Targeting with User Segments at Mavention

One great example of leveraging Content Targeting with User Segments in SharePoint 2013 can be found on mavention.com. If you visited our website in the past, you might have noticed that on the homepage we display a number of tiles with some content. Did you know however, that what content is displayed depends on what other pages you have visited on our website?

The first time you will visit mavention.com you will see the default configuration:

Default content on the homepage of mavention.com

In the wide tile we will display the latest news, then we show a static tile, a selected case study, the SharePoint Developer job offer, another static tile, a selected product, the latest blog post and yet another static tile.

Should you however visit a blog post related to WCM you will get a different configuration:

Mavention.com homepage after visiting a blog post related to WCM

This time around the first tile you will see will be the job offer for a SharePoint WCM Specialist, then a static tile, a WCM cased study, the latest WCM blog post, another static tile, the latest WCM news, a selected WCM product and yet another static tile.

As you can see the WCM-related content bubbles up and given that you have just visited a WCM-related blog post we hope that you will visit some other WCM-related content as well.

How we do it

The challenging part about how we use Content Targeting with User Segments on mavention.com has to do with the fact that we combine a number of different results in a single set of results. If you think about it, there is no way that you could build a search query that would give you the latest blog post, the latest news and a selected case study – all in one query.

To make the solution even more complex we not only mix different types of results in a single set of results but we also change the order in which they are displayed. By default we show a press release, a case study, a job offer, a product and a blog post. When targeting content to technical WCM visitors however we start with a job offer, and then show a case study, a blog post, a press release and a product instead.

The way we do it is by defining a number of different Query Rules. Each Query Rule covers only for a number of tiles, for example the Query Rule targeting visitors interested in technology and WCM only loads the relevant job offer, case and blog post. Other two tiles (press release and product) are covered for in a different Query Rule. Each Query Rule has a number of Result Blocks defined, where the content is loaded. The essential piece of configuration is that all Results Blocks should be Ranked so that all of the results can be mixed together in a single set of results.

Result blocks for the Query Rules targeting visitors with interest in technology and WCM

By combining the different Query Rules we can control not only the content displayed in the specific tiles but also the order in which that content is displayed.

On our homepage we have five Content Search Web Parts only one of which executes a search query. The other four Web Parts show results from the retrieved set of results, each one rendering the item at the specific position (ie. first, second, third, etc.) in the result set.

As you browse through our website, we store the types of pages that you have visited and their topics in session. Later on, when you visit the home page again, we retrieve that information and pass it to the search query. As a result, we get a very dynamic homepage showing relevant content all by its own.

Content Targeting with User Segments using the Search REST API

If you’re following my blog you might have known that this November I will be presenting a session on SharePoint Connect Amsterdam about building portals and public-facing websites using the Single Page Application (SPA) approach. As a preparation for that session I decided to build the mavention.com website as a SPA. And this is exactly when I stumbled upon the challenge of leveraging Content Targeting with User Segments when using the Search REST API.

When working with Content Targeting with User Segments and Content Search Web Parts how you pass the User Segments depends on whether you are using client-side or server-side rendering. For client-side rendering using Display Templates you would use the following code snippet:

public class UserSegmentsContentSearchWebPart : ContentBySearchWebPart {  
    protected override void OnLoad(EventArgs e) {
        if (AppManager != null &&
            AppManager.QueryGroups.ContainsKey(QueryGroupName) &&
            AppManager.QueryGroups[QueryGroupName].DataProvider != null) {
            AppManager.QueryGroups[QueryGroupName].DataProvider.BeforeSerializeToClient += DataProvider_BeforeSerializeToClient;
        }

        base.OnLoad(e);
    }

    void DataProvider_BeforeSerializeToClient(object sender, ScriptWebPart.BeforeSerializeToClientEventArgs e) {
        var segments = // your logic for retrieving User Segments
        DataProviderScriptWebPart dataProvider = (DataProviderScriptWebPart)sender;
        dataProvider.Properties["UserSegmentTerms"] = segments.ToArray();
    }
}

If you’re using server-side rendering with XSLT you will need to use the following code snippet instead:

public class UserSegmentsContentSearchWebPart : ContentBySearchWebPart {  
    protected override void OnLoad(EventArgs e) {
        if (AppManager != null &&
            AppManager.QueryGroups.ContainsKey(QueryGroupName) &&
            AppManager.QueryGroups[QueryGroupName].DataProvider != null) {
            var userSegments = // your logic for retrieving User Segments
            AppManager.QueryGroups[QueryGroupName].DataProvider.Properties["UserSegmentTerms"] = userSegments;
        }

        base.OnLoad(e);
    }
}

When working with the Search REST API things get a little more complicated.

You still have to pass the User Segment Term IDs for the current user to the search query using the UserSegmentTerms property. The value type of this property is an array. When setting query properties in Search REST queries you can only set string-value parameters in the query GET request. All other values can only be set using the postquery API which requires you to execute your search query as a POST request.

Following is a sample Search REST API request passing the User Segment Terms for the current user as a parameter:

{
    method: 'POST',
    url: "/_api/search/postquery",
    headers: {
        'Accept': 'application/json; odata=verbose',
        'Content-Type': 'application/json; odata=verbose',
        'X-RequestDigest': requestDigest
    },
    data: JSON.stringify({
        "request": {
            "__metadata": {
              "type":"Microsoft.Office.Server.Search.REST.SearchRequest"
            },
            "Querytext": "size>0",
            "SourceId": "00000000-0000-0000-0000-000000000000",
            "QueryTemplatePropertiesUrl": "spfile://webroot/queryparametertemplate.xml",
            "SelectProperties": {
                "results": [ "Title", "Path", ...]
            },
            "Properties": {
                "results": [{
                    "Name": "UserSegmentTerms",
                    "Value": {
                        "StrArray": {
                            "results": [ "00000000-0000-0000-0000-000000000001", "00000000-0000-0000-0000-000000000002" ]
                        },
                        "QueryPropertyValueTypeIndex": 4
                    }
                }]
            }
        }
    })
}

This object can be used in an AJAX POST request in jQuery, AngularJS or any other framework using similar objects for AJAX requests.

Because this search query is executed in a POST request, we first need to get a hold of a valid Request Digest and set it on the request object (line 7). Next, we define the search query (line 14). In the example above we don’t provide a specific query. Instead we just ask Search to return all results since the relevant results that we are interested in are going to be returned by the Query Rules triggered by User Segments anyway. Then we specify the Result Source that we want to use in the query and which has the Query Rules associated with it (line 15). Since we are using this query with anonymous users, we have to specify the QueryTemplatePropertiesUrl property (line 16). Next, we can specify Managed Properties that we want to retrieve as an array of string (line 18). Finally we pass the User Segment Terms (lines 21-29). It is essential that the array of Term IDs is passed as an array of string. Additionally the QueryPropertyValueTypeIndex property must be set to 4 to indicate that the value passed is of type array of string.

Before you will be able to execute this request with anonymous users you will need to change the queryparametertemplate.xml file to allow passing User Segment Terms, by adding the following line to the WhiteList:

<a:string>UserSegmentTerms</a:string>  

Inconvenient Content Targeting with User Segments and Search REST API

Although you have everything in place for the request to work as expected, whether it will or not depends on your Query Rules and how they return results. If they are configured the same way they are configured on mavention.com you will see only one result in the PrimaryQueryResult block, which is as expected due to the RowLimit = 1 property, but only top three results (instead of five as there are five dynamic tiles on the mavention.com homepage) in the SecondaryQueryResults block where the results from the Query Rules are stored.

Although you might consider a number of different solutions at this point, amongst which ditching the Content Targeting capability of SharePoint altogether and implementing your own, which would allow you to fire the relevant search queries yourself, it turns out that there is an easier way to solve this problem.

If you compare the search queries executed by Content Search Web Parts, where Content Targeting with User Segments just works, and the search queries executed by the Search REST API, where Content Targeting doesn’t work as expected, you will find some subtle differences. Consider the following query:

{
    method: 'POST',
    url: "/_api/search/postquery",
    headers: {
        'Accept': 'application/json; odata=verbose',
        'Content-Type': 'application/json; odata=verbose',
        'X-RequestDigest': requestDigest
    },
    data: JSON.stringify({
        "request": {
            "__metadata": {
              "type":"Microsoft.Office.Server.Search.REST.SearchRequest"
            },
            "Querytext": "size>0",
            "SourceId": "00000000-0000-0000-0000-000000000000",
            "QueryTemplatePropertiesUrl": "spfile://webroot/queryparametertemplate.xml",
            "SelectProperties": {
                "results": [ "Title", "Path", ...]
            },
            "EnableInterleaving": "false",
            "RowLimit": "1",
            "Properties": {
                "results": [{
                    "Name": "UserSegmentTerms",
                    "Value": {
                        "StrArray": {
                            "results": [ "00000000-0000-0000-0000-000000000001", "00000000-0000-0000-0000-000000000002" ]
                        },
                        "QueryPropertyValueTypeIndex": 4
                    }
                },
                {
                    "Name": "EnableStacking",
                    "Value": {
                        "BoolVal": "True",
                        "QueryPropertyValueTypeIndex": 3
                    }
                }]
            }
        }
    }
}

In line 20 we set the EnableInterleaving property to false and in lines 32-38 we set the EnableStacking property to True. By adding those two minor modifications to the search query, you can instruct SharePoint Search to arrange Result Blocks from Query Rules the same way it does it in Content Search Web Parts!

Disabling interleaving will allow you to have Result Blocks from all the Query Rules returned in the SecondaryQueryResults block. Unfortunately, if you look closely, you will see that the results from the different Query Rules are included in order different than what is specified in the configuration of Query Rules.

By enabling stacking Search will move the Result Blocks from the SecondaryQueryResults block to the PrimaryQueryResults block and have them ordered according to the configuration of your Query Rules.

Before you will be able to set those properties on the query for anonymous users you will need to modify the queryparametertemplate.xml file once again and add the following lines to the WhiteList:

<a:string>EnableInterleaving</a:string>  
<a:string>EnableStacking</a:string>  

With all of that in place you should be able to leverage Content Targeting with User Segments in Search REST API just as you would in Content Search Web Parts.

Summary

Leveraging Content Targeting with User Segments in SharePoint 2013 allows you to display more relevant content to your visitors increasing the number of pages per visit and conversion. Although Content Targeting with User Segments just works with Content Search Web Parts, some additional work is needed if you want to use it with the Search REST API. By changing a few search query parameters you can leverage the Content Targeting capability in your apps using the Search REST API.

Comments

comments powered by Disqus