Bringing knowledge to Microsoft 365 Copilot declarative agents using Azure AI Search is the one option that we don’t talk about enough. And it’s a shame, because it’s a great way to strike the balance between having more control over indexing and relevance, without the complexity of standing up a custom engine agent with a language model deployment. Here’s why.

Know it all

We all want our agents to know as much as possible. After all, the more they know, the more helpful they are, right? While the language models that back them keep improving, they’re still far from including everything. A lot of the knowledge that we need agents to have is specific to our organization. Training a custom model on organizational knowledge, in most cases, just doesn’t make sense. The information changes too often. It’s also not business-critical enough to justify the cost of training a model for most. In many cases, knowledge items also need to be secured so that only those with the right permissions can see them. This is why we’re resorting to RAG as the mechanism to bring additional knowledge to our agents.

RAG on Microsoft 365 Copilot agents

When building Microsoft 365 Copilot agents, we have several options for bringing additional knowledge to them.

OneDrive/SharePoint

The first, and the most obvious option, is to use OneDrive or SharePoint to store your knowledge items. It’s a hands-off approach that just works. All you need to do is upload your files to a document library, and Microsoft 365 will take care of the rest. It’ll automatically index the files and make them available to Microsoft 365 Copilot and its agents. It also automatically ensures that only those with the right permissions can see the files in agents’ responses. But this simplicity comes at a cost.

Microsoft 365 doesn’t support indexing all file types. While it does support a lot of them, if your organization uses one that’s not supported, Microsoft 365 won’t index them, and there’s nothing you can do about it.

You also don’t get to choose how Microsoft 365 indexes your files. You simply upload files, and Microsoft 365 takes care of the rest. You have no control over how the files’ contents are parsed and how they’re chunked.

Finally, you don’t get to choose the relevance of the files. For most, that’s a good thing, because let’s face it: most of us aren’t experts in relevance algorithms and search strategies. But if you understand your domain knowledge and have specific requirements for telling which knowledge items are more relevant than others, you might want to have more control over the relevance.

Custom engine agents

On the other side of the spectrum are custom engine agents. Building a custom engine agent means that you’re bringing the full infrastructure: from the compute resource that hosts the agent to the language model that powers it. While it gives you full control over indexing additional data sources, tool usage, and orchestration, it also means additional complexity to ensure security and responsible use of AI. There’s however a middle ground between the two extremes.

Microsoft Graph connectors

Another option at your disposal is to use Microsoft Graph connectors. Graph connectors ingest data from external systems into Microsoft 365. There are many connectors available in Microsoft 365 for you to use, and you can also build your own.

If your main challenge is having knowledge stored in unsupported data formats, then Graph connectors are a great option you should consider. Using Graph connectors, you basically create a push-based search index inside Microsoft 365. You define the schema for your data, specify which properties are searchable and filterable, and push the content, along with its access control list, to Microsoft 365.

Since you control pushing the content to Microsoft 365, you have some degree of control over how the content is indexed. It’s your responsibility to extract the content from the source system and transform it into a structure compatible with Microsoft 365. This allows you to parse complex elements and include all the necessary information to convey their meaning. If your content items are large, you can also break them down into manageable chunks to help Microsoft 365 find the relevant information. For each chunk, you can also specify the URL to the original content, so that when users get a citation, they’re pointed to the source system. There’s, however, one thing that Graph connectors don’t let you control.

When you ingest content into Microsoft 365 using Graph connectors, you don’t get controls to define the relevance of the content. Microsoft 365 uses its predefined relevance algorithms to determine the relevance of the content. If you have specific requirements for relevance, the default settings might not give you the results you’re looking for.

Azure AI Search

Often, we turn to building custom engine agents too soon, forgetting that we can use Azure AI Search to bring additional knowledge to Microsoft 365 Copilot agents. Azure AI Search is a managed search service that allows you to build search-based solutions. Similarly to Graph connectors, Azure AI Search allows you to have full control over indexing your content. But unlike other solutions, it also allows you to define custom ranking models. This means that you can define which information is the most relevant for your organization. As a bonus, you can decide whether you want to use lexical, semantic, vector, or hybrid search. This gives you the flexibility to choose the best search strategy for your organization and scenario.

Azure AI Search is a service on Microsoft Azure, and there is a cost involved with using it. Still, if your solution requires controlling context indexing and relevance, it’s a great solution that you should consider before building a custom engine agent.

Integrating declarative agents with Azure AI Search

Integrating declarative agents with Azure AI Search is no different than integrating an API plugin. For the declarative agent, Azure AI Search is just another API. By default, Azure AI Search is secured with an API key. To integrate it with a declarative agent, register the API key in the Teams Developer Portal and include the registration ID in the agent’s manifest. Then, you include the Azure AI Search’s API spec in the project. Here’s an example:

openapi: 3.0.4
info:
  title: Knowledge base search
  description: This API allows you to search for relevant knowledge in Azure AI Search indexes.
  version: v1.0
servers:
  - url: https://${{AI_SEARCH_INSTANCE}}.search.windows.net
security:
  - apiKeyAuth: []
paths:
  /indexes/{index}/docs/search:
    post:
      x-openai-isConsequential: false
      description: Search knowledge in Azure AI Search indexes.
      operationId: searchKnowledge
      security:
        - apiKeyAuth: []
      parameters:
        - name: api-version
          in: query
          required: true
          schema:
            type: string
            default: "2024-11-01-preview"
        - name: index
          in: path
          required: true
          schema:
            type: string
            default: "${{AI_SEARCH_INDEX}}"
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                search:
                  type: string
                count:
                  type: boolean
                  default: true
                queryType:
                  type: string
                  default: semantic
                semanticConfiguration:
                  type: string
                  default: knowledge
                captions:
                  type: string
                  default: extractive
                answers:
                  type: string
                  default: extractive|count-3
                queryLanguage:
                  type: string
              required:
                - search
                - count
                - queryType
                - semanticConfiguration
                - captions
                - answers
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: string            
components:
  securitySchemes:
    apiKeyAuth:
      type: apiKey
      in: header
      name: api-key
      description: API key for Azure AI Search

Finally, describe the search API operation in an API plugin for the agent:

{
  "$schema": "https://developer.microsoft.com/json-schemas/copilot/plugin/v2.2/schema.json",
  "schema_version": "v2.2",
  "name_for_human": "Knowledge base",
  "description_for_human": "This plugin allows you to search knowledge in Azure AI Search indexes.",
  "description_for_model": "This plugin allows you to search knowledge in Azure AI Search indexes.",
  "namespace": "m365knowledge",
  "functions": [
    {
      "name": "searchKnowledge",
      "description": "Search knowledge in Azure AI Search indexes.",
      "capabilities": {
        "response_semantics": {
          "data_path": "$.value",
          "properties": {
            "title": "title",
            "url": "url"
          },
          "static_template": {
            "file": "adaptiveCards/searchKnowledge.json"
          }
        }
      }
    }
  ],
  "runtimes": [
    {
      "type": "OpenApi",
      "auth": {
        "type": "ApiKeyPluginVault",
        "reference_id": "${{APIKEYAUTH_REGISTRATION_ID}}"
      },
      "spec": {
        "url": "apiSpecificationFile/openapi.yaml"
      },
      "run_for_functions": [
        "searchKnowledge"
      ]
    }
  ]
}

Summary

Bringing knowledge to Microsoft 365 Copilot declarative agents using Azure AI Search is a great way to strike the balance between having more control over indexing and relevance, without the complexity of standing up a custom engine agent with a language model deployment. It allows you to define custom ranking models and choose the best search strategy for your organization. If you’re looking for a solution that gives you more control over indexing and relevance, consider using Azure AI Search before building a custom engine agent.