Recently I’ve built a custom connector for Microsoft Flow. Here are a few things I wish the documentation was clearer about.

Process automation for the masses

Microsoft Flow is one of the latest additions to the Office 365 suite. It allows users to build powerful automation scenarios for their work processes. Using Microsoft Flow, they can for example easily manage their e-mail attachments, get notifications when a certain event occurs or share information with their colleagues with a single press of a button. There are tens of connectors and hundreds of templates for users to choose from. And the best part of it all is, that using Microsoft Flow doesn’t require programming skills, which makes it a very appealing tool for non-developer audiences. But there is more.

The real value of Microsoft Flow

When you start using Microsoft Flow, you can choose of tens of first and third party connectors and templates Microsoft made available for you to build your flows. Using what’s available out of the box, you will be able to automate mundane tasks in your every day’s work and spend your energy on what truly matters. But using the standard connectors will take you only that far. To truly benefit of the value that Microsoft Flow offers, you should consider building connectors for the applications your organization uses.

Microsoft Flow custom connectors

Microsoft Flow connectors allow you to use applications available in your organization in your flows. Some connectors give you access to applications provided by Microsoft, such as SharePoint or Outlook. There are also connectors that allow you to integrate third party applications such as Slack or Twitter. But you can also build custom connectors for line of business applications managed by your organization.

Microsoft Flow connectors are not code. While they are a bit more complex to build than flows, they still don’t require programming skills. All you need to know, is how the APIs of the application that you want to integrate in Microsoft Flow look like, which you should be able to get from the application’s documentation. To put it differently: Microsoft Flow connectors tell Microsoft Flow what APIs are there, and which requests and responses they support. Connectors are nothing but metadata for your applications. The actual logic and data processing remains in your application.

What you should know about building connectors for Microsoft Flow

Recently, I’ve built a Microsoft Flow connector that allows you to analyze the quality of your SharePoint applications. And while there is some documentation on how to build Flow connectors, it didn’t quite answer all my questions.

Getting started

When you start building a connector, you have three options. You can either import an existing Swagger definition of your API, import a Postman collection or start from scratch and define the actions and triggers manually.

In fact, there are four options:

1. Import Postman collection (and add missing information manually)
2. Import existing Swagger definition (and add missing information manually)
3. Build Swagger definition including Microsoft Flow metadata and import it in Microsoft Flow
4. Build the connector directly in Microsoft Flow

If you happen to have an existing Swagger definition for your API, it will give you a head start for creating a Flow connector, but most likely, you will need to add some information manually after importing it to Flow. Flow uses custom API properties, such as x-ms-visibility which controls the visibility of your properties in the Microsoft Flow editor, and which most likely your existing Swagger definition won’t have. You can either add these properties to your existing Swagger definition or you can configure them in Microsoft Flow after importing your Swagger definition. Having the custom properties in your Swagger definition gives you the benefit of having all information in one place, but depending how you create the Swagger definition, it might or might not be possible.

Building Swagger isn’t always trivial. While there are some editors that give you feedback whenever something isn’t correct, it’s all based on the JSON schema of a Swagger file, which isn’t always clear about what is wrong exactly. The good news is, that you can create the connector in Microsoft Flow, and once it’s working, you can export it to a Swagger file with everything you configured. If you don’t have Swagger definition for your API, you should definitely export your connector and store it somewhere safe in case something changes and you need to restore it to its previous state.

Can’t rename the connector

When creating a Microsoft Flow connector, one thing you should be certain of, is its name. Once you create a connector, you can’t rename it. The only way to change its name, is to remove it and recreate it from scratch with the new name, breaking all existing flows in result. The good news is, that you can recreate the connector either using the existing Swagger definition or by exporting the connector to a Swagger file, but still you will need to reconfigure all flows using that connector.

When choosing a name for your connector, keep in mind, that at any time you can add new actions and triggers and the connector’s name should be generic enough to accommodate them. Typically it will be the name of your application for which you’re building the connector.

More info…

When I started building my connector, one thing that I totally missed, was the fact, that each of my request parameters was actually a button allowing me to edit its details.

When you click the button, a context menu appears, allowing you to edit properties of the particular property, which is necessary in case you want to change its data type or visibility in the Flow editor.

Processing binary file contents

One of the most cryptic things I’ve come across, was how to get the contents of a binary file correctly in my API. When you build a flow using for example Outlook and OneDrive and analyze its execution, you’ll find the contents of the binary file passed between the actions as as a base64-encoded string. But when your API specifies one of its request parameters as a string, you will get the raw binary data instead, resulting in a broken payload.

It turns out, that in order to get the contents of a binary file encoded as a base64 string in your connector, the request parameter which will hold the data has to be marked as Type: string, Format: binary.

In Swagger it looks like this:

{
  // ...
  "properties": {
    "file": {
      "title": "File contents",
      "type": "string",
      "format": "binary"
    }
  }
}

Your API will then receive a payload with a file property (following the example above, but you are free to choose any name for the property) which by itself will be a JSON object that looks as follows:

{
  "file": {
    "$content-type": "application/octet-stream",
    "$content": "UEsDBAoAAAAAAEthx0wAAAAAAAAAAAAAAAAG..."
  }
}

The $content-type property specifies the mime-type of the file. The $content property contains the base64-encoded contents of the binary file. This is how Microsoft Flow will deliver the contents of a binary file to your API. If your API expects anything else, you will need to adjust it to support a request in this format.

Asynchronous connectors

When your API gets called from Microsoft Flow, it should return a response instantaneously or at least as quickly as possible. But what if your application needs to perform a long-running operation?

Microsoft Flow supports the notion of an async operation. If your API requires some time to complete the action, when called, it should return a 202 Accepted response. Along with the response, it should return two headers: location and retry-after. The retry-after header specifies the number of seconds that Microsoft Flow should wait before checking if the long-running operation completed. The location header, specifies the URL that Microsoft Flow should call to check if the long-running operation completed or not. If it did, that URL should return a 200 OK response. If the operation is still in progress, once again the API should return a 202 Accepted response with the location and retry-after headers.

Let’s take a look at a sample sequence of requests to an API that performs a long-running operation. At first, you issue a POST request to trigger the operation:

POST https://api.contoso.com/v1/long-running-operation
content-type: application/json

{
  "property1": "value1",
  "property2": "value2"
}

The API accepts the request and starts the long-running operation in the background:

202 Accepted
location: https://api.contoso.com/v1/status?id=ABC
retry-after: 10

After waiting for 10 seconds, as specified in the retry-after response header, your API is called again to check if the job completed:

GET https://api.contoso.com/v1/status?id=ABC

As the job is still in progress, the API once again responds with a 202 Accepted response:

202 Accepted
location: https://api.contoso.com/v1/status?id=ABC
retry-after: 10

After waiting for another 10 seconds, your API is called again.

GET https://api.contoso.com/v1/status?id=ABC

This time, the job is completed, so the API returns a 200 OK response along with the job result.

200 OK
content-type: application/json

{
  "resultProperty": "resultValue"
}

What’s impressive about this approach is, that it allows you to easily implement long-running operations in your API without troubling the user building a flow with your connector. As long as your API returns a 202 Accepted response, the flow will be stopped on your connector. As soon as your API returns a 200 OK response, the flow will proceed. And Microsoft Flow will handle all of that by itself, without the user having to do anything.

When async connectors work, they’re nothing short of magic. But they can get pretty inconvenient when they don’t. It’s hard to debug them in case they don’t work as expected. After your flow failed, only the first request and the last response are visible in the debug information. The calls in-between, responsible for checking if the long-running operation completed, are not visible in Microsoft Flow and can be traced only through your API’s logs.

It’s calling something, but not your API

When you’re building a Microsoft Flow connector, at some point you will want to test it. When you do, you might notice, that Microsoft Flow isn’t calling your API directly. Instead it calls a proxy, which then calls your API. In the network tools of your web browser you will find a URL similar to following:

https://europe-001.azure-apim.net/apim/contoso.5f0d59cdcc07448e52.5fb0d954e02c19715b/shared-contoso.5f0d5-7dad6eca-cb38-412e-9790-4811a2225196/api/operation

The proxy is registered when you save the connector for the first time. Typically, things work as expected, and you don’t need to worry about it. But sometimes, when you update the connector to reflect some changes in your API, it might take a moment for the changes to be propagated to the proxy. As a result, even though you have just updated the connector, for a while Microsoft Flow will keep using its previous version which will likely cause errors when calling the updated API. Unfortunately, there isn’t much you can do about it, other than wait a few minutes.

Sharing the connector in your organization

When you build a custom connector, it can be used only by yourself at first. When you’re ready, you can easily share it with others in your organization. You can either choose specific individuals, groups or the whole organization. Sharing a custom connector with others is straight-forward and doesn’t require any formal process to be followed.

If you’re using Microsoft Flow on a free plan, each user can create only one custom connector. So if your organization needs multiple connectors, you either have to switch to a paid plan or distribute the custom connectors over multiple users.

Summary

Through custom connectors for Microsoft Flow you can empower your users to automate using applications in your organization. The process of building a connector isn’t complex. Except for a few details, the Flow documentation is good enough to get you started, assuming you have a documentation of your application’s API. With custom connectors you will be able to truly benefit of using Microsoft Flow in your organization.