Cloud4Good: Building an Inclusive Chat Bot for Athletes with Disabilities


In the relatively new space of Conversations as a Platform, many organizations have begun to incorporate chat bots into the structure of their business as tools to drive sales and manage customer relations. But beyond these use cases, what else can bots do? For five days in February, a team composed of four Technical Evangelists, an MVP, and a Paralympian hunkered down in a board room at Microsoft Vancouver to answer this question.

.

.
The objective of the project was to build a conversational interface and analytics dashboard for viaSport BC’s newest inclusive sport initiative. Tasked by the provincial government to be the lead agency responsible for promoting and developing amateur sport in British Columbia, viaSport's purpose is to champion positive changes so that more British Columbians thrive via sport and physical activity. One challenge in this space is the access to resources for the community of athletes with disabilities. To tackle this, the team built an intelligent chat bot to simplify the search for resources, allowing coaches, sport leaders, and participants in all sports to provide appropriate programming and services to athletes with a disability.

Building the Bot

The core bot functionality was built leveraging the Bot Framework SDK in C#. Since conversation is still a fairly new platform, best practices for bot design and development are still being determined. In the exploration of this new genre of development, a few design patterns emerge.

  • Knowledge Base Pattern: when a bot leverages a corpus of data to retrieve answers to user queries;
  • Human Handoff Pattern: when a bot gathers information from the user and directs them to an appropriate human representative, sharing useful information with the representative to help them assist the user;
  • Bot to Web Pattern: when a bot send the user to a web page to complete some additional tasks, which then redirects the user back to the bot to complete the flow.

The viaSport bot leverages the Knowledge Base design pattern, where in this case the knowledge base is the database of resources curated by viaSport.

Building a bot using the Microsoft Bot Framework is a fairly straightforward process. Since bots are designed to meet users in familiar, simple chat interfaces, very little interface design needs to be done. Interaction design, however, is key. One of the key advantages of using a bot is simplifying the user experience and providing the user with a one-stop-shop for the task or tasks they want to accomplish. The development team quickly discovered that building a bot that appears simple to the user is a complex process. In order to be able to hold an effective and natural conversation, the bot’s dialog must be rich enough to recognize state, keep track of the user’s intention, recognize the many ways a user might phrase a query with the same intention, and proactively handle user error. Failing to do so will create a bot with a very dry user experience, eliminating the benefit of creating a bot.

 

Building the Brains

To build a better bot, three Cognitive Services APIs were implemented: LUIS, Bing Web Search, and Spell Check. LUIS is used to guide the conversation through recognizing intents, Bing Web Search is used when there are no results found in the database for the user’s query, and Bing Spell Check is used to reduce margin for human error by correcting the spelling of user input before querying the database or storing entities.

Implementing these APIs is as simple as generating an API key pair from microsoft.com/cognitive, structuring a call out to the API in question, and parsing the JSON response to pull out the desired information. Here is how the Bing Web Search API is implemented in the viaSport bot:

The Bing Web Search API is used in the bot to search the web for results related to the user’s query if no relevant results were found in the viaSport database. This feature was not in the original goals, but after discussion with viaSport, the team determined that the ultimate purpose of the bot is to be as helpful as possible to the user, so rather than having them leave the viaSport site if they don’t find the information they are looking for, the bot can continue to assist them in their web search.

The Bing Web Search API is implemented in the code in the OnWebSearch Task. This Task builds a query from the entities that have been collected from the user, and structures it as follows:

query = $"{intent} {sport} {subject} {disability}";

For example, if the user’s query was, “How do I coach volleyball to a student with spinal cord bifuda?” the query sent to Bing Web Search will be “coaching volleyball student spinal cord bifuda.” If the user’s query was, “I want to find a program where my daughter can ski. She is in a wheelchair,” the query sent to Bing Web Search will be, “programs for ski daughter wheelchair.” The intents are converted to natural phrases using a simple check, as shown below:

var intent = "coaching";

if (intent.Contains("resource"))

{

intent = "information on";

}

else if (intent.Contains("program"))

{

intent = "programs for";

}

 

The next step is to build the query strings, as follows:

var client = new HttpClient();

var queryString = HttpUtility.ParseQueryString(string.Empty);

client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "<BING WEB SEARCH API KEY HERE>");

 

var sport = _entities["SportName"].Entity ?? "sports";

var subject = _entities["Subject"].Entity ?? "people";

var disability = _entities["DisabilityType"].Entity ?? "with disabilites";

 

query = $"{intent} {sport} {subject} {disability}";

 

queryString["q"] = query;

queryString["count"] = _maxReferences.ToString();

queryString["offset"] = "0";

queryString["mkt"] = _language;

queryString["safesearch"] = "strict";

 

var uri = "https://api.cognitive.microsoft.com/bing/v5.0/search?" + queryString;

 

Once the query string is built it is posted to the API, the results are deserialized and sent to the user as cards using PostResultsToUserAsync:

 

var response = await client.GetAsync(uri);

if (response.IsSuccessStatusCode)

{

var jsonString = response.Content.ReadAsStringAsync();

var searchResults = JsonConvert.DeserializeObject<SearchResponse>(jsonString.Result);

 

var references = new List<Reference>();

foreach (var i in searchResults.WebPages?.Value)

{

var reference = new Reference()

{

Title = i.Name,

Subtitle = i.DisplayUrl,

CardText = i.Snippet,

ReferenceUri = i.Url,

};

references.Add(reference);

}

await SendResultsToPersonAsync(context, references);

}

else

{

await context.PostAsync("Sorry looks like we are having some issues with our websearch!");

await OnContactViaSport(context);

}

 

Here is an example of the result of asking the bot for resources for wheelchair equestrian activities (for which there were no resources in the database at the time of writing this post):

viaSport bot

Building the Impact

By simplifying the navigation of resources and identifying gaps through this bot and accompanying analytics dashboard, viaSport will see more inclusive programs come to fruition and more coaches become trained in working with those with a disability. Sport and physical activity for those with a disability will become a daily conversation in the sport sector. Combined, these will be the catalyst for more people with a disability participating locally, regionally, provincially, nationally and at all levels whether it be a playground, a school yard, a local sport team or increased participation on one of the Canadian national teams.

For full implementation details, check out and star our project on GitHub! http://github.com/viaSport/viaSport-Inclusion-Bot

 

Special thanks to Mark Schramm of FusionWare; Sergii Baidachnyi, Nastassia Rashid, and Yash Manivannan of Microsoft; and Elisabeth Walker-Young of viaSport for being part of this development team.

Comments (1)

  1. Mark Schramm says:

    Thanks Sage for laying this out so well. It was fun and we learned a lot along the way.

Skip to main content