microsoft / addingspeechtobots

Adding Speech to Conversational Bots

Overview

This repo contains the walk through for a 2 hour tutorial on adding speech to bots.

In this tutorial we will walk you through two scenarios for integrating speech to text (STT) and text to speech (TTS) capabilities.

The first scenario is based on the JFK Hoover Bot and will showcase how you integrate STT and TTS capabilities into a web app. You will also learn about integrating a custom voice.

The Hoover Bot is a single-page Web app that works in any modern browser and provides a conversational interface to The JFK Files. The JFK Files is a Web app that lets you search a corpus of documents related to the assassination of President John F. Kennedy on November 22, 1963, released by the United States government. Microsoft has presented this technology demonstration, which showcases the power of Azure Cognitive Search, on several occasions.

The second scenario is based on the Study Bot sample which showcases how to integrate STT capabilities into a C# UWP application. This application, lets users learn study terms in three subjects: Geology, Biology and Sociology through a conversational experience. The goal is to enable a more engaging study epxerience.

Prerequisites

1. Azure Account

2. Visual Studio 2017+

3. Bot Framework Emulator

4. Install and configure ngrok. Ngrok has a free version and you don't need to create an account, just download it. If Ngrok is not configured, you'll see a link in your emulator where you can click to configure (edit) it.

5. msbot and Dipatch CLI tools

6. Knowledge of .bot files

7. Knowledge of ASP.Net Core and asynchronous programming in C#

8. For the JFK Hoover bot, you will need access to an instance of the JFK Files demo and Custom Voice Model. You can either configure your own or ask your facilitator for the keys.

Tutorial Steps

Work through the following README files:

1. JFK Hoover Bot

2. Study Bot

Further reading and Documentation

• Bot Framework Documentation
• Bot basics
• Azure Bot Service Introduction
• The 14.nlp-with-dispatch sample. The Qna-Luis-Botv4 sample is largely based on this sample.
• LUIS
• QnA Maker
• QnA Maker Dcoumentation
• Activity processing
• Prompt Types
• Channels and Bot Connector Service
• Channels and Bot Connector Service
• QnA Maker API V4.0
• Add Chit-chat to a QnA Maker knowledge base
• Language Understanding (LUIS) Documentation
• LUIS Programmatic APIs v2.0
• LUIS Endpoint API
• Bing Spell Check API Documentation
• Bing Spell Check API v7 reference
• Integrating QnA Maker and LUIS bot v4 tutorial, using Dispatch
• Speech Services Documentation