isabella232 / cognitive-services-voice-assistant

Welcome to the Microsoft Voice Assistant samples repository! Here you will find samples to help you get started building client application for your bot or Custom Command service. You will also be able to easily deploy a working Custom Command based Voice Assistant to your own Azure subscription

page_type	description	languages	products
sample	Sample code for Microsoft Cognitive Services Voice Assistant	csharp c++	dotnet-core azure-bot-service azure-cognitive-services azure-iot-edge azure-language-understanding azure-speech-text azure-text-speech windows windows-iot windows-wpf windows-uwp Cognitive Services

Microsoft Cognitive Services - Voice Assistant Sample Code

Overview

This repository includes everything needed to build a sample Voice Assistant hosted in your own Azure subscription, with a client application of your choice. Voice Assistants use Microsoft's Speech SDK in the client application to connect to Direct Line Speech Channel and your Bot-Framework bot. Alternatively, Speech SDK can be used to connect to your Custom Commands voice application. Voice Assistants can be keyword activated using keyword model files in this repository or new ones you create.

 

 

You may also find the following 12-minute overview video useful:

If you are writing a Bot-Framework bot, read the tutorial Voice-enable your bot using the Speech SDK, and use sample clients in this repo to help you get started.

Or just browse individual components per the repository content below.

Repository Content

Sample Client Applications

Folder	Description	Language/Platform
clients	Landing page for sample clients
clients\csharp-uwp	UWP voice assistant client sample. An example UWP app that uses Window's new voice activation APIs, together with the Voice Assistant platform, to connect to your bot or Custom Commands application.	C#, Universal Windows Platform (UWP)
clients\csharp-wpf	Windows voice assistant client sample. Generic Windows tool to manually test your bot or Custom Commands application	C#, Windows Presentation Foundation (WPF)
clients\cpp-console	Console client sample for Smart Speaker devices, configured via a json file to connect to your bot or Custom Commands application. It supports microphone input and audio playback. Build scripts for Raspberry Pi, GGEC Speaker and Windows are provided	C++, Windows, Linux
clients\java-android	An Android application (phone or tablet) configured to connect to your bot or Custom Commands application	Java Android

Sample Custom Commands and Scripts

Folder	Description
custom-commands\demos	Landing page for Custom Command demos, including deployment instructions
custom-commands\demos\hospitality	Sample voice commands to control your room (lights, blinds, TV and temperature), including visualization of the results in a hotel room settings
custom-commands\demos\inventory	Sample voice commands to control warehouse inventory (adding or removing different color crates), including visualization of the results
custom-commands\demos\automotive	Sample voice commands to the environment in your car (temperature, .... ), including visualization of the results
custom-commands\demos\careroom	Sample voice commands to control your room (lights, blinds, TV and temperature), including visualization of the results in a hospital room or other healthcare facility
custom-commands\quick-start	JSON Custom Command file for the official quick-start guide
custom-commands\scripts	Continuous deployment scripts for Azure DevOps pipeline

Sample Keyword Models

Folder	Description
keyword-models	A few keyword models (.table files) for you to try out. Or you can create your own -- See Create a custom keyword using Speech Studio

Test Code

Folder	Description	Language/Platform
clients\csharp-dotnet-core\voice-assistant-test	Automated, multi-turn, end-to-end regression test for your bot or Custom Commands application. Supports WAV file input, text or Bot-Framework activities	C# .NET Core

Articles

File	Description
docs\CloudConnectionLogic.md	Discusses options for when to connect to Direct Line Speech channel and how long to keep the connection open
docs\LocalBot.md	Running & debugging your Bot locally while using Direct Line Speech Channel

Root Folder Files

File	Description	Language/Platform
.gitignore	Define what to ignore at commit time
CODE_OF_CONDUCT.md	Code of Conduct for all Microsoft repositories
CONTRIBUTING.md	Guidelines for contributing to these samples
README.md	This README file
LICENSE.md	The license for these samples
SECURITY.md	Information about reporting any security vulnerabilities to Microsoft
NOTICE.txt	License of third party software incorporated in these samples

Clone the Repo

To build any of the sample client applications or deploy custom commands to your Azure subscription, you will first need to clone this GitHub repository:

    git clone https://github.com/Azure-Samples/Cognitive-Services-Voice-Assistant.git
    cd Cognitive-Services-Voice-Assistant

Related Microsoft GitHub Repos

• https://github.com/Azure-Samples/cognitive-services-speech-sdk
• https://github.com/Azure-Samples/Cognitive-Services-Speech-Devices-SDK
• https://github.com/microsoft/botframework-solutions
• https://github.com/Microsoft/BotBuilder-Samples

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Reporting Security Issues

Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (MSRC) at [email protected]. You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Further information, including the MSRC PGP key, can be found in the Security TechCenter.

Copyright (c) Microsoft Corporation. All rights reserved.