intelligentroboticslabs / gb_dialog

GB-dialog

GB-dialog simplifies the task of developing actions or behaviors related to dialogue.

GB-dialog contains the library DialogInterface from which we will inherit to develop our dialogue actions. Each action would be specific to an intent (Dialogflow concepts). The library offers methods to do speech-to-text tasks through Google Speech sevice and methods to do Natural Language Processing tasks through Dialogflow, using the ROS package dialogflow_ros (official package, our custom dialogflow_ros). The library also offers a method to do text-to-speech through the package sound_play.

Installing

Install requirements

If you don't have vcs tool, install it with:

sudo apt-get install python3-vcstool wget libgst-dev libgst7 libgstreamer1.0-* libgstreamer-plugins-base1.0-dev

Clone dialog packages

First of all, clone all the relevant pacakges for dialog. Create a folder dialog and clone all required packages using the tool vcs from vcstool:

cd <workspace>/src
mkdir dialog
cd dialog
wget https://raw.githubusercontent.com/IntelligentRoboticsLabs/gb_dialog/ros2/gb_dialog.repos
vcs import < gb_dialog.repos

cd <workspace>
rosdep install --from-paths src --ignore-src -r -y

and build the workspace.

Setup

Original instructions are in this README. Use it as reference:

Install the requirements for dialogflow_ros:

sudo apt-get install portaudio19-dev
cd <workspace>/src/dialog/dialogflow_ros2
pip3 install -r requirements.txt

Google Cloud and DialogFlow Setup

1. Go to Google Cloud Console.
2. Create a new project.
3. Go to the Kick Setup.
4. Enable API.
5. Create Service Account. Put on the role of owner.
6. Click on your service account. Create key & download the JSON File. Rename and move it t your HOME as ~/df_api.json.
7. Go to DialogFlow Console.
8. Create new Agent & select the project.
9. Edit dialogflow_ros/config/param.yaml and write down your project id. You can find it in the DialogFlow Console, clicking in the gear icon.
10. Add export GOOGLE_APPLICATION_CREDENTIALS='/home/<user>/df_api.json' to your .bashrc and change user.

Use

Below is an example of using the GB-dialog library (example file. First of all we define our new class that inherits from DialogInterface. Then, I must use the function registerCallback to define the handler of the DialogFlow response. You can indicate a specific intent on each callback to receive the DialogFlow responses in differents callbacks, or not to receive all results from DialogFlow in the same callback, up to you!

class ExampleDF: public DialogInterface
{
  public:
    ExampleDF(): nh_()
    {
      this->registerCallback(std::bind(&ExampleDF::noIntentCB, this, ph::_1));
      this->registerCallback(
        std::bind(&ExampleDF::welcomeIntentCB, this, ph::_1),
        "Default Welcome Intent");
    }
...
};

When we have our class instantiated we can use the methods speak or listen. Both methods are syncronous

gb_dialog::DialogInterfaceTest di();
di.speak("Hello world!")

gb_dialog::DialogInterfaceTest di();
di.listen();

Tests

Compile and execute the test

ros2 launch gb_dialog gb_dialog_services_soundplay.launch.py
ros2 run gb_dialog example_df_node

And talk :)

Creating a virtualenv for install Python modules

If you have dependencies issues when you installed the above requirements
Create a virtualenv virtualenv venv --system-site-packages
Activate the virtual env source venv/bin/activate
Install the dependencies pip install -r [__gb_dialog_Path__]/dialogflow_ros/dialogflow_ros/requirements.txt

Remind activate the virtualenv in each shell where you want use dialogflow_ros

gb_dialog's Issues

Migrar sound_play y dialogflow_ros

Para que sea el robot el que habla habría que instalar sound_play en el robot y lo necesario para que dialogflow funcione dentro, entre otras cosas conseguir que se conecte a una red wifi.

Problema con Raspberry

Buenas,
Estoy intentando usar dialogflow en una Raspberry Pi 4, usando ROS Noetic y Ubuntu server , ROS funciona bien, pero al lanzar

roslaunch gb_dialog gb_dialog_services_soundplay.launch

y despues un nodo cliente en otro terminal, el primer terminal de el error

[WARN] [1660125629.085471]: DF_CLIENT: Unknown Exception Caught:
None Exception iterating requests!

Y no funciona, el mismo codigo funciona bien en el ordenador. ¿Alguna idea de cómo solucionarlo?

`[ERROR]` trace from `soundplay_node.py` when using `gb_dialog` interface.

Hi there!

I have observed a possible bug when using the gb_dialog interface. After launching...

ros2 launch gb_dialog gb_dialog_services_soundplay.launch.py

When you use the sound_play package from CLI, like so:

ros2 run sound_play say.py 'This is a test.'

soundplay_node.py outputs this:

[soundplay_node.py-1] [INFO] [1685357785.677093124] [sound_play]: sound_play action: Succeeded

However, when the speak feature is used from inside the gb_dialog interface (Using the method DialogInterface::speak(std::string str), soundplay_node.py outputs this as an [ERROR] trace:

[soundplay_node.py-1] [ERROR] [1685357793.194510997] [sound_play]: callback: sound_play.msg.SoundRequest(sound=-3, command=1, volume=1.0, arg='This is a test.', arg2='voice_kal_diphone')

This apparently does not interfere with the behavior of the package, other than being a bit confusing when debugging.

It's also not clear to me that this is directly triggered by gb_dialog, but I've assumed so since that's the use case where it happens to me.

EDIT. Forget to mention that this phenomenom was observed in the ros2 branch.

Image evidence:

C ya!

Función de parada o tiempo de espera

Buenas tardes @jginesclavero ,
Usando el repositorio de gb_dialog, hemos encontrado un problema. Al tener lanzado un nodo durante cierto tiempo haciendo listen(), cuando pasa un tiempo se queda bloqueado y deja de escuchar.