rocketchat / apps.dialogflow

Integration between Rocket.Chat and the Dialogflow Chatbot platform

Apps.Dialogflow

Integration between Rocket.Chat and the Dialogflow Chatbot platform

Installation steps:

1. Clone this repo and Change Directory:
   git clone https://github.com/RocketChat/Apps.Dialogflow.git && cd Apps.Dialogflow/

2. Install the required packages from package.json:
   npm install

3. Deploy Rocket.Chat app:
   rc-apps deploy --url http://localhost:3000 --username user_username --password user_password Where:

   • http://localhost:3000 is your local server URL (if you are running in another port, change the 3000 to the appropriate port)
   • user_username is the username of your admin user.
   • user_password is the password of your admin user.

   For more info refer this guide

How to get Google Credential File or Private key file

In order to connect to Dialogflow, this app requires a following credentials from Dialogflow.

1. Project Id
2. Client email
3. Private Key

You can find all these credentials in a JSON file, which u can get from here. Under Create a service account and download the private key file section there, you will find detailed instruction about how to get the JSON file. After obtaining this JSON file, you can proceed to the next section.

Rocket.Chat Apps Setup

1. First go ahead n create a Bot User. Login as administrator, then goto Setting > Users. There create a new Bot User. This new user should have these 2 roles.
   

   1. bot
   2. livechat-agent

2. Then configure the app to automatically assign a livechat-visitor to this bot. To do so, goto Setting > Livechat > Routing or Setting > Omnichannel > Routing. There enable Assign new conversations to bot agent Setting.

3. The app needs some configurations to work, so to setup the app Go to Setting > Apps > Dialogflow. There, fill all the necessary fields in SETTINGS and click SAVE. Note all fields are required.

   Some of the fields in SETTING include

   1. Bot Username (required)
      • This should contain the same bot username which we created above in Step 1
   2. Project Id (required)
      • This corresponds to project_id property of the Google Credentials File obtained from Rocket.Chat Apps Setup section above
   3. Client Email (required)
      • This corresponds to client_email property of the Google Credentials File obtained from Rocket.Chat Apps Setup section above
   4. Private Key (required)
      • This corresponds to private_key property of the Google Credentials File obtained from Rocket.Chat Apps Setup section above.
      • Kindly note that this value will be very long. So please take extra care while copy/paste.
   5. Fallback Responses Limit (optional)
      • The app will automatically trigger handover if consecutive fallback intents are triggered N no of times. This setting defines this value N.
      • Eg. Suppose the bot is not able to answer visitor's consecutive 3 answers, and this setting threshold is 3. In such case, the app will trigger an handover to an online agent on its own.
      • To Deactivate this feature, simply set the value to 0.
   6. Target Department for Handover (optional)
      • Enter the department name where you want the visitor to be transferred upon handover.
   7. Handover Message (optional)
      • The Bot will send this message to Visitor upon handover
   8. Service Unavailable Message (optional)
      • The Bot will send this message to Visitor if service is unavailable like suppose if no agents are online.
   9. Close Chat Message (optional)
      • This message will be sent automatically when a chat is closed
   10. Hide Quick Replies (required)
       • If enabled, then all quick-replies will hide when a visitor clicks on any one of them

4. (Optional Step) Lastly you can test your Dialogflow Connection by viewing App Logs. To view the logs, goto App Page (Setting > Apps > Dialogflow). There click on menu item (3 vertical dots icon) and then select View Logs. There select the most recent onSettingUpdated title. If you see ------------------ Google Credentials validation Success ---------------- message, then it means your setup is fine. If you don't see this message, then recheck your Dialogflow credentials.

Dialogflow's API

The app provides API to trigger specific actions. The URL for the API can be found on the Apps Page(Setting > Apps > Dialogflow). Currently the app provides 2 APIs.

1. Incoming API/Endpoint

   This endpoint can be used to trigger specific actions. The list of supported actions include

   1. Close Chat
      To close a chat
      • REST API Documentation for this endpoint can be found here
   2. Handover
      To perform a handover
      • REST API Documentation for this endpoint can be found here
   3. Trigger Event
      To trigger an event on Dialogflow
      • REST API Documentation for this endpoint can be found here
   4. Send-Message
      To send a message as a bot to Visitor
      • REST API Documentation for this endpoint can be found here

2. Fulfillment API/Endpoint

   The fulfillment endpoint will enable the app to handle asynchronous messages. More information on it here

Adding Quick Replies support to your Dialogflow Bot

• To add quick Replies you can make use of Custom-Payload for Responses on Dialogflow Console. ( Under any Intent > Responses > Add Responses > Custom Payload )
• A box labelled Custom Payload will appear. There enter the Quick Replies you want following the format here

apps.dialogflow's Issues

[New Feature] Add Synchronous handover support in the App

Sync handover support means - When the bot is not able to answer the user's query N no of times continuously, then the app will automatically transfer control from Bot to an online Livechat Agent.

[FEATURE] Configurable bot handover messages in app settings

• Live Agent Transfer Message
  • Message to post when attempting handoff
• Live Agent Transfer Error Message
  • Message to post when an error occurs during handoff
• Technical Difficulty
  • Message to post when dialogflow is unreachable, or other client-side exceptions occur

[FEATURE] Option to keep dialogflow session alive while user is in a handed-off session

When enabled, this feature would tell the app to maintain its session with Dialogflow.

Dialogflow has a default session timeout of 20 minutes, meaning if a user spends >20min in another department and then returns to the bot, Dialogflow would create a new session/sessionID for it and lose any contexts associated with the original chat.

One option to achieve this, is to document an innocuous event like session_maintenance, and have an intent in Dialogflow that accepts it but doesn't have a text response.

When the user closes the widget and terminates the chat, the app can stop maintaining the session and send the chat terminated event #19 (its probably best to cap the maintenance at some configurable max time, like 2 hours, to prevent runaway maintenance processes)

[Improve] Add support for multiple messages within a response

If the app receives a response with multiple messages, it should be able to display them as separate messages in proper order to the visitor.

Example - If the dialogflow receives a message like this, then it should generate 1 text message and one quick reply message

[FEATURE] Accept embedded token to send to dialogflow via welcome event

within the embedded widget wide, we plan to pass a token (in a dict) to rocket.chat. Ideally the dialogflow app should make the dict it recieves into parameters that it sends along with the welcome event when chats start

Deployment error: Direct installation of an App is disabled.

Hi,

When I try to perform the deployment according to the manual the error appears ...

[FEATURE] Parsing of disable-input payloads

Until the UIKit is updated, we cant actually implement the disabling of the input, but in the meantime we should define and parse what disable-input payloads look like, and ensure to the best of our ability that they work alongside quick-replies

Example payload format:

{
  "disableInput": true,
  "disableInputMessage": "Please select from the provided options"
}

[FEATURE] API hook to trigger a Dialogflow event

Instead of follow-up event payloads...

Add aa POST API endpoint that accepts a session ID and an event and/or message

When a message is provided, it posts the message to the room as if its from the bot.
When an event is provided, it sends an event trigger to dialogflow from the user's session.

How do I install Google Dialogflow the configuration connection information?

https://www.youtube.com/watch?v=rZjvxS48RUM
this youtube no Steps to connect to Google Dialogflow

I wish the tutorial was more detailed
The robot can't be used now

Error code 475 trying to install app via Marketplace

Hi guys,

I'm trying to make a simple install of this App but I get this error after clicking at the install button:

I have tried in two different instances.

Thanks!

sending messages to the user from an external API

How sending messages to the user from an external API when my bot connect with dialogflow?

[FEATURE] Send end_live_chat dialogflow event when chat ends with agent and is returned to bot

[New Feature] Add Asynchronous handover support

Async handover support means - When a visitor asks the bot that they want to speak with an online agent, then Dialogflow will detect this intent and trigger a Webhook event. The apps responsibility will be to catch this event and transfer the control from Bot to an online agent

[FEATURE] confirm necessary values are set when enabling a feature

You could enable Ask Visitor if They Would Like a Transcript After Chat Closed on /admin/Omnichannel without setting any message on Message to Show When Asking About Transcript
With this setting the widget displays an error message (something went wrong) to the visitor, which is not good.

Also, even if the server doesn't have a SMTP credential set, you could turn on this setting, and the agent can press the "send transcript" button. This means that the agent thinks they have sent the email while they actually don't.
(To remove the SMTP credential for confirmation, as of 3.14.0, you need to go to the SMTP setting on the Admin and clear the setting and then restart the server)

This is a minor feature request compared to others such as "detect if the user's still on page" as you could get around them if you are careful, but I still want this level of basic validation.

[Feature] Add new action to Incoming endpoint to trigger custom events on Dialogflow

Proposal

I am proposing to add a new action which will enable a developer to trigger a custom-event on Dialogflow.

The endpoint will receive a payload something like this

{
   "action":"trigger_custom_event",
   "sessionId":"<session_id>",
   "actionData":{
      "eventName":"<name-of-the-event-to-be-triggered>"
   }
}

Upon receiving this request, it will make use of detectIntent Rest API endpoint of Dialogflow as described here. The response from this endpoint will be the message corresponding to the eventName configured in Dialogflow.

[FEATURE] Option to trigger handover if errors/exception occur

If we run into a technical issue in the middle of a chat, or right from the start, we should have an option to trigger a hand off to the live agent department to take over.
An important thing is to stop doing so if the chat is transferred back to the bot, since we dont want an infinite loop of being transferred due to errors on either side

Example errors:
failure to reach DF
incorrect credentials when calling DF endpoints

Update Quick replies documentation for handover

update the handover button documentation in the QuickReplies.md as the departmentName key should be inside the data key:

• Example Structure:

{
   "text": "Perform Handover",
   "buttonStyle": "primary",
   "actionId": "df_perform_handover",
   "data": {
      "departmentName": "sales"
   }
}

Missing permissions when installing from the marketplace

Rocket Chat version: v3.16.0

Upon installing from the marketplace, after configuring settings in the app's settings, the "onSettingsUpdated" event logs return:

debug: 2021-06-29T13:21:19.475Z Caller: anonymous OR constructor
[
  "onSettingUpdated is being called..."
]

error: 2021-06-29T13:21:19.481Z Caller: run -> run
[
  "Error occurred while sending a HTTP Request"
]

debug: 2021-06-29T13:21:19.481Z Caller: anonymous OR constructor
[
  "'onSettingUpdated' was successfully called! The result is:",
  null
]

Digging into the server logs I see:

server.js:204 API ➔ debug POST: /api/apps/21b7d3ba-031b-41d9-8ff2-fbbfa081ae90/settings
Error: Failed to call the method  as the app (21b7d3ba-031b-41d9-8ff2-fbbfa081ae90) lacks the following permissions:
["networking"]. Declare them in your app.json to fix the issue.
reason: undefined
    at AppHttpBridge.<anonymous> (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/bridges/HttpBridge.js:20:22)
    at Generator.next (<anonymous>)
    at /app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/bridges/HttpBridge.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/bridges/HttpBridge.js:4:12)
    at AppHttpBridge.doCall (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/bridges/HttpBridge.js:19:16)
    at Http.<anonymous> (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/accessors/Http.js:60:63)
    at Generator.next (<anonymous>)
    at /app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/accessors/Http.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/accessors/Http.js:4:12)
    at Http._processHandler (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/accessors/Http.js:37:16)
    at Http.post (/app/bundle/programs/server/npm/node_modules/@rocket.chat/apps-engine/server/accessors/Http.js:28:21)
    at DialogflowClass.generateNewAccessToken (evalmachine.<anonymous>:35:41)
    at OnSettingUpdatedHandler.run (evalmachine.<anonymous>:21:43)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
server.js:204 API ➔ debug GET: /api/apps/21b7d3ba-031b-41d9-8ff2-fbbfa081ae90?marketplace=true&update=true&appVersion=1.2.2
server.js:204 API ➔ debug GET: /api/apps/21b7d3ba-031b-41d9-8ff2-fbbfa081ae90/apis
server.js:204 API ➔ debug Success {
  statusCode: 200,
  body: { apis: [ [Object], [Object] ], success: true }
}

Compiling the application locally and installing the zip works, presumably due to the permissions not being checked for self-compiled apps?

[FEATURE] Option to send 'welcome' event on chat start

When a chat with the bot first initializes, send the dialogflow default 'WELCOME' event to the bot.

Make sure it can also provide parameters in the call, starting with the roomId

(this is especially useful for when RocketChat/Rocket.Chat.Livechat#441 happens)

[FEATURE] Send event to dialogflow when user terminates chat

Add an App configuration setting for sending an event when the user closes a chat
Send the event when users closes the chat

Dialogflow handover

I have a webhook (Python / Flask) receiving requests from Dialogflow. I configured this webhook so that when I receive a request containing an action called 'livechat', this request will be forwarded, via requests.post, to the internal rocket.chat server, according to the app's configuration, however, I am receiving "{" error "as a return: "Cannot read property \ 'isFallback ' of undefined"} ", in the body of the request received:" intent ": {" name ":" projects / atendimentoorbitel-ea9x / agent / intents / aceb9f52-185d-4954- 8bac-46b2a9c57916 "," displayName ":" human attendance "," isFallback ": true},". What am I doing wrong?

[FEATURE] Bot Typing indicators

< Requires UIKit to enable in-app setting of typing indicators >

• Add configurable delay time between messages (will probably be around 0.8s) so user is not overwhelmed by multiple messages at once
• Activate the typing indicator when requests go outbound to dialogflow, then deactivate when we get our response back
• Add typing indicator for delays for followup event payloads (this would just be a toggle on, and would deactivate on the next message sent from either party)

DF custom payload:

{
  "followup": {
    "event": "event_name",
    "delay": 5000
  }
}

Define a Structure for Quick-Replies

I'm proposing the following structure which will allow users to add Quick Replies type of messages

Structure

{
  "quickReplies": {
    "title": string,
    "quickReplies": [
      string
    ]
  }
}

Field description

1. title

• Type: string
• Description: The title of the collection of quick replies

2. quickReplies[]

• Type: Array of string
• Description: The collection of quick replies.

Example

In this example, there are 2 simple text messages and one quickReply message

• Dialogflow Dashboard
  
• Corresponding Livechat widget
  
• Corresponding Dialogflow payload

{
   "responseId":"9375ee56-b2cb-475f-bc04-2d9b8bab9d3b-e13762d2",
   "queryResult":{
      "queryText":"I want to talk with a live agent",
      "parameters":{

      },
      "allRequiredParamsPresent":true,
      "fulfillmentText":"My Name is John. I'm here to help you!",
      "fulfillmentMessages":[
         {
            "text":{
               "text":[
                  "Hello"
               ]
            }
         },
         {
            "text":{
               "text":[
                  "My Name is John. I'm here to help you!"
               ]
            }
         },
         {
            "payload":{
               "quickReplies":{
                  "title":"First of all, Are u a Rocket.Chat Customer?",
                  "quickReplies":[
                     "Yes, I am",
                     "No, I am not"
                  ]
               }
            }
         }
      ],
      "intent":{
         "name":"projects/dialogflowtest-270008/agent/intents/df5d4300-331b-4fc8-bcd4-159acec5d491",
         "displayName":"transfer_to_liveagent"
      },
      "intentDetectionConfidence":0.71203804,
      "languageCode":"en"
   }
}

Getting Error while installing

Hello,
I am trying to implement the project
I have followed these steps - https://github.com/RocketChat/Apps.Dialogflow
my node version is - v12.16.1
but I am getting this error how can I solve this?

https://imgur.com/SZQPBZQ.jpg

[Optimisation Task] Caching the JWT used for authentication

In #2 the JWT is being generated for each message. This task can be optimized caching the JWT for reuse.

Personal Note -> Pay attention to the timezones while handling expiration.

[FEATURE] Add any unrecognized custom payloads as message attachments

this should allow for future livechat widget changes to interpret these attachments and act accordingly, without having to update this app each time

this should be able to handle any number of payloads on an intent

[IMPROVE] Suppress default fallback response when we are handing off

When we set a fallback limit for handoffs, we should suppress the 'sorry i didnt understand that, can you rephrase' -type messages from Dialogflow in favor of the configured 'Handover Message', text if set

[FEATURE] Quick-reply buttons

• Display quick reply buttons from DF response payloads
• When clicked, it should send payload string to DF and display in client as if the user entered it

custom DF payload example

{
  "quick_replies": [
    {
      "payload": "Chat with live agent",
      "content_type": "text"
    },
    {
      "content_type": "text",
      "payload": "Not now"
    }
  ]
}

[FEATURE] Followup message support

• When given a followup message payload, client should wait delay milliseconds and then trigger the Dialogflow event

DF custom payload example

{
  "followup": {
    "event": "some_event",
    "delay": 5000
  }
}

Getting Error while connecting Dialogflow cx to Rocket.chat gold account

Hello @murtaza98 , @renatobecker, and others,
ok, so I have connected Dialogflow es chatbot to rocket.chat and it is working fine but when I try to connect Dialogflow cx agent using the service account credentials it is giving me an error like this "IAM permission ‘dialogflow.sessions.detectIntent’ on ‘projects/xxxxxxxxxxx/agent’ denied.“PERMISSION_DENIED” so is it possible to connect dialogflow cx bot to rocket.chat?

and also right now I am using
https://dialogflow.cloud.google.com/cx/projects/xxxxxx/locations/us-central1/agents/xxxxx_projectID_xxxxxx/
Project Id which is at the last of the URL in the rocket.chat dialogflow app(key has all the permissions(dialogflow admin and client))

maybe I think I am not using project ID correctly, dialogflow es has seperate project ID for each agent but about dialogflow cx there is one project id and multiple agents in the projects

Note - dialogflow es is working fine with rocket.chat

[New Feature] Add support for async hooks in App

The main goal will be to add async hooks to the app which will enable the app to leverage Dialogflow's Webhook feature.

My plan will be to carry out the following tasks

• Study the Dialogflow's webhook structure
• Identify all the generic webhooks which we can provide by default (eg - Chat finished)
• Implementation

[FEATURE] Bot Session Timeout warnings

• Allow a configurable timeout warning for user-bot conversations
  • Similar to the agent timeout but based on a configurable value set on the rocket.chat server. Could also be based on the Dialogflow timeout value, but currently we just set this value manually.
  • An event should be sent to the NLP when this timeout is reached so we can send a message to the user saying that their session has timed out before ending the chat.
  • Send end_session dialogflow event when chats timeout

Getting Error while installing the project

Hello @murtaza98 ,
I am trying to implement the project
I have followed these steps - https://github.com/RocketChat/Apps.Dialogflow
my node version is - v12.16.1
but I am getting this error how can I solve this?
https://imgur.com/SZQPBZQ.jpg

[FEATURE] Support for a handoff-triggering custom payload

In the case of a fulfillment error, or a Dialogflow instance without a fulfillment server to do API calls, add handling (and documentation) for a custom payload for hand-off triggering.

Ideally, we should be able to include a buttonId as an optional arg, to override the default in settings of SF app

[Feature] Add asynchronous message support

Asynchronous messages will allow a bot to send a message proactively.

How is it different from the synchronous message

In the synchronous message, a bot will only answer to user's query. Eg if the user asks What is the customer care number? then the Bot will respond The customer care number is XXXX.
On the contrary, in asynchronous message, a bot will take the initiative to ask user what they what. Eg. A bot will proactively reach out to a new user and ask them How can I help u today?

Arrow function context

Hi!
I'm try to compile code, but there is error

Apps.Dialogflow/lib/Message.ts

Line 57 in 9e8f247

this.app.getLogger().error(Logs.EMPTY_BOT_USERNAME_SETTING);

I think about it forgotten when dividing into files in 1d8bcf5 commit

[FEATURE] Disable user input support

• NLP service should be able to send a payload that disables a users text-input with a message such as "please wait"

Example custom DF payload

{
  "disableInputMessage": "Please wait",
  "disableInput": true
}