gaoq1 / rasa_chatbot_cn

building a chinese dialogue system based on the newest version of rasa(基于最新版本rasa搭建的对话系统)

Rasa Core and Rasa NLU

rasa对话系统系列文章

• rasa对话系统踩坑记（一）
• rasa对话系统踩坑记（二）
• rasa对话系统踩坑记（三）
• rasa对话系统踩坑记（四）
• rasa对话系统踩坑记（五）
• rasa对话系统踩坑记（六）
• rasa对话系统踩坑记（七）
• rasa对话系统踩坑记（八）
• rasa对话系统踩坑记（九）
• rasa对话系统踩坑记（十）
• rasa-nlu的究极形态
• 闲聊模型的实践并应用到rasa

Introduction

rasa版本已经更新到了2.0版本，改动比较大，等2.0版本稳定后再跟进了。现在这里的代码还是去年上半年的版本，后面rasa做了很多改动，component已经支持bert，对中文的支持也更好。所以这个之前基于1.1.x的版本就转到1.1.x分支，目前master分支的话就分享最新的基于1.10.18的一套支持中文的pipeline

欢迎加入rasa微信闲聊群，微信请加：coffee199029

Running by command

install packages

• python >= 3.6

pip install -r requirements.txt

下载依赖package

train model

make train

run model

make run

test in cmdline

make shell

可以在命令行中测试

test by http server

http://localhost:5005/webhooks/rest/webhook post请求，请求参数例如：

{
    "sender": "0001",
    "message": "你好"
}

可以使用postman去请求调用

use rasa x

make run-x

rasa_chatbot_cn's Issues

Is the content of this demo changed to the medical field, or is the original rasa_chatbot code?

policy文件夹下的xx.yml没有加载

policy文件夹下的attention_policy.yml或embed_policy.yml或keras_policy.yml在bot.py里根本没有看到加载的代码，是否在这个demo里没有用到上述yml里的配置？

想问下这个多轮对话怎么能跟FAQ问答结合在一起使用呢？

FAQ问答一般是通过知识库的检索+匹配两阶段过程，跟意图识别+对话管理好像不是一个思路。如果把FAQ问答和多轮对话以及闲聊做三项集成，我考虑是不是可以这样，先用意图识别看是否是多轮对话story里面包含的意图，如果不在范围内，就直接走检索阶段，如果检索OK就通过FAQ进行匹配，如果检索不OK就走闲聊。期待您的回复。

ModuleNotFoundError: No module named 'bert_serving'

(.venv) ub16c9@ub16c9-gpu:~/ub16_prj/rasa_chatbot_cn$ python bot.py train-nlu-gao
Using TensorFlow backend.
2019-03-11 15:01:16 INFO rasa_nlu_gao.training_data.loading - Training data format of data/rasa_dataset_training.json is rasa_nlu
2019-03-11 15:01:16 INFO rasa_nlu_gao.training_data.training_data - Training data stats:
- intent examples: 1019 (13 distinct intents)
- Found intents: 'inform_package', 'deny', 'goodbye', 'inform_item', 'request_search', 'unknown_intent', 'inform_other_phone', 'greet', 'request_management', 'confirm', 'thanks', 'inform_time', 'inform_current_phone'
- entity examples: 883 (6 distinct entities)
- found entities: 'time', 'price', 'package', 'phone_number', 'data_plan', 'item'

Traceback (most recent call last):
File "bot.py", line 136, in
train_nlu_gao()
File "bot.py", line 112, in train_nlu_gao
trainer = Trainer(config.load("configs/config_embedding_bilstm.yml"))
File "/home/ub16c9/ub16_prj/rasa_chatbot_cn/.venv/lib/python3.6/site-packages/rasa_nlu_gao/model.py", line 152, in init
components.validate_requirements(cfg.component_names)
File "/home/ub16c9/ub16_prj/rasa_chatbot_cn/.venv/lib/python3.6/site-packages/rasa_nlu_gao/components.py", line 49, in validate_requirements
from rasa_nlu_gao import registry
File "/home/ub16c9/ub16_prj/rasa_chatbot_cn/.venv/lib/python3.6/site-packages/rasa_nlu_gao/registry.py", line 38, in
from rasa_nlu_gao.featurizers.bert_vectors_featurizer import BertVectorsFeaturizer # customize
File "/home/ub16c9/ub16_prj/rasa_chatbot_cn/.venv/lib/python3.6/site-packages/rasa_nlu_gao/featurizers/bert_vectors_featurizer.py", line 16, in
from bert_serving.client import BertClient
ModuleNotFoundError: No module named 'bert_serving'
(.venv) ub16c9@ub16c9-gpu:/ub16_prj/rasa_chatbot_cn$
(.venv) ub16c9@ub16c9-gpu:/ub16_prj/rasa_chatbot_cn$
(.venv) ub16c9@ub16c9-gpu:~/ub16_prj/rasa_chatbot_cn$

train.py: error: unrecognized arguments: --online --endpoints endpoints.yml

如果在启动server的时候报如下错误，
train.py: error: unrecognized arguments: --online --endpoints endpoints.yml

那么你可以试一试下面这个语句，当然models/dialogue_embed要换成你自己的目录。
python -m rasa_core.run --enable_api -d models/dialogue_embed -u models/nlu/default/current

Exception: Not all required packages are installed. Please install duckling

When i run “python bot.py train-nlu”, it occured as follows:

`
Using Theano backend.
2019-03-14 15:27:13 INFO rasa_nlu.training_data.loading - Training data format of data/rasa_dataset_training.json is rasa_nlu
2019-03-14 15:27:13 INFO rasa_nlu.training_data.training_data - Training data stats:
- intent examples: 1019 (13 distinct intents)
- Found intents: 'confirm', 'inform_time', 'deny', 'request_search', 'inform_current_phone', 'goodbye', 'thanks', 'inform_package', 'unknown_intent', 'request_management', 'greet', 'inform_other_phone', 'inform_item'
- entity examples: 883 (6 distinct entities)
- found entities: 'time', 'phone_number', 'price', 'package', 'item', 'data_plan'

Traceback (most recent call last):
File "bot.py", line 134, in
train_nlu()
File "bot.py", line 98, in train_nlu
trainer = Trainer(config.load("configs/nlu_embedding_config.yml"))
File "/home/my/miniconda3/envs/rasa_nlu_gao/lib/python3.6/site-packages/rasa_nlu/model.py", line 152, in init
components.validate_requirements(cfg.component_names)
File "/home/my/miniconda3/envs/rasa_nlu_gao/lib/python3.6/site-packages/rasa_nlu/components.py", line 63, in validate_requirements
"Please install {}".format(", ".join(failed_imports)))
Exception: Not all required packages are installed. To use this pipeline, you need to install the missing dependencies. Please install duckling
`
my environments:
python==3.6.8
tensorflow==1.9.0
theano==1.0.4
rasa-nlu==0.13.2
rasa-nlu-gao==0.2.1

Failed to execute custom action

While running the model, I experienced the following error:
2019-07-04 04:31:46 ERROR flask.app - Exception on /webhook [POST]
Traceback (most recent call last):
File "/home/oscar/venv/lib/python3.7/site-packages/flask/app.py", line 2311, in wsgi_app
response = self.full_dispatch_request()
File "/home/oscar/venv/lib/python3.7/site-packages/flask/app.py", line 1834, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/oscar/venv/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/home/oscar/venv/lib/python3.7/site-packages/flask/app.py", line 1737, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/oscar/venv/lib/python3.7/site-packages/flask/_compat.py", line 36, in reraise
raise value
File "/home/oscar/venv/lib/python3.7/site-packages/flask/app.py", line 1832, in full_dispatch_request
rv = self.dispatch_request()
File "/home/oscar/venv/lib/python3.7/site-packages/flask/app.py", line 1818, in dispatch_request
return self.view_functionsrule.endpoint
File "/home/oscar/venv/lib/python3.7/site-packages/flask_cors/decorator.py", line 128, in wrapped_function
resp = make_response(f(*args, **kwargs))
File "/home/oscar/venv/lib/python3.7/site-packages/rasa_sdk/endpoint.py", line 59, in webhook
response = executor.run(action_call)
File "/home/oscar/venv/lib/python3.7/site-packages/rasa_sdk/executor.py", line 237, in run
"No registered Action found for name '{}'.".format(action_name)
Exception: No registered Action found for name 'action_fallback_consume'.
127.0.0.1 - - [2019-07-04 04:31:46] "POST /webhook HTTP/1.1" 500 411 0.002637
2019-07-04 04:31:46 ERROR rasa.core.processor - Encountered an exception while running action 'action_fallback_consume'. Bot will continue, but the actions events are lost. Make sure to fix the exception in your custom code.
2019-07-04 04:31:46 DEBUG rasa.core.processor - Failed to execute custom action.
Traceback (most recent call last):
File "/home/oscar/venv/lib/python3.7/site-packages/rasa/core/actions/action.py", line 398, in run
json=json_body, method="post", timeout=DEFAULT_REQUEST_TIMEOUT
File "/home/oscar/venv/lib/python3.7/site-packages/rasa/utils/endpoints.py", line 142, in request
resp.status, resp.reason, await resp.content.read()
rasa.utils.endpoints.ClientResponseError: 500, INTERNAL SERVER ERROR, body='b'\n<title>500 Internal Server Error</title>\n

Internal Server Error

\n

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

\n''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/oscar/venv/lib/python3.7/site-packages/rasa/core/processor.py", line 439, in _run_action
events = await action.run(output_channel, nlg, tracker, self.domain)
File "/home/oscar/venv/lib/python3.7/site-packages/rasa/core/actions/action.py", line 420, in run
raise Exception("Failed to execute custom action.") from e
Exception: Failed to execute custom action.

My stories.md file is:

greet

• greet
  • utter_greet

goodbye

• goodbye
  • utter_goodbye

greetnbye

• greet
  • utter_greet
• goodbye
  • utter_goodbye

unknown

• unknown_intent
  • action_fallback_consume

meaning1

• greet
  • utter_greet
• request_meaning_of{"service": "Dedicated Servers"}
  • slot{"service": "Dedicated Servers"}
  • action_search_meaning
• goodbye
  • utter_goodbye

meaning2

• greet
  • utter_greet
• request_meaning_of{"service": "cPanel Dedicated Servers"}
  • slot{"service": "cPanel Dedicated Servers"}
  • action_search_meaning
• request_meaning_of{"service": "Microsoft Window Software"}
  • slot{"service": "Microsoft Window Software"}
  • action_search_meaning
• request_meaning_of{"service": "Network Firewall"}
  • slot{"service": "Network Firewall"}
  • action_search_meaning
• goodbye
  • utter_goodbye

my config file:

• name: FallbackPolicy
  fallback_action_name: 'action_fallback_consume'
  nlu_threshold: 0.7
  core_threshold: 0.5

my domain file:
actions:

• utter_greet
• utter_goodbye
• utter_advantage_action_one
• utter_advantage_action_two
• utter_how_action_three
• utter_ask_server_CPU
• utter_ask_morehelp
• utter_confirm
• action_search_meaning
• action_fallback_consume
• action_search_advantage
• action_search_price
• action_search_specific

my actions file:
rom future import absolute_import
from future import division
from future import print_function
from future import unicode_literals

from rasa_core_sdk import Action
from rasa_core_sdk.events import SlotSet

support_search = ["Dedicated Servers", "UNIX Dedicated Servers", "Windows Dedicated Servers"]

def extract_service(service):
if service is None:
return None
for name in support_search:
if name in service:
return name
return None

class ActionSearchMeaning(Action):
def name(self):
return 'action_search_meaning'

def run(self, dispatcher, tracker, domain):
    service = tracker.get_slot("service")
    service = extract_service(service)
    if service is None:
        dispatcher.utter_message("'service' is None")
        return []

    dispatcher.utter_message("okay, please wait.")
    return []

class ActionSearchAdvantage(Action):
def name(self):
return 'action_search_advantage'

def run(self, dispatcher, tracker, domain):
    service = tracker.get_slot("service")
    service = extract_service(service)
    if service is None:
        dispatcher.utter_message("'service' is None")
        return []
    return[]

class ActionSearchPrice(Action):
def name(self):
return 'action_search_price'

def run(self, dispatcher, tracker, domain):
    service = tracker.get_slot("service")
    service = extract_service(service)
    if service is None:
        dispatcher.utter_message("'service' is None")
        return []
    return[]

class ActionFallbackConsume(Action):
"""Executes the fallback action and goes back to the previous state
of the dialogue"""

def name(self):
    return 'action_fallback_consume'

def run(self, dispatcher, tracker, domain):
    from rasa_core.events import UserUtteranceReverted

    text = tracker.latest_message.text

    key = '***'
    api = 'https://www.cleverbot.com/getreply?input={}&key={}'.format(text, key)
    message = requests.get(api).json()

    dispatcher.utter_message("{}".format(message['output']))

    return [UserUtteranceReverted()]

I have not start writing this file but defined actions.

Can you please help me with this error>
Thanks.

请问训练好的模型可以直接当成rasa_nlu模型使用吗？

博主有人一直看，并感谢你的分享

看到博主在踩坑文章6开头的一段话，想告诉博主，其实是有人在看的，只是没有简书的账号，无法表发感谢而已。我浏览了rasa相关的很多实践项目，博主的项目和博客是我看到最具有阅读价值的，我已经把你关于rasa的项目分享给了我的同事。
最后，再对博主的经验分享表示感谢。

您好，能给出requirements.txt么？谢谢！

kidx: command not found?

when I run:
make run-cmdline
/bin/sh: kidx: command not found make: kidx: Command not found make: *** [run-cmdline] Error 127

python bot.py train-nlu-gao失败，请问如何处理？

TimeoutError: no response from the server (with "timeout"=10000 ms), please check the following:is the server still online? is the network broken? are "port" and "port_out" correct? ar
e you encoding a huge amount of data whereas the timeout is too small for that?

其中，已经执行 bert-serving-start -model_dir /tmp/chinese_L-12_H-768_A-12/ -num_worker=4

日志：

             ARG   VALUE

---

       ckpt_name = bert_model.ckpt
     config_name = bert_config.json
            cors = *
             cpu = False
      device_map = []

fixed_embed_length = False
fp16 = False
gpu_memory_fraction = 0.5
graph_tmp_dir = None
http_max_connect = 10
http_port = None
mask_cls_sep = False
max_batch_size = 256
max_seq_len = 25
model_dir = E:\Environment\python\chinese_L-12_H-768_A-12
num_worker = 4
pooling_layer = [-2]
pooling_strategy = REDUCE_MEAN
port = 5555
port_out = 5556
prefetch_size = 10
priority_batch_size = 16
show_tokens_to_client = False
tuned_model_dir = None
verbose = False
xla = False

I:?[35mVENTILATOR?[0m:freeze, optimize and export graph, could take a while...
I:?[36mGRAPHOPT?[0m:model config: E:\Environment\python\chinese_L-12_H-768_A-12\bert_config.json
I:?[36mGRAPHOPT?[0m:checkpoint: E:\Environment\python\chinese_L-12_H-768_A-12\bert_model.ckpt
I:?[36mGRAPHOPT?[0m:build graph...
I:?[36mGRAPHOPT?[0m:load parameters from checkpoint...
I:?[36mGRAPHOPT?[0m:optimize...
I:?[36mGRAPHOPT?[0m:freeze...
I:?[36mGRAPHOPT?[0m:write graph to a tmp file: C:\Users\Jiangdg\AppData\Local\Temp\tmpc8m5mmy8
I:?[35mVENTILATOR?[0m:optimized graph is stored at: C:\Users\Jiangdg\AppData\Local\Temp\tmpc8m5mmy8
I:?[35mVENTILATOR?[0m:bind all sockets
I:?[35mVENTILATOR?[0m:open 8 ventilator-worker sockets
I:?[35mVENTILATOR?[0m:start the sink
I:?[32mSINK?[0m:ready
I:?[35mVENTILATOR?[0m:get devices
W:?[35mVENTILATOR?[0m:only 1 out of 1 GPU(s) is available/free, but "-num_worker=4"
W:?[35mVENTILATOR?[0m:multiple workers will be allocated to one GPU, may not scale well and may raise out-of-memory
I:?[35mVENTILATOR?[0m:device map:
worker 0 -> gpu 0
worker 1 -> gpu 0
worker 2 -> gpu 0
worker 3 -> gpu 0
I:?[33mWORKER-1?[0m:use device gpu: 0, load graph from C:\Users\Jiangdg\AppData\Local\Temp\tmpc8m5mmy8
I:?[33mWORKER-3?[0m:use device gpu: 0, load graph from C:\Users\Jiangdg\AppData\Local\Temp\tmpc8m5mmy8
I:?[33mWORKER-2?[0m:use device gpu: 0, load graph from C:\Users\Jiangdg\AppData\Local\Temp\tmpc8m5mmy8
I:?[33mWORKER-0?[0m:use device gpu: 0, load graph from C:\Users\Jiangdg\AppData\Local\Temp\tmpc8m5mmy8

您好 我在使用make train 后出现报错 ，报错如下。还望解答。（已解决）

make train
rasa train --domain domain.yml --data data --config config.yml --out models
2019-07-12 15:41:01 INFO rasa.model - Data (core-config) for Core model changed.
2019-07-12 15:41:01 INFO rasa.model - Data (nlu-config) for NLU model changed.
Training Core model...
2019-07-12 15:41:01 INFO root - Generating grammar tables from /usr/lib/python3.6/lib2to3/Grammar.txt
2019-07-12 15:41:01 INFO root - Generating grammar tables from /usr/lib/python3.6/lib2to3/PatternGrammar
.txt
Using TensorFlow backend.
Processed Story Blocks: 100%|████████████████████████████████| 10/10 [00:00<00:00, 1451.12it/s, # trackers=1]
Processed Story Blocks: 100%|█████████████████████████████████| 10/10 [00:00<00:00, 291.18it/s, # trackers=7]
Processed Story Blocks: 100%|████████████████████████████████| 10/10 [00:00<00:00, 106.21it/s, # trackers=19]
Processed Story Blocks: 100%|████████████████████████████████| 10/10 [00:00<00:00, 105.48it/s, # trackers=16]
Processed trackers: 100%|███████████████████████████████████| 491/491 [00:05<00:00, 96.73it/s, # actions=164]

---

Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) (None, 3, 49) 0

---

attention_1 (Attention) (None, 3, 1024) 150528 input_1[0][0]
input_1[0][0]
input_1[0][0]

---

attention_2 (Attention) (None, 3, 1024) 3145728 attention_1[0][0]
attention_1[0][0]
attention_1[0][0]

---

global_average_pooling1d_1 (Glo (None, 1024) 0 attention_2[0][0]

---

dropout_1 (Dropout) (None, 1024) 0 global_average_pooling1d_1[0][0]

---

dense_1 (Dense) (None, 19) 19475 dropout_1[0][0]

Total params: 3,315,731
Trainable params: 3,315,731
Non-trainable params: 0

---

2019-07-12 15:41:09 INFO rasa.core.policies.keras_policy - Fitting model with 164 total samples and a va
lidation split of 0.1
Epoch 1/100
164/164 [==============================] - 1s 8ms/step - loss: 2.4029 - acc: 0.3841
Epoch 2/100
164/164 [==============================] - 1s 4ms/step - loss: 1.4817 - acc: 0.6159
Epoch 3/100
164/164 [==============================] - 1s 4ms/step - loss: 1.0786 - acc: 0.6341
Epoch 4/100
164/164 [==============================] - 1s 3ms/step - loss: 0.7755 - acc: 0.7683
Epoch 5/100
164/164 [==============================] - 1s 4ms/step - loss: 0.5705 - acc: 0.8110
Epoch 6/100
164/164 [==============================] - 1s 4ms/step - loss: 0.4409 - acc: 0.8598
Epoch 7/100
164/164 [==============================] - 1s 3ms/step - loss: 0.3448 - acc: 0.9451
Epoch 8/100
164/164 [==============================] - 1s 3ms/step - loss: 0.2685 - acc: 0.9634
Epoch 9/100
164/164 [==============================] - 1s 3ms/step - loss: 0.1996 - acc: 0.9817
Epoch 10/100
164/164 [==============================] - 1s 4ms/step - loss: 0.1648 - acc: 0.9695
Epoch 11/100
164/164 [==============================] - 1s 4ms/step - loss: 0.1443 - acc: 0.9939
Epoch 12/100
164/164 [==============================] - 1s 3ms/step - loss: 0.1191 - acc: 1.0000
Epoch 13/100
164/164 [==============================] - 1s 3ms/step - loss: 0.1039 - acc: 0.9939
Epoch 14/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0801 - acc: 0.9939
Epoch 15/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0704 - acc: 1.0000
Epoch 16/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0547 - acc: 1.0000
Epoch 17/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0475 - acc: 1.0000
Epoch 18/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0338 - acc: 1.0000
Epoch 19/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0292 - acc: 1.0000
Epoch 20/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0262 - acc: 1.0000
Epoch 21/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0235 - acc: 1.0000
Epoch 22/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0206 - acc: 1.0000
Epoch 23/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0205 - acc: 1.0000
Epoch 24/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0179 - acc: 1.0000
Epoch 25/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0185 - acc: 1.0000
Epoch 26/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0148 - acc: 1.0000
Epoch 27/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0135 - acc: 1.0000
Epoch 28/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0124 - acc: 1.0000
Epoch 29/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0121 - acc: 1.0000
Epoch 30/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0094 - acc: 1.0000
Epoch 31/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0093 - acc: 1.0000
Epoch 32/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0100 - acc: 1.0000
Epoch 33/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0065 - acc: 1.0000
Epoch 34/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0072 - acc: 1.0000
Epoch 35/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0067 - acc: 1.0000
Epoch 36/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0061 - acc: 1.0000
Epoch 37/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0058 - acc: 1.0000
Epoch 38/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0053 - acc: 1.0000
Epoch 39/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0064 - acc: 1.0000
Epoch 40/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0059 - acc: 1.0000
Epoch 41/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0050 - acc: 1.0000
Epoch 42/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0044 - acc: 1.0000
Epoch 43/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0043 - acc: 1.0000
Epoch 44/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0045 - acc: 1.0000
Epoch 45/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0041 - acc: 1.0000
Epoch 46/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0044 - acc: 1.0000
Epoch 47/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0035 - acc: 1.0000
Epoch 48/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0032 - acc: 1.0000
Epoch 49/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0032 - acc: 1.0000
Epoch 50/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0032 - acc: 1.0000
Epoch 51/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0032 - acc: 1.0000
Epoch 52/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0029 - acc: 1.0000
Epoch 53/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0027 - acc: 1.0000
Epoch 54/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0026 - acc: 1.0000
Epoch 55/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0022 - acc: 1.0000
Epoch 56/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0023 - acc: 1.0000
Epoch 57/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0025 - acc: 1.0000
Epoch 58/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0021 - acc: 1.0000
Epoch 59/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0023 - acc: 1.0000
Epoch 60/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0018 - acc: 1.0000
Epoch 61/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0017 - acc: 1.0000
Epoch 62/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0016 - acc: 1.0000
Epoch 63/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0021 - acc: 1.0000
Epoch 64/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0016 - acc: 1.0000
Epoch 65/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0020 - acc: 1.0000
Epoch 66/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0018 - acc: 1.0000
Epoch 67/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0018 - acc: 1.0000
Epoch 68/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0014 - acc: 1.0000
Epoch 69/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0015 - acc: 1.0000
Epoch 70/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0014 - acc: 1.0000
Epoch 71/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0016 - acc: 1.0000
Epoch 72/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0012 - acc: 1.0000
Epoch 73/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0014 - acc: 1.0000
Epoch 74/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0011 - acc: 1.0000
Epoch 75/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0013 - acc: 1.0000
Epoch 76/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0014 - acc: 1.0000
Epoch 77/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0013 - acc: 1.0000
Epoch 78/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0014 - acc: 1.0000
Epoch 79/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0011 - acc: 1.0000
Epoch 80/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0010 - acc: 1.0000
Epoch 81/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0012 - acc: 1.0000
Epoch 82/100
164/164 [==============================] - 1s 3ms/step - loss: 0.0010 - acc: 1.0000
Epoch 83/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0010 - acc: 1.0000
Epoch 84/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0012 - acc: 1.0000
Epoch 85/100
164/164 [==============================] - 1s 4ms/step - loss: 0.0011 - acc: 1.0000
Epoch 86/100
164/164 [==============================] - 1s 4ms/step - loss: 8.4983e-04 - acc: 1.0000
Epoch 87/100
164/164 [==============================] - 1s 4ms/step - loss: 8.9001e-04 - acc: 1.0000
Epoch 88/100
164/164 [==============================] - 1s 4ms/step - loss: 9.1472e-04 - acc: 1.0000
Epoch 89/100
164/164 [==============================] - 1s 4ms/step - loss: 8.9015e-04 - acc: 1.0000
Epoch 90/100
164/164 [==============================] - 1s 4ms/step - loss: 8.5846e-04 - acc: 1.0000
Epoch 91/100
164/164 [==============================] - 1s 4ms/step - loss: 8.0720e-04 - acc: 1.0000
Epoch 92/100
164/164 [==============================] - 1s 4ms/step - loss: 7.1269e-04 - acc: 1.0000
Epoch 93/100
164/164 [==============================] - 1s 4ms/step - loss: 9.6782e-04 - acc: 1.0000
Epoch 94/100
164/164 [==============================] - 1s 4ms/step - loss: 8.5787e-04 - acc: 1.0000
Epoch 95/100
164/164 [==============================] - 1s 4ms/step - loss: 7.9524e-04 - acc: 1.0000
Epoch 96/100
164/164 [==============================] - 1s 4ms/step - loss: 7.3859e-04 - acc: 1.0000
Epoch 97/100
164/164 [==============================] - 1s 4ms/step - loss: 7.8808e-04 - acc: 1.0000
Epoch 98/100
164/164 [==============================] - 1s 4ms/step - loss: 6.7807e-04 - acc: 1.0000
Epoch 99/100
164/164 [==============================] - 1s 4ms/step - loss: 6.8284e-04 - acc: 1.0000
Epoch 100/100
164/164 [==============================] - 1s 4ms/step - loss: 6.6815e-04 - acc: 1.0000
2019-07-12 15:42:09 INFO rasa.core.policies.keras_policy - Done fitting keras policy model
Processed trackers: 100%|████████████████████████████████████| 10/10 [00:00<00:00, 1067.77it/s, # actions=50]
Processed actions: 50it [00:00, 6449.01it/s, # examples=50]
2019-07-12 15:42:10 INFO rasa.core.agent - Persisted model to '/tmp/tmpfua68ef0/core'
Core model training completed.
Training NLU model...
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 206, in arg_wr
apper
return func(self, *args, **kwargs)
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 232, in server
_status
return jsonapi.loads(self._recv(req_id).content[1])
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 164, in _recv
raise e
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 153, in _recv
response = self.receiver.recv_multipart()
File "/home/ubuntu/.local/lib/python3.6/site-packages/zmq/sugar/socket.py", line 470, in recv_multipart
parts = [self.recv(flags, copy=copy, track=track)]
File "zmq/backend/cython/socket.pyx", line 796, in zmq.backend.cython.socket.Socket.recv
File "zmq/backend/cython/socket.pyx", line 832, in zmq.backend.cython.socket.Socket.recv
File "zmq/backend/cython/socket.pyx", line 191, in zmq.backend.cython.socket._recv_copy
File "zmq/backend/cython/socket.pyx", line 186, in zmq.backend.cython.socket._recv_copy
File "zmq/backend/cython/checkrc.pxd", line 19, in zmq.backend.cython.checkrc._check_rc
zmq.error.Again: Resource temporarily unavailable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/ubuntu/.local/bin/rasa", line 11, in
sys.exit(main())
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/main.py", line 76, in main
cmdline_arguments.func(cmdline_arguments)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/cli/train.py", line 84, in train
kwargs=extract_additional_arguments(args),
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/train.py", line 42, in train
kwargs=kwargs,
File "uvloop/loop.pyx", line 1451, in uvloop.loop.Loop.run_until_complete
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/train.py", line 100, in train_async
kwargs,
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/train.py", line 203, in _train_async_internal
kwargs=kwargs,
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/train.py", line 256, in _do_training
fixed_model_name=fixed_model_name,
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/train.py", line 463, in _train_nlu_with_validate
d_data
config, nlu_data_directory, _train_path, fixed_model_name="nlu"
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/nlu/train.py", line 81, in train
trainer = Trainer(nlu_config, component_builder)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/nlu/model.py", line 151, in init
self.pipeline = self._build_pipeline(cfg, component_builder)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/nlu/model.py", line 163, in build_pipeline
component = component_builder.create_component(component_cfg, cfg)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/nlu/components.py", line 459, in create_componen
t
component = registry.create_component_by_config(component_config, cfg)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/nlu/registry.py", line 196, in create_component
by_config
return component_class.create(component_config, config)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa/nlu/components.py", line 244, in create
return cls(component_config)
File "/home/ubuntu/.local/lib/python3.6/site-packages/rasa_nlu_gao/featurizers/bert_vectors_featurizer.py",
line 61, in init
identity=identity
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 451, in __init
__
self.available_bc = [BertClient(**kwargs) for _ in range(max_concurrency)]
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 451, in
self.available_bc = [BertClient(**kwargs) for _ in range(max_concurrency)]
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 108, in __init
__
s_status = self.server_status
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/init.py", line 215, in arg_wr
apper
_raise(t_e, _e)
File "/home/ubuntu/.local/lib/python3.6/site-packages/bert_serving/client/_py3_var.py", line 9, in _raise
raise t_e from _e
TimeoutError: no response from the server (with "timeout"=10000 ms), please check the following:is the server
still online? is the network broken? are "port" and "port_out" correct? are you encoding a huge amount of da
ta whereas the timeout is too small for that?
Makefile:2: recipe for target 'train' failed
make: *** [train] Error 1

用bert yml配置出错了

你好，我用了
language: "zh"

pipeline:

• name: "JiebaTokenizer"
• name: "rasa_nlu_gao.featurizers.bert_vectors_featurizer.BertVectorsFeaturizer"
  ip: '127.0.0.1'
  port: 5555
  port_out: 5556
  show_server_config: True
  timeout: 10000
• name: "rasa_nlu_gao.classifiers.embedding_bert_intent_estimator_classifier.EmbeddingBertIntentEstimatorClassifier"
• name: "SpacyNLP"
• name: "CRFEntityExtractor"

出错文件是：在网上查了但没有找到可用答案
File "/home/oscar/venv/lib/python3.7/site-packages/rasa/nlu/registry.py", line 196, in create_component_by_config
return component_class.create(component_config, config)
TypeError: create() takes 2 positional arguments but 3 were given

意图识别并没有结合命名实体识别的特征，只有结巴特征，我认为这个对意图识别有影响

我发现一个问题，意图识别的特征向量的分词没有考虑命名实体识别而是直接拿的结巴分词的特征，比方说，用户语料里面标的实体是“中华人民共和国”，但是结巴分词不能对其做整体命名识别，只能识别“中华”，“人民”，“共和国”，这个我认为对识别意图是有影响的，你觉得呢？

运行Train时出错

你好，我在运行train训练模型中报错，错误如下
2019-07-02 21:30:00 INFO rasa.model - Data (core-config) for Core model changed.
2019-07-02 21:30:00 INFO rasa.model - Data (nlu-config) for NLU model changed.
Training Core model...
Traceback (most recent call last):
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\core\registry.py", line 20, in policy_from_module_path
return class_from_module_path(module_path, lookup_path="rasa.core.policies")
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\utils\common.py", line 188, in class_from_module_path
m = importlib.import_module(module_name)
File "c:\users\atian\appdata\local\programs\python\python37\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 953, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'policy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\core\policies\ensemble.py", line 279, in from_dict
constr_func = registry.policy_from_module_path(policy_name)
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\core\registry.py", line 22, in policy_from_module_path
raise ImportError("Cannot retrieve policy from path '{}'".format(module_path))
ImportError: Cannot retrieve policy from path 'policy.attention_policy.AttentionPolicy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\atian\appdata\local\programs\python\python37\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\users\atian\appdata\local\programs\python\python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\atian\AppData\Local\Programs\Python\Python37\Scripts\rasa.exe_main.py", line 9, in
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa_main.py", line 70, in main
cmdline_arguments.func(cmdline_arguments)
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\cli\train.py", line 84, in train
kwargs=extract_additional_arguments(args),
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\train.py", line 40, in train
kwargs=kwargs,
File "c:\users\atian\appdata\local\programs\python\python37\lib\asyncio\base_events.py", line 584, in run_until_complete
return future.result()
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\train.py", line 135, in train_async
kwargs=kwargs,
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\train.py", line 174, in _do_training
kwargs=kwargs,
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\train.py", line 303, in _train_core_with_validated_data
kwargs=kwargs,
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\core\train.py", line 36, in train
policies = config.load(policy_config)
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\core\config.py", line 24, in load
return PolicyEnsemble.from_dict(config_data)
File "c:\users\atian\appdata\local\programs\python\python37\lib\site-packages\rasa\core\policies\ensemble.py", line 287, in from_dict
"".format(policy_name)
rasa.core.policies.ensemble.InvalidPolicyConfig: Module for policy 'policy.attention_policy.AttentionPolicy' could not be loaded. Please make sure the name is a valid policy.

应该是Policy加载失败，请问有遇到过么？

你好，我在训练rasa-nlu的时候出现了错误，想请教一下

我自己写了一个component，然后使用这个component进行rasa-nlu的训练，然后出现了如下的错误：

因为刚接触，所以对这个框架一知半解，希望能够得到您的解答，谢谢~

我在component代码里，只实现了__init__,process和load三个函数，里面并没有train，create等一些函数，是不是这个原因？有哪些函数是创建component必需的？？

Failed to register package 'actions.actions'

Error:

action_server_1 | 2019-04-19 09:31:35 INFO main - Starting action endpoint server...
action_server_1 | 2019-04-19 09:31:35 ERROR rasa_core_sdk.executor - Failed to register package 'actions.actions'.
action_server_1 | Traceback (most recent call last):
action_server_1 | File "/app/rasa_core_sdk/executor.py", line 150, in register_package
action_server_1 | self._import_submodules(package)
action_server_1 | File "/app/rasa_core_sdk/executor.py", line 137, in _import_submodules
action_server_1 | package = importlib.import_module(package)
action_server_1 | File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
action_server_1 | return _bootstrap._gcd_import(name[level:], package, level)
action_server_1 | File "", line 994, in _gcd_import
action_server_1 | File "", line 971, in _find_and_load
action_server_1 | File "", line 953, in _find_and_load_unlocked
action_server_1 | ModuleNotFoundError: No module named 'actions.actions'
action_server_1 | 2019-04-19 09:31:35 INFO main - Action endpoint is up and running. on ('0.0.0.0', 5055)
duckling_1 | no port specified, defaulting to port 8000
duckling_1 | Listening on http://0.0.0.0:8000
rasa_nlu_1 | Traceback (most recent call last):
rasa_nlu_1 | File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
rasa_nlu_1 | "main", mod_spec)
rasa_nlu_1 | File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
rasa_nlu_1 | exec(code, run_globals)
rasa_nlu_1 | File "/app/rasa/core/run.py", line 204, in
rasa_nlu_1 | "Calling rasa.core.run directly is "
rasa_nlu_1 | RuntimeError: Calling rasa.core.run directly is no longer supported. Please use rasa shell instead.
chatwithme_rasa_nlu_1 exited with code 1

Docker-Compose file:

version: '3.0'

services:
  rasa_core:
    image: rasa/rasa_core:latest
    networks: ['rasa-network']
    ports:
    - "5005:5005"
    volumes:
    - "./rasa-app-data/models/current/dialogue:/app/model"
    - "./rasa-app-data/config:/app/config"
    - "./rasa-app-data/project:/app/project"
    command:
    - start
    - -d
    - ./model
    - -c
    - rest
    - -u
    - current/nlu
    - --endpoints
    - config/endpoints.yml

  action_server:
    image: rasa/rasa_core_sdk:latest
    networks: ['rasa-network']
    ports:
    - "5055:5055"
    volumes:
    - "./rasa-app-data/actions:/app/actions"

  rasa_nlu:
    image: rasa/rasa_nlu:latest-full
    networks: ['rasa-network']
    ports:
    - "5000:5000"
    volumes:
    - "./rasa-app-data/models/:/app/projects"
    - "./rasa-app-data/logs:/app/logs"

  duckling:
    image: rasa/duckling:latest
    networks: ['rasa-network']
    ports:
    - "8000:8000"

networks: {rasa-network: {}}

默认pip install -r requirements.txt报错，能否升级下

我使用conda新建了一个干净的环境，python=3.6.8
然后，pip install -r requirements.txt，报错：

ERROR: tensorflow 1.10.0 has requirement numpy<=1.14.5,>=1.13.3, but you'll have numpy 1.16.3 which is incompatible. ERROR: tensorflow 1.10.0 has requirement setuptools<=39.1.0, but you'll have setuptools 41.0.1 which is incompatible. ERROR: rasa-core 0.12.3 has requirement jsonschema~=2.6, but you'll have jsonschema 3.0.1 which is incompatible. ERROR: rasa-core 0.12.3 has requirement packaging~=17.0, but you'll have packaging 19.0 which is incompatible. ERROR: rasa-core 0.12.3 has requirement pyyaml~=3.12, but you'll have pyyaml 5.1 which is incompatible.

看了下，这些包都是pip关联安装的，版本已经不对了，
1、能否麻烦共享下您的python版本，还有全部的requirements.txt
2、不知能否升级下rasa_nlu，新版本的nlu变化很大了
谢谢！

训练是否存在问题

{
"intent": {
"name": null,
"confidence": 1
},
"entities": [],
"text": "你好",
"project": "default",
"model": "fallback"
}

Unknown layer:name

试了tensorflow 1.9，1.10，1.12 py版本也试了3.6.6 怎么都是这个错

如何贡献代码

前辈您好，首先非常感谢您实践了这么一套系统，并为后来人指明了方向。提这个 issue 的目的是想了解如何给该项目贡献，哪些部分是您想优化而一直没有优化的，以及工作流上您是怎么安排的。

小弟不才，也在您的基础上做了部分简化和修改，以方便项目组内的同事使用。修改过程中也发现，实际的代码有部分冗余，也有些配置繁琐的地方；萌生了给代码做贡献的想法。看您的历史提交，似乎只有您一个人在提交。但在文章中，有看到你在号召一起贡献代码。因而产生了矛盾点，希望前辈您能大概表达一下您对这个项目的安排，以及后面规划的功能点。类似 TODO 这种形式，以方便其它人跟进贡献。

最后，再次感谢楼主的贡献~

从《SeqGAN训练生成小黄鸡语料》而来，但是没有找到项目代码

hello，从《SeqGAN训练生成小黄鸡语料》而来，但是没有找到项目代码，请问您方便提供一下吗？非常感谢！

测试nlu服务时，向127.0.0.1发送post请求返回了页面的html元素

nlu服务已成功启动，且5000端口已有信息显示，但我向localhost发送curl -XPOST 127.0.0.1:5000/parse -d"{"q":"帮我查一下上个月流量"}" 请求时，返回了页面html元素

• 下面是返回的一部分页面元素信息：

221            request_params = simplejson.loads(

222                    request.content.read().decode('u tf-8', 'strict'))

223        print(request_params)

builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 3: invalid start byte

• 这里是出错信息：

2019-05-24 10:35:27+0800 [-] Unhandled Error
Traceback (most recent call last):
File "D:\Users\cjh\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_nlu_gao\server.py", line 135, in decorat
ed
return f(*args, **kwargs)
File "D:\Users\cjh\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_nlu_gao\server.py", line 118, in decorat
ed
return f(*args, **kwargs)
File "D:\Users\cjh\AppData\Local\Programs\Python\Python36\lib\site-packages\twisted\internet\defer.py", line 1613, in unw
indGenerator
return _cancellableInlineCallbacks(gen)
File "D:\Users\cjh\AppData\Local\Programs\Python\Python36\lib\site-packages\twisted\internet\defer.py", line 1529, in _ca
ncellableInlineCallbacks
_inlineCallbacks(None, g, status)
--- ---
File "D:\Users\cjh\AppData\Local\Programs\Python\Python36\lib\site-packages\twisted\internet\defer.py", line 1418, in _in
lineCallbacks
result = g.send(result)
File "D:\Users\cjh\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_nlu_gao\server.py", line 222, in parse
request.content.read().decode('utf-8', 'strict'))
builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 3: invalid start byte

2019-05-24 10:35:27+0800 [-] "127.0.0.1" - - [24/May/2019:02:35:26 +0000] "POST /parse HTTP/1.1" 500 3202 "-" "curl/7.55.1"

求问，是我curl发送的post请求有问题吗？已被此问题纠结很久了，求大佬帮忙看看，十分感谢

支持正则提取实体和查表提取实体吗？

rasa_nlu支持正则提取实体和查表提取实体吗？

train.py: error: unrecognized arguments: --online --endpoints endpoints.yml

when train dialogue in online mod, it traceback:
train.py: error: unrecognized arguments: --online --endpoints endpoints.yml

what's the relationship with 'rasa_nlu_gq' project

rasa.nlu.extractors.crf_entity_extractor WARNING

非常感谢您的贡献！
我尝试将您的系统迁移到我自己的项目中，具体步骤如下:
在nlu.md中添加了训练数据，其中有一条如下:
- 麻烦帮我发布条[通告](chat)说[国庆节](date)[下午2点](time)[张三的家长](name)来学校[教室](address)参加[毕业典礼](acticities)
但是当我执行
rasa train nlu -u data/nlu -c config.yml --out models/nlu
出现了下面的警告信息:

我的配置文件如下:

language: "zh"

pipeline:
- name: "JiebaTokenizer"
- name: "CRFEntityExtractor"

- name: "rasa_nlu_gao.extractors.jieba_pseg_extractor.JiebaPsegExtractor"
  part_of_speech: ["nr"]

- name: "rasa_nlu_gao.featurizers.bert_vectors_featurizer.BertVectorsFeaturizer"
  ip: '127.0.0.1'
  port: 8001
  port_out: 8002
  show_server_config: True
  timeout: 10000
  check_version: False
- name: "rasa_nlu_gao.classifiers.embedding_bert_intent_classifier.EmbeddingBertIntentClassifier"

看到您主页上写到 find-a-part-time job，欢迎联系我👏，可远程，待遇丰厚！
我的邮箱是 [email protected]
期待您的解答与合作！！

自己训练的效果不是很理想呢

用 gao 模型 和 keras 对话训练

模型改动到支持英文版本

谢谢分享。如果想支持英文版本是不是要替换

JiebaTokenizer
rasa_nlu_gao.extractors.jieba_pseg_extractor.JiebaPsegExtractor
bert模型

还有别的(bertFeaturizer和bertIntentClassifier) 要替换的吗？rasa官网上有WhitespaceTokenizer， 有没有其他推荐的tokenizer和extractor？

还有集成第三方闲聊接口
train_dialogue在什么地方?
ActionDefaultFallback新版本好像改动了? 怎么加入api的调用？

原谅新手。 如果我有问得不清楚的地方请指出。

谢谢啦

你好，按照你给的配置，训练效果不理想啊

language: "zh"

pipeline:

• name: "tokenizer_jieba"
• name: "ner_crf"
• name: "jieba_pseg_extractor"
  part_of_speech: ["nr", "ns", "nt", "nz"]
• name: "intent_featurizer_count_vectors"
  OOV_token: oov
  token_pattern: '(?u)\b\w+\b'
• name: "intent_classifier_tensorflow_embedding"
  ，用的是这个配置，
  然后数据格式：
  {"entities":[{"start":0,"end":7,"value":"数学","entity":"publiser"}],"text":"数学","intent":"search.publisher"},
  {"entities":[{"start":0,"end":7,"value":"大学","entity":"publiser"}],"text":"大学","intent":"search.publisher"},
  像这样的组织数据有上万条，训练完成后，比如输入大学，返回的结果意图是空，实体也为空，求解啊

Attention 类的实现问题

最近入坑了，想看看先进们是如何实现不同功能的，所以钻进代码里看 AttentionPolicy 是如何实现的。首先要谢谢你的分享。

看 Position_Embedding 的时候觉得没有什么问题，但看 Attention 的时候总觉得不太对劲，可能只是我对代码或 Keras 认知不足，特来请教。

看不懂的代码是这句：

    A = K.batch_dot(Q_seq, K_seq, axes=[3, 3]) / self.size_per_head**0.5

问题出于，在论文中这个步骤大概想做一个叫 scaled dot product 的动作，把两个向量 Q 和 K 做个简单内积，然后把结果做过简单缩放。但好像 keras 的 batch_dot 在 dimension 超过 2 的时候，做的不是内积，而是矩阵积。感觉上这会使结果大了很多。

后面有另一个 batch_dot 的调用，也当然看不很懂。

希望有什么提示可以指点迷津，万分感激。

该系统能否实现差异化回答

hi，请教您一下。是聊天机器人差异化回答的问题。需求：针对某个特定的场景，根据不同的输入，机器人输出对应的差异化、个性化的内容。比如，保险行业用户问询。用户说，我想为爸爸买保险，或者是我想为妈妈买保险，。机器人能够针对的回答出，请问您的母亲或者父亲的年龄是多少这样。您提供的源码可以实现么？

请问nlu训练样本中，unknown_intent的作用是？

最后如何开始进行对答呢？

执行完最后启动了一个 5000端口的服务器 ，访问了一下 ，返回了‘ hello from Rasa NLU: 0.13.4 ’
恩恩 然后怎么开始对答呢？

启动完了，能做些啥？看着命令台发呆？

这个没有python的接口全部是命令行执行的，能开放一下python的接口吗？

官方的rasa指导提供过demo代码，但是版本更新后，很多接口都变了，比如agent.train()
@GaoQ1 rasa大王gao 是否能提供一下这方面的参考，多谢

Unable to initialize persistor

rasa-nlu == 0.13.4
rasa-nlu-gao == 0.2.0
rasa-core == 0.12.3
rasa-core-sdk == 0.12.1

python bot.py train-nlu-gao
python -m rasa_nlu_gao.server -c configs/config_embedding_bilstm.yml --path models/nlu_gao/

然后用 postman 请求 localhost:5000/parse?query=xxx 报错

2019-04-01 06:04:18+0000 [-] Log opened.
2019-04-01 06:04:18+0000 [-] Site starting on 5000
2019-04-01 06:04:18+0000 [-] Starting factory <twisted.web.server.Site object at 0x7f7784e01160>
2019-04-01 06:04:24+0000 [-] 2019-04-01 06:04:24 WARNING rasa_nlu_gao.project - Using default interpreter, couldn't fetch model: Unable to initialize persistor
2019-04-01 06:04:24+0000 [-] 2019-04-01 06:04:24 ERROR main - Unable to initialize persistor
2019-04-01 06:04:24+0000 [-] Traceback (most recent call last):
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/rasa_nlu_gao/server.py", line 238, in parse
2019-04-01 06:04:24+0000 [-] self.data_router.parse, data))
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
2019-04-01 06:04:24+0000 [-] result = inContext.theWork()
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in
2019-04-01 06:04:24+0000 [-] inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
2019-04-01 06:04:24+0000 [-] return self.currentContext().callWithContext(ctx, func, *args, **kw)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
2019-04-01 06:04:24+0000 [-] return func(*args,**kw)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/rasa_nlu_gao/data_router.py", line 249, in parse
2019-04-01 06:04:24+0000 [-] model)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/rasa_nlu_gao/project.py", line 119, in parse
2019-04-01 06:04:24+0000 [-] interpreter = self._interpreter_for_model(model_name)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/rasa_nlu_gao/project.py", line 197, in _interpreter_for_model
2019-04-01 06:04:24+0000 [-] metadata = self._read_model_metadata(model_name)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/rasa_nlu_gao/project.py", line 212, in _read_model_metadata
2019-04-01 06:04:24+0000 [-] self._load_model_from_cloud(model_name, path)
2019-04-01 06:04:24+0000 [-] File "/usr/local/lib/python3.6/site-packages/rasa_nlu_gao/project.py", line 251, in _load_model_from_cloud
2019-04-01 06:04:24+0000 [-] raise RuntimeError("Unable to initialize persistor")
2019-04-01 06:04:24+0000 [-] RuntimeError: Unable to initialize persistor
2019-04-01 06:04:24+0000 [-] "172.17.0.1" - - [01/Apr/2019:06:04:23 +0000] "GET /parse?query=%E6%88%91%E8%A6%81%E6%9F%A5%E6%98%A8%E5%A4%A9%E4%B8%8B%E5%8D%88%E7%9A%84%E6%B5%81%E9%87%8F HTTP/1.1" 500 47 "-" "PostmanRuntime/7.6.0"

发现2处BUG

OOV_token 具体有什么作用呢

库文件缺少啊

我自己整理一下，不知道对不对
numpy==1.14.5
jsonschema==2.6
pyyaml==3.12
packaging==17.0
setuptools == 39.1.0
sklearn_crfsuite
jieba
duckling
keras
##twisted
rasa-nlu == 0.13.4
rasa-nlu-gao == 0.2.0
rasa-core == 0.12.3
rasa-core-sdk == 0.12.1

预训练词向量的问题

之前看到config.yml文件配置有：

language: "zh"
pipeline:
- name: "nlp_mitie"
  model: "data/total_word_feature_extractor.dat"
- name: "tokenizer_jieba"

然后你在百度网盘提供了total_word_feature_extractor.dat的下载，我下来发现只有40几M，
请问你提供的这个total_word_feature_extractor.dat是英文的吗？
因为我看到你简书博客里提到的中文total_word_feature_extractor.dat有300多M

训练非常慢啊

用了configs/nlu_model_config.yml训练非常慢, 有什么模型可以训练快点, 效果合适的吗

FormPolicy执行不正常

您好！我现在有这么一个场景：当我们输入“”查询车牌号码粤A12345“”，rasa core能够返回请求的数据给我，但当我再次输入“”再帮我查一下身份证号码“”时，rasa core却把上一次车牌的结果返回给我，而没有问我继续输入身份证号码，请问是什么原因导致的呢？slots值得填充也不对。期待解答！谢谢！

Rasa Core logs：

192.168.0.110 - - [2019-06-08 11:44:37] "POST /webhooks/rest/webhook HTTP/1.1" 200 452 1.066897
2019-06-08 11:44:51 DEBUG    rasa_core.tracker_store  - Recreating tracker for id '152869574032212890:123'
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Received user message '查询车牌号码粤 A12345' with intent '{'name': 'request_number', 'confidence': 0.964844109378438}' and entities
'[{'entity': 'item', 'value': '车牌', 'start': 2, 'end': 6, 'confidence': None, 'extractor': 'ner_mitie', 'processors': ['ner_synonyms']}, {'entity': 'number', 'value': 'A12345', 'st
art': 6, 'end': 13, 'confidence': None, 'extractor': 'ner_mitie'}]'
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Logged UserUtterance - tracker now has 170 events
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Current slot values:
        business: None
        item: None
        number: None
        requested_slot: None
2019-06-08 11:44:52 DEBUG    rasa_core.policies.memoization  - Current tracker state [None, None, None, {}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_action_listen': 1.0, 'inten
t_request_number': 1.0}]
2019-06-08 11:44:52 DEBUG    rasa_core.policies.memoization  - There is no memorised next action
2019-06-08 11:44:52 DEBUG    rasa_core.policies.form_policy  - There is no active form
2019-06-08 11:44:52 DEBUG    rasa_core.policies.ensemble  - Predicted next action using policy_1_MobilePolicy
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Predicted next action 'number_form' with prob 0.99.
2019-06-08 11:44:52 DEBUG    rasa_core.actions.action  - Calling action endpoint to run action 'number_form'.
2019-06-08 11:44:52 WARNING  py.warnings  - E:\ComPython\LittleTeligen\RasaCore\venv\lib\site-packages\urllib3\connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request i
s being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)

2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Action 'number_form' ended with events '['Form(number_form)', 'SlotSet(key: item, value: 车牌)', 'SlotSet(key: number, value:粤A12345)', 'SlotSet(key: item, value: None)', 'Form(None)', 'SlotSet(key: requested_slot, value: None)']'
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Bot utterance 'BotUttered(text: {"method": "queryCarInfo", "wccpproxy": "{\"result\": 1, \"resultMsg\": \"none\", \"list\": [{\"owne
rName\": \"\\u5f20\\u4e09\", \"idCard\": \"450981199305174536\", \"color\": \"\\u9ec4\\u8272\", \"firstRegDate\": \"2019-03-23\", \"brand\": \"BMW\", \"model\": \"x3\", \"carType\": \"
normal\", \"numType\": \"plate\", \"carNum\": \"\\u7ca4a12345\"}]}"}, data: {
  "elements": null,
  "buttons": [],
  "attachment": null
})'
2019-06-08 11:44:52 DEBUG    rasa_core.policies.memoization  - Current tracker state [None, None, {}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_action_listen': 1.0, 'intent_requ
est_number': 1.0}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_number_form': 1.0, 'intent_request_number': 1.0}]
2019-06-08 11:44:52 DEBUG    rasa_core.policies.memoization  - There is no memorised next action
2019-06-08 11:44:52 DEBUG    rasa_core.policies.form_policy  - There is no active form
2019-06-08 11:44:52 DEBUG    rasa_core.policies.ensemble  - Predicted next action using policy_1_MobilePolicy
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Predicted next action 'action_listen' with prob 1.00.
2019-06-08 11:44:52 DEBUG    rasa_core.processor  - Action 'action_listen' ended with events '[]'

192.168.0.110 - - [2019-06-08 11:44:52] "POST /webhooks/rest/webhook HTTP/1.1" 200 1184 1.067888
2019-06-08 11:44:58 DEBUG    rasa_core.tracker_store  - Recreating tracker for id '152869574032212890:123'
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Received user message '帮我查一下身份证号码 with intent '{'name': 'request_number', 'confidence': 0.9043155210530247}' and entities '[{'e
ntity': 'item', 'value': '身份证', 'start': 2, 'end': 7, 'confidence': None, 'extractor': 'ner_mitie', 'processors': ['ner_synonyms']}]'
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Logged UserUtterance - tracker now has 180 events
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Current slot values:
        business: None
        item: None
        number: 粤A12345
        requested_slot: None
2019-06-08 11:44:59 DEBUG    rasa_core.policies.memoization  - Current tracker state [None, {}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_action_listen': 1.0, 'intent_request_nu
mber': 1.0}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_number_form': 1.0, 'intent_request_number': 1.0}, {'entity_item': 1.0, 'prev_action_listen': 1.0, 'intent_request_number':
 1.0}]
2019-06-08 11:44:59 DEBUG    rasa_core.policies.memoization  - There is no memorised next action
2019-06-08 11:44:59 DEBUG    rasa_core.policies.form_policy  - There is no active form
2019-06-08 11:44:59 DEBUG    rasa_core.policies.ensemble  - Predicted next action using policy_1_MobilePolicy
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Predicted next action 'number_form' with prob 1.00.
2019-06-08 11:44:59 DEBUG    rasa_core.actions.action  - Calling action endpoint to run action 'number_form'.
2019-06-08 11:44:59 WARNING  py.warnings  - E:\ComPython\LittleTeligen\RasaCore\venv\lib\site-packages\urllib3\connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request i
s being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)

2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Action 'number_form' ended with events '['Form(number_form)', 'SlotSet(key: number, value: 粤12345)', 'SlotSet(key: item, value: None)', 'Form(None)', 'SlotSet(key: requested_slot, value: None)']'
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Bot utterance 'BotUttered(text: {"method": "queryCarInfo", "wccpproxy": "{\"result\": 1, \"resultMsg\": \"none\", \"list\": [{\"owne
rName\": \"\\u5f20\\u4e09\", \"idCard\": \"450981199305174536\", \"color\": \"\\u9ec4\\u8272\", \"firstRegDate\": \"2019-03-23\", \"brand\": \"BMW\", \"model\": \"x3\", \"carType\": \"
normal\", \"numType\": \"plate\", \"carNum\": \"\\u7ca4a12345\"}]}"}, data: {
  "elements": null,
  "buttons": [],
  "attachment": null
})'
2019-06-08 11:44:59 DEBUG    rasa_core.policies.memoization  - Current tracker state [{}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_action_listen': 1.0, 'intent_request_number':
 1.0}, {'entity_item': 1.0, 'entity_number': 1.0, 'prev_number_form': 1.0, 'intent_request_number': 1.0}, {'entity_item': 1.0, 'prev_action_listen': 1.0, 'intent_request_number': 1.0},
 {'entity_item': 1.0, 'prev_number_form': 1.0, 'intent_request_number': 1.0}]
2019-06-08 11:44:59 DEBUG    rasa_core.policies.memoization  - There is no memorised next action
2019-06-08 11:44:59 DEBUG    rasa_core.policies.form_policy  - There is no active form
2019-06-08 11:44:59 DEBUG    rasa_core.policies.ensemble  - Predicted next action using policy_1_MobilePolicy
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Predicted next action 'action_listen' with prob 1.00.
2019-06-08 11:44:59 DEBUG    rasa_core.processor  - Action 'action_listen' ended with events '[]'
192.168.0.110 - - [2019-06-08 11:44:59] "POST /webhooks/rest/webhook HTTP/1.1" 200 1184 1.083746

story：

# happy with request number
* greet
    - utter_answer_greet
* request_number
    - number_form
    - form{"name": "number_form"}
    - form{"name": null}
* thanks
    - utter_answer_thanks

action code：

class NumberForm(FormAction):

    def name(self):
        """Request Number action唯一标识"""
        return "number_form"

    def validate_number(self, value, dispatcher, tracker, domain):
        pass


    def validate_item(self, value, dispatcher, tracker, domain):
        pass


    @staticmethod
    def required_slots(tracker):
        number = tracker.get_slot('number')
        if number is not None:
            return ["number"]

        return ["item", "number"]


    def submit(self, dispatcher, tracker, domain):
        inputItem = tracker.get_slot("item")
        inputNum = tracker.get_slot("number")

        print("##### current slot:item={0},number={1}".format(inputItem, inputNum))
        
        return []

train core model configs：

def train_dialogue_keras_form(domain_file="configs/core/domain_form.yml",
                         model_path="models/dialogue_keras_form_new",
                         training_data_file="data/stories_data/form"):
    fallback = FallbackPolicy(
        fallback_action_name="action_default_fallback",
        nlu_threshold=0.4,
        core_threshold=0.3
    )

    agent = Agent(domain_file,
                  policies=[MemoizationPolicy(max_history=5),
                            MobilePolicy(epochs=100, batch_size=16),
                            FormPolicy(), fallback])

    training_data = agent.load_data(training_data_file)
    agent.train(
        training_data,
        validation_split=0.2
    )

    agent.persist(model_path)
    return agent

domain.yml：

forms:
  - number_form

intents:
  - affirm
  - deny
  - greet
  - goodbye
  - thanks
  - whoareyou
  - whattodo
  - whereyoufrom
  - mod_happy
  - mod_unhappy
  - mod_angry
  - mod_worried
  - request_number
  - request_business
  - search_item
  - say_jokes
  - ask_age
  - ask_sex
  - out_of_scope
  - welcome

slots:
  item:
    type: unfeaturized
    auto_fill: false
  number:
    type: unfeaturized
    auto_fill: false
  business:
    type: unfeaturized
    auto_fill: false

entities:
  - item
  - number
  - business

ERROR:rasa_core_sdk.executor:Failed to register package 'actions'.

执行这个python -m rasa_core_sdk.endpoint --actions actions的时候报上面的错误，说是没有注册

用了BERT吗？

怎么用BERT微调训练模型呢？

Knowledge Graph 心得

谢谢楼主的建议， 现在已经可以支持英文模式了。 在踩坑记中提到会分享写knowledge graph的心得。 请问knowledge graph文本是story.md吗？

每种intent是不是只能一种回答？如果同一个intent但是entity不同能更换回答吗？具体文件在哪里写， 有范例吗？

非常感谢楼主的耐心答疑。谢谢。

rasa不支持tensorflow-gpu吗？

请问下你这个支持 look up table 的特性吗

安装的时候碰到如下错误

ERROR: rasa-nlu 0.15.1 has requirement matplotlib~=2.2, but you'll have matplotlib 3.1.0 which is incompatible.
ERROR: rasa-nlu 0.15.1 has requirement packaging~=18.0, but you'll have packaging 19.0 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement fakeredis~=0.10.0, but you'll have fakeredis 1.0.3 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement fbmessenger~=5.0, but you'll have fbmessenger 6.0.0 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement packaging~=18.0, but you'll have packaging 19.0 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement pika~=0.12.0, but you'll have pika 1.0.1 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement python-socketio~=3.0, but you'll have python-socketio 4.1.0 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement pytz~=2018.9, but you'll have pytz 2019.1 which is incompatible.
ERROR: rasa-core 0.14.5 has requirement redis~=2.0, but you'll have redis 3.2.1 which is incompatible.
ERROR: rasa-core-sdk 0.14.0 has requirement ConfigArgParse~=0.13.0, but you'll have configargparse 0.14.0 which is incompatible.
ERROR: fakeredis 1.0.3 has requirement six>=1.12, but you'll have six 1.11.0 which is incompatible.
Installing collected packages: greenlet, gevent, rasa-sdk, redis, fakeredis, python-socketio, async-generator, fbmessenger, async-timeout, typing-extensions, idna-ssl, yarl, aiohttp, python-dateutil, matplotlib, sanic-jwt, packaging, rasa, rasa-nlu-gao
Found existing installation: greenlet 0.4.13
ERROR: Cannot uninstall 'greenlet'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.