Multiple plugins of the same resource about unimrcp

[GoogleCodeExporter]

Supported in r697. 

Multiple plugins of the same resource can be specified and loaded
    <plugin>
      <engine name="Cepstral-Swift-1" class="mrcpcepstral" enable="1"/>
      <engine name="Demo-Synth-1" class="demosynth" enable="1"/>
      <engine name="Demo-Recog-1" class="demorecog" enable="1"/>
    </plugin>

One profile may use Cepstral-Swift-1 plugin, while another one Demo-Synth-1 
plugin

    <profile name="MRCPv2-Default">
      <param name="signaling-agent" value="SIP-Agent-1"/>
      <param name="connection-agent" value="MRCPv2-Agent-1"/>
      <param name="media-engine" value="Media-Engine-1"/>
      <param name="rtp-factory" value="RTP-Factory-1"/>
      <pluginmap>
        <param name="speechsynth" value="Cepstral-Swift-1"/>
        <param name="speechrecog" value="Demo-Recog-1"/>
      </pluginmap>
    </profile>
    <profile name="MRCPv2-Alter">
      <param name="signaling-agent" value="SIP-Agent-2"/>
      <param name="connection-agent" value="MRCPv2-Agent-1"/>
      <param name="media-engine" value="Media-Engine-1"/>
      <param name="rtp-factory" value="RTP-Factory-1"/>
      <pluginmap>
        <param name="speechsynth" value="Demo-Synth-1"/>
        <param name="speechrecog" value="Demo-Recog-1"/>
      </pluginmap>
    </profile>

Sample unimrcpserver.xml configuration file is attached.

Original comment by [email protected] on 18 Dec 2008 at 8:59

Attachments:

• unimrcpserver.xml

[GoogleCodeExporter]

Original comment by [email protected] on 18 Dec 2008 at 8:59

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by [email protected] on 12 Jan 2009 at 8:17

• Added labels: Milestone-Release-0.4

[GoogleCodeExporter]

I was trying to test this, but I had only 1 PC (and no virtual machines...)
Although one unimrcpclient can create multiple sip-agents, there is no way to 
choose 
the agent to run the synth app. So I created two directory trees, one for each 
client, each with its own configuration file.
However, the 2nd instance of the client always has trouble (both clients work 
if I 
run them one after the other, but not simultaneously)
The second client will log the following warnings:

2009-01-14 22:39:45:715724 [INFO]   Start Media Processing Engine
2009-01-14 22:39:45:715724 [WARN]   Failed to Create Control Socket
2009-01-14 22:39:45:715724 [WARN]   Failed to Create MRCPv2 Agent Socket

Perhaps I made an error in the conf files (see attached files)

Original comment by [email protected] on 14 Jan 2009 at 10:01

Attachments:

• unimrcpclient.xml
• unimrcpclient.xml
• unimrcpserver.xml

[GoogleCodeExporter]

You can test it using 1 unimrcpclient and 1 unimrcpserver running on the same 
PC.
I've checked your unimrcpserver.xml, it looks good. Find attached modified
unimrcpclient.xml.
From unimrcpclient console input
> run synth MRCPv1-Default
or
> run synth MRCPv1-Alter

The signature is 
run "app_name" ["profile_name"]
However it seems I just forgot to mention the last optional param in help.

Original comment by [email protected] on 14 Jan 2009 at 10:29

Attachments:

• unimrcpclient.xml

[GoogleCodeExporter]

Thanks. 

Aha, I should have checked the ultimate documentation: the source code...
I'll check it later today.

Still makes me wonder though why running two or more instances of 
unimrcpclienct on 
the same PC is not OK (i.e. Should I worry about the following scenario where I 
need 
a seperate client for each IVR line, and each ivr line may want to "use" 
different 
plugins (the best TTS for French and another best one for Dutch etc...)

Original comment by [email protected] on 15 Jan 2009 at 8:51

[GoogleCodeExporter]

> Still makes me wonder though why running two or more instances of 
unimrcpclienct on 
the same PC is not OK

OK, I've just checked this. The issue is in hard coded port number used to 
create
control socket. See mrcp_client_connection.c:90. I used port 7856 temporarily,
actually any free port can be used. There are several easy workarounds, however 
it's
better to use wakeup pollset instead of control socket, as I did it for RTSP 
stack.
You may want to log another issue for this :)

> Should I worry about the following scenario where I need a seperate client 
for each
IVR line, and each ivr line may want to "use" different plugins
No, you don't. I guess you need separate signaling agent, but not the client.
I would not recommend to use several client instances for that. Typically you 
should
use the same client instance, while each IVR line can be connected through
appropriate profile / signaling agent.

Original comment by [email protected] on 15 Jan 2009 at 10:22

[GoogleCodeExporter]

Created the new issue...

Sometimes the IVR development environment (or those who built the IVR 
application) 
will not allow you to use the multiple agent scenario, and of course one can 
create 
another client-server/client layer. I have no particular IVR in mind though 
(and it 
was some time ago I was involved in building IVR applications ...)

I must confess I suspected a more or less hard coded port, however I didn't 
check 
the code yet. Perhaps - hopefully soon - I will have a better understanding of 
the  
the control socket - wakeup pollset design differences...

Original comment by [email protected] on 16 Jan 2009 at 8:44

[GoogleCodeExporter]

OK, it's not a great deal and it must be possible to run several instances of 
the
client on the same PC.
BTW, do you happen to verify the original issue. Does it work on your end?

Original comment by [email protected] on 16 Jan 2009 at 9:00

[GoogleCodeExporter]

I don't think I fully understand how it works or how it should work.
I wanted to try out the thread-safety of my TTS plugin.
So I want to synthesize a large text with multiple clients simultaneously.
What I see in the client is that the second run will not be able to complete 
the SIP 
call until the first call has ended. 

Original comment by [email protected] on 16 Jan 2009 at 10:48

[GoogleCodeExporter]

I'm afraid there is some kind of misunderstanding here.
What I have implemented in the scope of this issue is to be able to register
different TTS plugins with unimrcpserver and bind/attach each plugin to one of
signaling agents. For instance, Cepstral TTS plugin is used, if call comes from
SIP-Agent-1. Demo TTS plugin is used, if call comes from SIP-Agent-2, and yet 
another
plugin is used if call comes from SIP-Agent-3. 

Is this what you are looking for?
If yes, please proceed to how to configure and run unimrcpclient 
Configure multiple signaling agents and profiles as I suggested in attached
unimrcpclient.xml (Comment 5).
Run unimrcpclient.exe

> run synth MRCPv2-Default - to make call via SIP-Agent-1

> run synth MRCPv2-Alter - to make call via SIP-Agent-2

If you still have problems with the setup mentioned above, please upload logs 
and
configs of server and client to check.

Original comment by [email protected] on 16 Jan 2009 at 11:38

[GoogleCodeExporter]

I guess I understand it right. If you place calls one after the other is works 
fine. 
What I don't understand is whether one unimrcpclient should be able to allow 
two (or 
more) SIP agents to make calls in parallel. 
Things go wrong when using the two different agents in parallel, but I can't 
tell 
exactly what - I sometimes end up with two .pcm files of rougly the same size; 
to 
the demosynth "copy" of demo.pcm silence is appended, extending the duration 
to "match" the larger pcm file resulting from the fluency plugin. Additionally, 
one 
call has to wait for the other to complete.

I did not thorougly check the attached logs yet. So I don't know whether this 
is a 
client or a server (or plugin) problem (or a client "feature"). I'll try some 
more 
tests from another machine to see if it is the unimrcpserver that is causing 
this 
behaviour . However, it took a while to figure out how to distribute the dll's 
for 
the release versions of the software (I happened to use MSVC08 SP1, which 
caused 
trouble because I apparently installed incompatible "SP0" MSVC80 dll's from a 
microsoft vcredist*.exe). More later.

Original comment by [email protected] on 16 Jan 2009 at 3:50

Attachments:

• unimrcpclient.log
• unimrcpserver.log

[GoogleCodeExporter]

Well, I see we are on the right track.

Please note, unimrcpclient alllows to configure [0 - n] SIP signaling agents, 
while
each agent allows to make [0 - m] SIP calls in parallel.

What I've observed from logs so far.
I see SIP/MRCPv2 sessions with in-progress SPEAK requests in parallel. I'm 
pretty
sure, the problem is not in signaling layer, neither in SIP nor in MRCPv2 
stacks. My
guess is problem is in server side, most probably in the way media processing is
implemented in Fluency plugin.
There is a note on top of source files of plugins
 * 4. Methods (callbacks) of the MPF engine stream MUST not block.
Did you follow this instruction in synth_stream_read?

Original comment by [email protected] on 16 Jan 2009 at 5:33

[GoogleCodeExporter]

The 0 - n agents doing 0 - m calls, that sounds good! I'm still deeply 
impressed by 
your work. 
And you are almost right. Not the MPF engine, but the MRCP engine. The problem 
lies 
in the "fluency_synth_channel_speak" function. The actual synthesis has to be 
carried out in a separate thread, and that's not the way I implemented it. And 
to 
make things worse I added a apr_sleep as to not let the TTS engine generate 
more 
audio samples than can be retrieved from the client (the MPF engine). Sleeping 
in 
this case is blocking... (and I was thinking, this would yield some other 
threads) 
Unfortunately, in contrast to the cepstral/swift TTS engine, I have no callback 
function to retrieve the generated audio. I'll do some more homework...

I still don't fully understand why two different SIP agents using two different 
plugins have problems though - because there is only one connection agent ? - 
and 
having multiple connection agents ... ? 

Original comment by [email protected] on 16 Jan 2009 at 9:16

[GoogleCodeExporter]

>The problem lies in the "fluency_synth_channel_speak" function. The actual 
synthesis
has to be carried out in a separate thread, and that's not the way I 
implemented it

OK, this is definitely the reason.

> I still don't fully understand why two different SIP agents using two 
different 
plugins have problems though - because there is only one connection agent ? - 
and 
having multiple connection agents ... ? 

See the Sequence Diagram below
http://unimrcp.webhosting-for-free.com/ea/index.htm?goto=3:4:6:3:392

MRCP Server task/thread is the main entity in the hierarchy. It processes 
incoming
messages from Signaling, Connection agents, Media and Resource processing 
engines.

For instance, "fluency_synth_channel_speak" request is called from server task
context, thus it MUST NOT block, instead the request must be processed 
asynchronously.

Original comment by [email protected] on 17 Jan 2009 at 3:41

[GoogleCodeExporter]

Thanks! I did not find the sequence diagram yet. I thought I clicked my way 
through 
the ea docs, but I never saw the sequence diagram (I didn't find anything of 
interest in the plugin directory of ea). The activity and sequence diagrams of 
the 
MRCP client and server deserve links from the project home page! 

I should go back to school and learn to read properly - and take more time to 
study 
the project before starting to code. And while back at school, I should do some 
reading on software design, UML and enterprise architect... The quality of your 
work 
keeps on amazing me. In fact the project is much more mature than I ever 
expected.

To clarify my previous post (as far as I currently understand it): the hard 
coded 
port for the control socket(s) blocks the usage of multiple connection agents, 
even 
though you may configure multiple connection agents. However, as you explained, 
it 
would not have made a difference with respect to my problematic code.

BTW do you want another issue for the missing help parameters :-) ?

Original comment by [email protected] on 18 Jan 2009 at 8:56

[GoogleCodeExporter]

1.UML and docs
I know this stuff is not complete enough, and I have been trying to improve it 
as I
go along in background. Sometimes I have to sleep :)

2. Multiple connection agents in client
I'll address this issue, when possible. Anyway it would make no difference with
respect to server/plugin integration issue you experienced.

3. Missing help parameters
This is a trivial one, which I non intentionally forget to fix. I'll try to fix 
the
help during the day, otherwise please log it as another issue.

Original comment by [email protected] on 19 Jan 2009 at 7:50

[GoogleCodeExporter]

I saw you fixed the missing help parameter. Thanks.

I did another attempt at building a better plugin.
Somehow the plugins only log to the console - is that intentional?
(didn't check the documentation yet ;-)

Original comment by [email protected] on 21 Jan 2009 at 2:59

[GoogleCodeExporter]

Logger is probably the only static/global variable used in the project. See
apt_log.c:51. I prefer to have one globally available logger not to burden the
signature of the log function. Thus one instance of logger is created in the 
context
of unimrcpserver executable and yet another one in the context of your plugin 
dll.
You can initialize the logger the way you need in the mrcp_plugin_create() 
function.
For instance, 
    /* set the log level */
    apt_log_priority_set();
    /* set the log output mode */
    apt_log_output_mode_set();
    /* open log file */
    apt_log_file_open(log_file_path,MAX_LOG_FILE_SIZE,pool);

Original comment by [email protected] on 21 Jan 2009 at 3:46

[GoogleCodeExporter]

I created the threaded fluency_speak version and it works a whole lot better 
now.

With respect to logging: do you have a suggestion to retrieve the server's 
command 
line options and use them in the dynamically loaded plugin?

Original comment by [email protected] on 26 Jan 2009 at 3:21

[GoogleCodeExporter]

Nice to see you are making progress implementing plugin.

Hopefully soon I'll be able to proceed to the issue-11, which concerns to
capabilities of plugins, including logging. I think plugin should provide 
interface
to set logging options from the core depending on command line options or
configuration of the server.

Original comment by [email protected] on 26 Jan 2009 at 5:48

[GoogleCodeExporter]

Original comment by [email protected] on 28 Jan 2009 at 6:28

• Changed state: Verified