How to know when plugin is done listening and has a result about speech_to_text

[sowens-csd]

The issue seems to only happen in Android. The duplicate onResult calls seem to be happening only when Android is auto-terminating the listen session due to a pause. In the example app if you can click the stop button before the pause detection shuts it down then you get one callback. From what I can tell so far Android is sending a duplicate notification in that case. I'm trying to figure out how to detect and block the duplicate.

[sowens-csd]

Yes, you can do that using the partialResults flag on the listen method. It defaults to true, set it to false to only receive the final results. See the documentation on the listen method for details.

speechToText.listen( onResult: resultCallback, partialResults: false )

[ElZombieIsra]

Thanks. I didn't know this param existed because I was using an older version. This seems to solve my problem, thanks.

Although the onResult function seems to be triggering twice when the plugin recognize words. You can even hear the result audio in android playing twice.

At first I thought i was triggering twice the "listen" function but I checked my logs and doesn't seem so.

[sowens-csd]

I'm not sure what you mean by "You can even hear the result audio in android playing twice."? The plugin doesn't play audio, it just listens. I'll check for duplicate invocation of the onResult.

[ElZombieIsra]

There's a default audio playing in Android every time you listen for speech. It's in the README too. You have a section where you can set an audio to play in iOS.

[sowens-csd]

Interesting, it does seem there's something odd going on with the onResult callback. Checking now.

[ElZombieIsra]

Thanks. Also, if it's not a lot to ask. You think you can add an onFinished callback or something that executes when the plug-in is done besides the onResult callback?

[sowens-csd]

When would you want this new onFinished to be called? What do you consider the 'plugin is done' state?

[ElZombieIsra]

I think this could be executed when the plug-in has recognized words but it is done listening. Like when the done audio is reproduced in Android.
That way you can still get the partial results of the recognized speech with the onResult callback and, say, print them in the UI, but you can execute another callback when there's no more speech to recognize since the plug-in is not listening anymore. And you can, say so, make an http call to your service with the final recognized speech.

I think this would add a lot of value to this plug-in since it would be a lot easier to use and implement than having to write that logic by yourself every time you want to use this.

[sowens-csd]

I think the finalResult property of the SpeechRecognitionResult does that. If that is true, then the recognition is done and you have the final results. The recognizer is no longer listening nor will it return further results until a new listen session is triggered. At that point you could trigger an http call or do whatever else you'd like with the results.

The bug with duplicate onResult notifications causes a problem with that, but I'll resolve that issue.

[ElZombieIsra]

Ok. Let me check out it's behaviour and see if that's useful to me.

Thanks

[sowens-csd]

New version in the repo now. It should resolve the duplicate notifications.

[yoali11]

New version in the repo now. It should resolve the duplicate notifications.

FYI the new version didn't fix the duplicate onResults call. This bug is from within Android SpeechRecognizer. I'm experiencing it myself on my own SpeechRecognition plugin.

[sowens-csd]

I agree that it's coming from within the Android part of the plugin, that's where I put the fix. You say "...on my own SpeechRecognition plugin...' and I'm not quite sure what you meant by that? Are you using the latest version of Speech_to_text plugin and seeing duplicates? Or have you written your own plugin for SpeechRecognition and are seeing a similar problem? Could you describe the behaviour you are seeing? What OS and device type? Are you getting partial results or final results? You'll potentially see duplicates for partial as I didn't try to address that.

There are possible problems with the fix since it is really more of a work around. The issue seemed to me to be that the Android callback is getting triggered multiple times for the same recognition task. To resolve it I don't fire a notification if the trigger happens < 100 ms apart. It is possible on a slow device that the duplicates could come further apart than that. If you turn on debugLogging on the initialize call then there's a log message every time a duplicate is dropped. If you're still seeing the duplicates issue in this plugin then I'll add some logging and ask you to try running it to see what the behaviour is on your device.

[ElZombieIsra]

I think the finalResult property of the SpeechRecognitionResult does that.

You were right about this statement. That works perfectly for me. Thanks.

I will close this issue because it has been resolved.
I think the issue about the double call in android may be treated in a different issue unless you think otherwise.

[yoali11]

I agree that it's coming from within the Android part of the plugin, that's where I put the fix. You say "...on my own SpeechRecognition plugin...' and I'm not quite sure what you meant by that? Are you using the latest version of Speech_to_text plugin and seeing duplicates? Or have you written your own plugin for SpeechRecognition and are seeing a similar problem? Could you describe the behaviour you are seeing? What OS and device type? Are you getting partial results or final results? You'll potentially see duplicates for partial as I didn't try to address that.

I am using the android.speech.SpeechRecognizer API in my own custom Flutter plugin and I am too experiencing the duplicate call of onResult. Only on final results. I am pretty sure this bug is within the Android API and not your (or my own) plugin.

[sowens-csd]

Ah, thanks for letting me know! I agree with you, that does seem to be what's happening. Strangely I haven't seen any discussion of this problem. If you're curious have a look at the work around I put into the Android plugin logic to work around the issue. It might help with your use case.

[yoali11]

Thanks. Although the fix won't remove the annoying duplicate stop-sound. It seems to be that Google don't prioritise fixing issues in SpeechRecognizer since they launched their payed version, Google Cloud Speech API...