Failed to open audio device about mimic1

[LongBoolean]

do we know what version of ubuntu?

[ryanleesipes]

@LongBoolean 15.10, if there is anything you need from me. Let me know. It's not truly stock Ubuntu, but someone has the same issue on Mac OS X after getting it compiled.

[zeehio]

Make sure you have installed libasound2-dev. Then flite/mimic shoud detect alsa as the audio module as far as I can remember 😄

[ryanleesipes]

Added libasound2-dev, I didn't have it, but it didn't seem to change the error. Perhaps I have to restart my session?

[zeehio]

I am cloning and checking by myself. Perhaps removing config/config and recreating with ./configure?

[LongBoolean]

I will test it out on ubuntu 15.10 in a vm then so we know for sure what dependencies are needed. @zeehio if that is the case I will need to add libasound2-dev to the list of dependencies

[zeehio]

There is an issue somewhere in the audio module detection. oss_audio is being used although alsa should be used if available.
I can't take a look on it right now, but just to do a quick test I could force the use of alsa (and make everything work) by renaming src/audio/au_oss.c to src/audio/au_oss2.c and removing the following lines from src/audio/native_audio.h:

#ifdef CST_AUDIO_LINUX

#define AUDIO_OPEN_NATIVE audio_open_oss
#define AUDIO_CLOSE_NATIVE audio_close_oss
#define AUDIO_SET_SAMPLE_RATE_NATIVE audio_set_sample_rate_oss
#define AUDIO_WRITE_NATIVE audio_write_oss
#define AUDIO_DRAIN_NATIVE audio_drain_oss
#define AUDIO_FLUSH_NATIVE audio_flush_oss

#endif

Further work is needed for a proper solution.

EDIT: You may also want to try on Ubuntu using padsp ./bin/mimic "Flite is a small engine". padsp is the PulseAudio OSS wrapper.

[LongBoolean]

@zeehio padsp ./bin/mimic -t "Hello" worked

[LongBoolean]

Ok I tried it out (also a problem in 14.04), got the error, installed libasound2-dev ran make clean then ./configure and then make. This command ran without error: ./bin/mimic -t "Hello" hazzah!!!

[siavasht]

I'm also on Trusty and libasound2-dev is not installed but I could run mimic without any problem. Maybe the problem is in another package?

[zeehio]

There are several ways to output audio in Linux. For instance: OSS, ALSA, PulseAudio

Flite/mimic provides an interface to several of these modules. At some point (I would say at compile time, but I am not sure right now), an audio module is selected based on the available libraries. You may not have ALSA, but you may have OSS or PulseAudio, or you may have a compatibility library/plugin (like padsp) able to provide an OSS interface to your PulseAudio sound system.

Without further information it is hard to know what audio modules are installed on your system, what libraries do you have available (with headers for compiling). Additionally we would need to check what are the mimic audio preferences if multiple audio options are available.

[LongBoolean]

@zeehio for now I have added libasound2-dev to the dependencies list as it seems this will allow it to work for the majority of people. Do you see any harm in asking users to install this?
@siavasht does installing libasound2-dev cause any problems with your particular system?

[zeehio]

@LongBoolean Great (and nice to meet you)! There is no harm in install audio library headers in an audio related application 👍 . In the end probably we could remove all the audio modules not properly tested by us. If we support multiple audio modules then we would increase the space of possible configurations to test (and testing and debugging audio issues is not easy).

[siavasht]

@LongBoolean can confirm that installing libasound2-dev (and recompiling mimic) didn't cause any errors on my machine.

[ryanleesipes]

Works well for me. Anyone have a Mac that can test what needs to be there to run it in that environment?

[siavasht]

I just did a quick build on my university's macbook. Although there were warnings, compilation was done successfully but there is no audio when mimic is run. Unfortunately That's as far I can say about the situation since I don't have that much experience with mac systems. If you need more info or I should try other things just ping me and I'll do my best. @ryanleesipes

[zeehio]

I haven't seen any audio backend for OS X in flite/mimic and Flite's changelog says that mac support started on flite 1.2 without audio output. Work is needed to make mimic talk on a Mac.

What do we want

• Audio output system
• Multiplatform
• Low latency

What do we have

• Several audio clients
  • Open sound system (Linux, FreeBSD?)
  • ALSA (Linux)
  • PalmOS
  • PulseAudio (Linux)
  • Sun/Solaris
  • WinCE

Possible roads forward

a) Write more clients (OSX?, Windows?...), test the clients we want to (are able to) support
b) Write a client based on a library such as PortAudio that abstracts platforms away from us.
c) Generate wav files and play them using an external player
d) What is the speech recognition plan for audio input in mycroft? Is it ALSA based or is it multiplatform?

I would choose b). Maybe d) if it suits us. What do you think?

[forslund]

To have a single audio backend (as proposed in b) would probably be nice. It moves away a bit from the original thought to have flite as stand alone as possible, I'm not sure how important we think this is for mimic. portaudio seem to be good choice, it's readily available as a package in ubuntu, debian and seems to be included in os x homebrew.

As an initial step we could add portaudio as a sound client in addition to the existing ones, this would cause no trouble and in the future we could make the build system prefer portaudio if it's available.

And in the faaaar future if we find that the alsa/oss/pulseaudio clients are unused and causes problem we could remove them.

[julianrichen]

Hello,
I'm running Debian Sid and had libasound2-dev installed before compiling. I'm also getting the same issue as @ryanleesipes:

$ mimic -t "Hello"
oss_audio: failed to open audio device /dev/dsp

As @LongBoolean suggest running the PA wrapper works:

padsp mimic -t "Hello"

Only difference I did during compile was using checkinstall to create a deb file, so I could easily remove it afterwards.

I'm not technically skilled with C (still learning) and I can't really contribute but if you want more testing on others systems I'm always happy to help.

[zeehio]

I'll pull request tomorrow the fix for your issue @julianrichen. Your issue is that flite needs libasound > 1.0.11 and this check is done in the configure script by comparing just the "11". In sid there is libasound 1.1.0 so the dumb check fails.

Look for SND_LIB_SUBMINOR inthe configure script and replace the eleven with a -1 for a quick workaround if you don't want to wait for my PR 😄.

[rhdunn]

Some comments regarding using portaudio:

1. It does not support pulseaudio, audioio (some BSDs, Solaris and SunOS) and several other audio systems;
2. It has a complex API -- you cannot just use the open/write/close model;
3. The API has changed from v18 to v19;
4. It is complicated to enumerate and select a given device.

NOTE: PortAudio supports:

1. ALSA (Linux)
2. ASLHPI
3. ASIO
4. CoreAudio (Mac)
5. DSound (Windows 95+ -- emulated via WASAPI when running on Vista+)
6. JACK
7. OSS (POSIX, such as BSD and Linux -- need to use padsp if running PulseAudio)
8. Skeleton
9. WASAPI (Windows Vista+ -- need to use the matching sample rate/channel count/format of the device)
10. WDMKS (Windows)
11. WMME (Windows)

If you are willing to incorporate GPLv3 code, I have a https://github.com/rhdunn/pcaudiolib project that I am working on supporting multiple audio system APIs, with an API matching the flite one (open/close/write/drain/flush/strerror/create/destroy). This currently supports:

1. ALSA (Linux)
2. OSS (POSIX, such as BSD and Linux -- need to use padsp if running PulseAudio)
3. PulseAudio (Linux, via the simple PulseAudio API) -- currently missing a runtime check to see if it is running
4. QSA (QNX)
5. XAudio2 (Windows) - basic, initial implementation

I intend to add more (patches welcome!) and do other things like support writing to wave, ogg/vorbis, etc. files through this API.

[zeehio]

I plan to submit a patch in <10 hours (when I get back home) that adds support to PortAudio. I just need time after work to polish some details. I will need someone to try it on a Mac, as I don't have the hardware.

With respect to pcaudiolib, it looks smaller than PortAudio (fitting better to mimic needs probably) but it currently seems to lack OSX support.

[forslund]

@zeehio Great! I created a separate issue for the OS X support as this issue is in my opinion more focused around the requirements and config script. Looking forward to your pull request!

[ryanleesipes]

@rhdunn I'm not sure the GPLv3 license is compatible with the BSD-based on CMU thought up to put on Flite (What Mimic is based on). But if anyone knows more about this and could speak to this I would be very interested to hear feedback.

But ultimately I refer to @LongBoolean @forslund and @zeehio to weigh in on this. Great discussion on this issue so far!

[rhdunn]

The only discussion I could find relating to CMU software license was https://www.mail-archive.com/debian-legal%40lists.debian.org/msg40595.html, which regards the clauses used in flite as acceptable for open source (regarding the Debian Free Software Guidelines (DFSG)). In that discussion, the Debian Legal team were discussing the [cmu_hatian_speech])http://www.speech.cs.cmu.edu/haitian/speech/) license terms, specifically regarding clause 5 (not present in other CMU speech projects) that is effectively an advertising clause and thus not DFSG (nor GPL) compatible.

There are also GPL projects that wrap flite (e.g. a GPLv2 plugin for asterisk).

[zeehio]

I believe that with pull request #14 we have good support on Windows, Mac OSX and GNU/Linux so this issue can be closed.

Adding GPL-v3 code to mimic or linking to a GPL shared library would imply to relicense mimic under GPL-v3.

• From a legal perspective, and in my non-lawyer opinion, this is feasible as I have not found any clause in mimic source incompatible with GPL-v3 (however I believe a lawyer would need to be consulted).
• From a technical perspective, this would make mimic changes not easily backportable to flite or bellbird breaking the already existent BSD-like ecosystem.

Given that pcaudiolib is not the library that fixes all our audio problems today (hopefully in the future!), and considering that it is still a young library, I would not consider to provide support through it in mimic. However, I am looking forward to its development and I will consider it in the future, whenever I need audio support.

If any of you wants to continue discussing mimic licence terms or further audio modules, feel free to open an issue. I will close this issue as the audio device can be opened if the dependencies are installed after #14 .