Jarbas is a fork of mycroft-core

This repo is now deprecated, fork being redone from scratch, updated links

• new account -https://github.com/JarbasAl
• dev branch - https://github.com/JarbasAl/jarbas-core
• private edition -https://github.com/JarbasAl/jarbas-core/tree/private
• server edition - https://github.com/JarbasAl/jarbas-core/tree/server

everything can break anytime! things designed for mycroft may not work with jarbas! some things designed for jarbas wont work in mycroft! support for this branch is limited to me alone, folks from mycroft community may not be able to help

that said, everything that works in mycroft i try to make work with jarbas, but i give absolutely no warranty of this!

• bus address is different, but it is supposed to be configured in config file, that means the 3rd party should support it and not hard code it
• msm temporarly removed, skill install is old-school manual install
• no use of mycroft.ai, this means no web ui (yet) and harder setup
• no online config, must edit files
• no mycroft stt, must configure Kaldi or other online service
• no shared api keys (weather/wolfram...)
• some skills not fully functional / added yet / configured for server / undocumented setup

Still interested?

This branch (patch-15) is a work in progress, forever in development, the other branches should all be considered broken / abandoned / temporary

Check the jarbas_docs folder for more info on each change, expect this to be outdated

This should install in a virtual env just like mycroft, only debian install scripts were updated

virtual env is named jarbas , it is possible to have both mycroft and jarbas at same time, each one running in a different virtual env and using different bus adress

whenever official mycroft documentation mentions "workon mycroft" use "workon jarbas"

not everything will work out of the box, expect bumps

Personal assistants today are glorified voice activated script runners, with the bonus of mining your data for some company

Stick some hardware and they are glorified alarm clocks than run scripts instead

Jarbas fork intends to show that mycroft is much more than this by harnessing the power of open source, an AI assistant should do "AI stuff" on demand, where is the AI in mycroft/jarbas?

     #### TODO links in each one ####

The essential parts that make it work, many options available for each

    - speech to text
    - text to speech
    - intent from text (text to action)

The "AI stuff" it can already do for you

    - Image classification
    - Object recognition
    - Face recognition
    - Voice recognition (wip)
    - Deep Dreaming
    - Generate names with Restricted Boltzman Machines
    - Generate Poetry with MarkovChains
    - Copy the style of any artist using Style Transfer
    - Colorize Black and White photos
    - Chatbot using AIML
    - Auto complete your stories with a Char-RNN (training...)
    - Learn and build a database of what it learns with LILACS

And dont forget the "not really AI" stuff

    - post to facebook / twitter / instagram
    - control a browser like a real human and do any kind of stuff with it
    - generate psychedelic images using mathematics
    - search the internet for answers
    - Search and play music from youtube (or spotify or more places!)
    - Stuff not worth mentioning like parrot back, dictation, tell jokes, hows
    the weather, which asteroids are near earth, how many sunspots currently
    visible, show you the latest image of earth from orbit..

In practice it can act as a vocal search engine, a helper for disabled people, a desktop automation/OS framework, an artificial pet, a Art assistant, a Writers Assistant, an educational toy for children, A work of art by its own merit...

        - no pairing
        - no config update from mycroft.ai
        - added "intent.start", "intent.end", "intent.failed" messages
        - skills/core.py SKILLS_DIR is now jarbas_skills folder instead of opt but configurable
        - do not remove articles in normalize (too much normalization IMO)
        - allow for multiple hot words of various kinds
        - offline stt (pocketpshinx) <- WIP
        - wakeword/hotword/standupword snowboy support
        - fallback skill order override in config file
        - apis centralized in self.APIS for all skills
        - message_context update methods added to MycroftSkill
        - update_config method added to MycroftSkill
        - screen display service
        - speech messages transmit metadata together with speech
        - speech messages set adapt context for LastSpeech and every metadata field
        - recognizer_loop:speak / hotword / external stt message types
        - save utterances if desired (configurable, disabled by default)
        - utterance and hotword save paths configurable
        - external STT requests (transcribe audio from a skill)
        - runtime shutdown/reload of skills
        - runtime changes of configuration (tts/stt/hotwords)
        - centralized API section in config
        - ConfigurationManager.save checks for config existence before loading it
        - webchat client
        - encrypted multi instance networking (server/client)
        - multiple connected clients support, authorization per client per action
        - many helper classes
        - many skills bundled by default

        # temporary changes
        - no msm

Display Service Demo

Hotwords/Snowboy Demo

TTS runtime change Demo

Facebook chat client Demo

LILACS learning demo

Morse Code TTS engine joke

AIML chatbot demo

only debian officially supported, ubuntu complains because firefox package name is different

• build_host_setup_debian.sh
• build_dev_setup.sh

if you don't care about pre-packaged skills / using a pi

• build_host_setup_debian_light.sh
• build_dev_setup_light.sh

this won't install most skill requirements, only core and client/server requirements

in the future will install pymimic instead of mimic

to update packages:

• git pull -> for merging this fork changes
• re-run host setup script -> for updating system packages
• run dev-update.sh or dev-update-light.sh -> update python packages in virtual env

using full update scripts:

this means overwrite all changes and revert to current github state, update host packages, and update dev packages

• full_update_debian.sh or full_update_debian_light.sh

to start it is the same as mycroft-core, jarbas.sh and mycroft.sh are duplicates

• TODO - all usage flags -c -v -cl -s -w
• ./jarbas.sh start

or with different processes

• ./start.sh service
• ./start.sh skills
• ./start.sh audio
• ./start.sh display
• ./start.sh voice
• ./start.sh cli
• ./start.sh server
• ./start.sh client
• ./start.sh webchat

all others scripts are not meant for use, open them to understand what they do, comments explain most things

of interest:

• pymimic install scripts
• tensorflow model download scripts in jarbas_models/model_folder
• all mimic voices download script

IntentLayers class, allows to activate and deactivate intents grouped in layers

Example usage in KonamiCode Skill

Backend Service, A helper class to emit and wait for bus messages was added, needed in several skills

Used for inter skill communication in image recognition, face recognition, object recognition, deep dream, style transfer, vision, user manager...

Markov Chain helper class, save and load functionality

Example usage in Poetry Skill

Generate random pictures utility based on this blog post

Example usage in Art Skill

IntentParser class

        - helper class added to intent_service
        - intent from utterance
        - skill from intent
        - skill from utterance

priority skills list

        - priority skills are loaded first
        - read from config

needed for intent_layers

        - enable / disable intent can be requested in messagebus

control skills externally, allow run-levels, external skill load and reload)

        - skills have internal id, internal id used instead of name in intent messages
        - skills main now listens for external messages for skill_manifest, skill_shutdown and skill_reload

target bus messages

        - uses message context to target messages to source of utterance
        - uses message context to mute (not trigger tts)
        - uses message context to tell if more speech is expected
        - adds "metadata" field to speak method
        - clients check if they are the target of utterance before processing it
        - adds data to message context:
            - target = cli/speech/facebook - default = "all"
            - mute = trigger tts or not - default = "false"
            - more_speech = more speak messages expected- default = "False"
            - destinatary = used to track remote_client to send message to
            - source = source of message, default = source skill / source client

stop speak messages

        - adds disable_speak_flag to speech and cli
        - this flag is a global mute switch, if set no tts is ever made
        - skill to mute/unmute not yet merged

jarbas networking

        - client / server (a "firewall" that connects to the mycroft messagebus )
        - client connects to server, by secure websockets
        - server sends public pgp key
        - client sends public pgp encrypted with server pgp
        - server sends AES key encrypted with client pgp, client pgp key authorizes client / creates account in server
        - all communication after is AES encrypted, if plaintext received client is kicked
        - server does all kinds of processing (receiving files, requesting stuff from clients, authorizing action per client)
        - on first run a new pgp key is automatically created (client and server)

client server basically connect jabas/mycroft instances, server can work as:

        - fallback mechanism
        - chat room
        - relay
        - ...

Lots of skills pre-packaged

Browser Service

Objectives Service

TTS Control - change voice at runtime vocally

Control Center - shutdown/restart skills , enforce run levels

Vision Service - process webcam input

RBM Service - Generate new words

Object Recognition

Image Classification

Face Recognition

Dictation - Write to txt user speech

News - Plays news

Mute - Mute TTS

Again - Repeat last action

LILACS - Search wikipedia, wikidata, dbpedia, conceptnet, wordnik, wolframalpha and learn

TODO finish this

some skills are blacklisted because they are either not common, heavy, or because i run them in server

• browser service
• server fallback
• LILACS_core
• LILACS_storage
• LILACS_teach
• LILACS_users
• LILACS_curiosity
• deep dream
• image recognition
• object recognition
• colorize
• vision
• facebook
• twitter
• instagram

these skills have a flag in their init method to ask server or run locally

consider them broken, get them from server branch if you really want them

• image recognition
• face recognition
• object recognition
• deep dream
• style transfer
• colorize

these skills have a flag to ask client

• vision service # wip

not updated often

• Help Skill
• R2D2 TTS
• instagram skill
• pocket sphinx offline stt
• kaldi install scripts
• class vizualization/deep draw in tensorflow
• models download script (they auto download on first skill run which takes...long)