intel / acat

Assistive Context-Aware Toolkit (ACAT)

Official release of ACAT is available here

Background

The Assistive Context-Aware Toolkit (ACAT) is an open-source platform created at Intel Labs. It is designed to enhance communication for individuals with restricted speech and typing capabilities. It achieves this by providing functionalities like keyboard simulation, word/sentence prediction, and speech synthesis.

ACAT was originally developed by researchers in Intel Labs for Professor Stephen Hawking. He was instrumental to the design process and was a key contributor to the project design and validation. After deploying the system to Professor Hawking, Intel turned its attention to the larger community and continued to make ACAT more configurable to support a larger set of users.

By making this configurable platform open source, the goal is to encourage developers to continue to expanding its capabilities by adding new user interfaces, new sensing modalities, language prediction, and other features.

ACAT is developed in C# using Microsoft Visual Studio* 2022 and .NET 4.8.1. It runs on Windows® 10 (20H2 or higher) and 11.

Versioning

The master branch on Github will always contain the most recent release of ACAT.

The dev branch contains commits from the ACAT development team towards upcoming releases of ACAT. Do not clone the dev branch as it has changes that are still in flux and the solution may not be stable.

Cloning the repo

This repo uses Large File Storage (LFS) and includes submodules. You may run into problems if you use GitHub Desktop to clone it. As a workaround, use Git from the command line.

Clone the repo:
$ git clone https://github.com/intel/acat.git

If you are cloning for the first time:
$ git submodule update --init --recursive

Pull in all the latest changes for the submodules:
$ git pull origin master --recurse-submodules

License

ACAT is distributed under the Apache License, Version 2.0.

Project Website

Click here for the ACAT project website.

Contact the team

Click here to contact the ACAT team.

acat's Issues

Mouse control - Radar Scanning Mode

Hi,

Has the mouse radar scanning been removed? I can't find it among the tools.

Thanks

Sean

Where are the ACAT Vision's algorithms of facial gestures?

Hello! I am realizing that Vision is a external tool used through a client-server architecture. I would want to study its code and get the algorithms of facial gestures to develop a driver for Android. That system have the feature "access through switch" and I think that it could be used with the camera. Is the Vision a closed source project? But, same so, where is it? Who is developing it? (#2 seen already)

problems to install and set up

Hello,
I have a helmet to use this software.
Somebody can help me?
Give guidelines or tutorials , please

License for Documentation

@saiprasadb01, a translation in brazilian portuguese was started by @brlima94 and others guys. I would to change it to Gitbook and continue it , and for this I need know about the legality for our derivative work. See brlima94#1 (comment) please.

TERMS OF SERVICE for the 01.org site says:

Except as otherwise provided, Content on this site, including all materials posted by 01.org, is licensed under a Creative Commons Attribution 3.0 License.

More:

This Web site is a service made available by 01.org. All software, documentation, information and/or other materials provided on and through this Web site ("Content") may be used solely under the following terms and conditions ("Terms of Use").

I will send email to [email protected] while there is no answer here.

Cannot select voice in V1.0

I am running ACAT App on Windows 10 in Version 1.0.0 and I cannot change the voice for TTS.

In my windows settings, I have following voices available (german):

Stefan is selected and in my ACAT configuration I picked male, but I can't select a voice in the top:

I need a male voice and ACAT TTS reads with a female voice, ignoring my settings.

What can I do here? Thanks.

Feature Request: Is there a way to add an option for choosing contextual options?

Is there a way to add an option for choosing contextual options? I would like a way to edit the contextual options for the supported apps as well as an option to set the contextual menu for apps that have not already been optimized. To be clear, it would beneficial for end users to be able to change these as they wanted for their needs. @saiprasadb01

New pre-release of ACAT (v0.99) now available

Localization support added, plus a host of other features. Check out the release notes on the release page. Documentation has also been updated.
This release includes the base English version and a French version which can be downloaded separately as a Language Pack.
This is a binary-only pre-release. The source code and the official release will be in a couple of weeks.
The User Guide for Localization will be uploaded shortly which will cover details on translating strings, creating scanners etc.
Please let us know of issues/problems so we can address them before the final release.

How to change the voice from male to female?

My father has ALS and is new to the software. One issue that bothers him is the default voice is female, but he would like the default voice to be male. How can we change the default voice?

Search button disappeared for "Phrases" in v 1.0.0

The search button seems to have disappeared in the Phrases window list for "Phrases" application and when selecting the Phrases icon in all the "Talk" applications in ACAT v 1.0.0. So one cannot search for phrases starting with a typed in character using the search button from the "Talk" applications. Is there a fix for this?

Localization.

Hi guys! I am russian programer. I create keyboard for disables, whoose control only one button.
I wanna translete acat for russians. Can i do it?

More Language Support?

Hi,

first thank you very much for sharing this application with us! I have a disabled cousin who cannot really move or speak. I would like to make this app usable for him.

As we're living in Germany, it would be great if this app worked for the german language as well. Is that possible already or planned for the future?

Best regards,

Marcel

User Interface Translation!

Hello everyone,

i'm working on a French translation of ACAT UI,
i'll would like to translate all the menus, settings panel etc in order to facilitate the use of the program for French people, therapist, etc

my knowledge of code, xml, data bases etc is not very big,
but I managed to start with the translation of the smallest part of the UI: the Buttons!

here are some pictures of that first step:

the problem is I found out that in the xml files I could translate only the buttons or change the size of the toggle buttons

for the buttons "OK" "Cancel" "Defaults", we only have ton change the field: label="Someting"

but for the other lines that pertain to the text: "Auto-save scanner's last position", "Hide scanners on idle", "Show Talk window on startup", etc, ther are 8 Lines,
for thoses lines the label is: label="M"

when I change that M by something other, nothing happen,

then I tried to change the fontsize="36" by fontsize="24"

I found out the toggle button in front of the line, had changed size

So it does not have anything to do with the TEXT of the UI and my question is

where could I look for, in wich files are the English Texts???

thank you very much

Lack of buttons icon

I have installed the latest version of ACAT with the PT-BR translation installed on a notebook with updated Windows 10 64bit. In the first 2 runs all button icons popped up. From the 3rd execution there is the button but there is no icon to identify it as shown below. These same buttons have no icon on all screens. What can this be?

https://ibb.co/mmTusk

Exception thrown: 'System.BadImageFormatException' in mscorlib.dll

Hi, I have been trying to resolve a runtime problem with the source project I cloned from the master branch. I am getting the following error:

LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Attempting download of new URL file:///C:/Users/mmanka/ACATProject/acat/src/Applications/ACATApp/bin/Debug/Extensions/Default/Actuators/VisionActuator/opencv_videostab2412.dll.
ERR: Failed to complete setup of assembly (hr = 0x80131018). Probing terminated.

Additional Information:
Exception thrown: 'System.BadImageFormatException' in mscorlib.dll
DEBUG:[ACAT.Lib.Core.ActuatorManagement][Actuators.onFileFound] Could not get types from assembly. Exception : System.BadImageFormatException: Could not load file or assembly or one of its dependencies. The module was expected to contain an assembly manifest.

There seems to be a problem loading the assemblies (.dll) compiled, but I am not sure if a runtime compatibility problem is to blame for the mismatch in the format of the .dll assemblies. I tried changing the project build target in Visual Studio from 'X86' to 'Any CPU', but the same problem arised. I am using the .NET target framework version v4.5 and working in Visual Studio 2017 (32-bit) on a Windows 10 computer.

Any help would be greatly appreciated.

acat_gestures.exe has stopped working

Hello!

Everything is working fine, except on the ACAT Vision software. Everytime I try to open it, a message appears saying "acat_gestures.exe has stopped working". I've alredy followed the instructions and installed both VSU_4\vcredist_x64.exe and VSU_4\vcredist_x86.exe. For the fisrt one, the "acat_gestures.exe has stopped working" error appears and for the second one, the "The program can't start because MSVCP110.DLL is missing from your computer" error appears. I don't know what to do anymore and I'll be very grateful if someone could help me!

Thank you very much,
Leticia Botelho

Feature Request: Is there a way to add the scanning to other running programs?

Is there a way to add the scanning to other running programs? I am trying to use another program for help communicating and it has scanning built-in. The problem is that ACAT will not let me either use the built-in scanning or scan the already scan optimized program. If there were a way to scan programs that would be helpful. Even if something had to be built-in to the programs that would be fine. @saiprasadb01

UI y Diccionario en Español

Buenas tardes para toda la comunidad, mi nombre es Emiliano Martin Lafuente, no se si realmente ya se realizo, pero acabo de terminar de traducir toda la UI y configurar para que todos este en español el ACAT. La fonética la utiliza del narrador de windows así que queda en cada uno cambiarla, los usuarios de Windows 7 no la van a poder cambiar por que solo trae una y es la de Anna, pero tranquilos que se pueden bajar la de Jorge (Español - España) e instalarla. Ahora les muestro un par de capturas de pantallas para que vean mas o menos como queda.

Bueno les dejo los links para descargarselo:
acat.zip:
https://drive.google.com/open?id=0B_EUOrlEnWqSVHR5NXRlNFhJRDA
Presage.zip:
https://drive.google.com/open?id=0B_EUOrlEnWqSbnRRUmJyZFp1Szg

La instalación se hace de la siguiente manera:
Descargar e instale el programa Acat oficial, una vez finalizada la instalación, descomprimir los archivos zip modificados, copiar y reemplazar las carpetas en sus correspondientes rutas de instalación.

El presage se instala en C:\Program Files (x86)\presage *1
El ACAT se instalo en c:\INtel\Acat

*1 - cuando instalan el Acat oficial se instala tambien el Presage, este se activa ni bien se termina la instalación, asi que al momento de reemplazar la carpeta cierrenlo asi no les apareceran errores.

Espero que les sea útil, cualquier inconveniente o modificación avisen.

Saludos a todos.
Emiliano Martin Lafuente.

LOCALIZATION

Hello , we managed to finish the translation of the ACAT system for Brazilian Portuguese , we will make it available at the following link, http://brlima.com.br/acat if someone is interested !

Thanks for your help .

How to connect switch for F12

Hi,

how to connect a switch in order to send the F12 key to ACAT?

A friend of mine has this switch:
http://www.auxilia.it/site/lang/it-IT/page/18/product/147

Ideally this USB adapter should work but doesn't allow to send F12:
http://www.bjliveat.com/access/usb-switch-interface-2.html

Any suggestion or alternative?

Thanks in advance

Need help in getting ACAT to run on Windows 7

I have a friend who is suffering from ALS. I was able to get ACAT working on a Windows 8.1 laptop for her. However, I need to install it on another laptop which has Windows 7. I am able to successfully install it and run it on Windows 7. However, any gestures that I do on ACAT Vision are detected on the ACAT vision but are not sent to ACAT tryout.

Also when I exit ACAT tryout, ACAT vision doesn't exit. So I think ACAT tryout and ACAT vision are not able to communicate to each other on windows 7. But when I installed ACAT on Windows 8.1 then everything works.

@saiprasadb01 or anyone could someone please suggest on how to get it running on Windows 7.

The windows 7 laptop is 64 bit. I also tried disabling firewall on Windows 7 but it didn't help.

Cannot build installer

Although I installed InstallShield, I cannot build the installer of the ACATSetup.sln project
There clearly is hard-coded dependency to the authors drive items (more than 200!) such as
C:\Users\sprasad1\Documents\Projects\EPL Glen Ellyn\Hawking\ACAT-1\OpenSource\Src\Applications\Setup\Extensions\Default\FunctionalAgents\AbbreviationsAgent\AbbreviationEditDeleteConfirm.xml
Any workarounds? Also, when I setup my local development environment, it looked like as if there is a dependency to an actual installation even if the presage database engine gets copied to its correct location?

Tobii Eye-Tracker and ACAT

Can one integrate a Tobii or The Eye Tribe eye-tracker into ACAT, if yes, how? Our user can no longer mover her fingers sufficiently to use them as a trigger, nor is a webcam an option due to limited facial movement.

Many thanks in advance.

Port to GNU/Linux platform request

Just this. What about a GN/Linux port?

ACATsetup.exe not found in downloaded acat-0.9.1.zip

Your ACAT page in this repository is saying you would be releasing the new version of ACAT soon is that the reason the downloaded acat-0.9.1.zip is not having ACATsetup.exe file. If that is so how soon would intel put the ACATsetup.exe in the GITHUB repository for the us to download.
Thanks & regards,
Tapan Aggarwal
Delhi India

Problem with selection of word of interest from predicted word list in Talk (Alt), Talk (ABC Alt), App (Alt) & App (ABC Alt) applications

@saiprasadb01 There seems to be a problem/bug when trying to select a word from the "predicted word list" in the Talk (Alt), Talk (ABC Alt), App (Alt) & App (ABC Alt) applications using the keyboard or vision actuator (cheek or eyebrow gesture). Instead of selecting the word of interest, the next word in the word prediction list gets selected. Please note that the alphabets from the keyboard are selected correctly.

A work-around (without changing code) is to select the word predicted previous to the word of interest, but if the word of interest is the first in the list after the undo button, one has to type the word completely, which is quite frustrating. Can anyone figure out what needs to be corrected in the source code and how to implement the change/correction in code in the above-mentioned applications?

Thank you for your help.

Word prediction engine and database

Hi, I am a master student in natural language processing. For my dissertation and case study, I focus on Presage/ACAT and test different french corpus/n-gram with presage_simulator and disabled users because I study word prediction systems. I have read the documentation (https://github.com/01org/acat/wiki/Changing-language-and-creating-new-dictionnaries) on GitHub and SourceForge (http://presage.sourceforge.net/).

What did you use as the source material (corpus) for the Presage database in the french language pack 1.0.0? Google ngram or a book? I would like to have the text, can someone give me the link or references? @saiprasadb01

Is it possible to use/implement a simulator of Keystroke Savings Rate (KSR) inside ACAT?

And the database_ptbr.db (https://github.com/01org/acat/wiki/Criando-um-novo-dicion%C3%A1rio-Pt-Br-e-Alterando-o-idioma-ACAT) is built on a recent open source book (GoT)?

Thank you!

acat using rear facing camera

I have ALS and am trying to prepare for not being able to use a mouse .
I know very little about coding . I have a surface pro 3 iCore7 128 gig with Tobii eye tracking.
I find the eye tracking unreliable and erratic so i install acat.. when I start acat vision it picks the rear camera . I have read instructions and searched for a way to switch but no luck. please direct me to where I can find an answer

No scanner - settings modification

Hi,
I encounter a very bad issue after having modified the "Scan Settings" in the "Timings" section.

First of all I noticed the scanner don't select any field of the "Timings" section. So I tryed to change one setting like "WordList Pause" (but the issue come with any change in these fields) with the mouse and just after that the yellow scanner disappeared and never come back (even when pressing the F12 or F5 key).
There is a Default button but the Scanner doesn't come back.
Even worse, it doesn't change anything if you restart ACAT and your computer. I had do delete manually "Users" directory to make the Scanner work again.

Source code?

Do you have a date for the source code? Thanks

New pre-release of ACAT v0.99.1 now available

Includes a number of new features plus bug fixes. Download it from here

• Spanish Language pack added. ACAT is now available in English, French and Spanish.
• Multi-modal gestures. Commands (ACAT shortcuts) can be mapped to gestures to allow richer interactions. For example, the eyebrow-raise gesture in ACAT Vision can be mapped to "undo the last editing change", or to "restart the scanning sequence". Mapping is done through the "Actuators" option in the ACAT Config utility.
• A new Phrases application that displays a list of configurable phrases to convert to speech. The app can be launched from the ACAT Dashboard.
• Select preferred Text-to-speech voice by voice name or by gender. Voice selection can be done through the "Text-to-speech" option in the ACAT Config utility
• A new color scheme for scanners . Color scheme can be selected from the ACAT Config utility.
• A new parameter to configure the number of iterations for typing accented letters in non-English languages. Parameter can be controlled through the "Scan" settings in the ACAT settings menu.

New release of ACAT now available

The binaries for ACAT v0.97.9.1 are now available here. A number of new features have been added including a QWERTY Alphabet scanner layout, Talk applications, support for Windows Explorer, Outlook, Media Player.
Please upgrade your current version.
Source code for this release will be uploaded soon.

click2speak - alternative application

Check this out http://www.click2speak.net/
https://www.youtube.com/watch?v=RcK9NK0l2Qs
Is it going to be better than ACAT? Or it is different? Can we mix them? ;)
click2speak got more language support, word prediction, can be used with camera, speech ecngine already done or work in progress.

Mouse Scanner issue and ACAT losing focus

I am unable to select anything along the edges of the screen due to my inability to click fast enough before the rectangle goes in the opposite direction or the line restarts scanning. Could you make it so the rectangles and lines stop for a brief amount of time at the edges?

There are times when ACAT becomes unresponsive. I think it loses focus of current application when that happens. If that is the issue, could you have ACAT focus on the desktop as a backup default?

Thank you,
Michael Huehmer

Running ACAT as a user

Hi,

Is it possible to run ACAT as a user?
I have installed ACAT as administrator. I can run acat but I cannot run as a simple user.

Thanks for your answer.

ACAT Vision : missing dll msvcp110.dll

Hi,
On one of my computer running Windows 8 64bit, Windows tells me the file msvcp.dll is missing when I'm trying to launch Vision. All the other apps (ACAT abc, Talk, Tryout...) work well.

I assume it's a problem with the Framework .NET 4.5 (but maybe I'm wrong) so I tryed to re-install it but nothing changed.

I also tried to download manually the missing dll to put it in the System32 directory but now the program tells me another dll is missing, so I think it will be endless if I keep downloading each missing dlls.

Any suggestion ?

ACAT Vision: "This app can't run on your PC"

When I try starting ACAT Vision, I get an error saying "This app can't run on your PC". The rest of the ACAT suite runs perfectly fine and works.

Windows 10 PRO (32-bit) on a Toshiba Laptop.

Chat

Is there some?

• IRC
• Gitter
• Slack
• Other

@saiprasadb01, would you use some them?

@brlima94, nós poderíamos fazer em português...

Mouse Click/IR Finger as an F12 Trigger

I downloaded ACAT a few days ago, it's amazing! Using F12 as a trigger is not practical in my prospective application, neither is the webcam trigger (the MS patient has limited facial/hand movement).

1. Is it possible to configure a mouse left and right-clicks to be an F12 trigger?
2. Is it possible to also scroll backwards, rather than scrolling through all the words to get back to the one that was missed? Left-click scrolls forwards, right-click scrolls backwards?
3. How would I configure an IR sensor connected to a finger to simulate the F12 trigger? Where could one purchase such a sensor?

New pre-release of ACAT v1.00.0 now available

New features added, bugs fixed from the previous release. ACAT for Portuguese is also included. Click here

Fatal error. Error setting word prediction engine to [Presage Word Predictor]

Installed in Win 7 x64..

No Support to PT-BR (Limitation)

HI guys
We already created the needed files but didn't receive any feedback by e-mail

So we're opening this as a limitation.
Please try to help us to support pt-br because we have several people in Brazil that can use this tool.

Thanks
Rafael
ACATResources.pt_BR.RESX.zip

Why the software history was not kept?

Hi there,

I'm a researcher studying software evolution. As part of my current research, I'm studying the implications of open-sourcing a proprietary software, for instance, if the project succeed in attracting newcomers. However, I observed that some projects, like acat, deleted their software history.

39577a1

Knowing that software history is indispensable for developers (e.g., developers need to refer to history several times a day), I would like to ask acat developers the following four brief questions:

1. Why did you decide to not keep the software history?
2. Do the core developers faced any kind of problems, when trying to refer to the old history? If so, how did they solve these problems?
3. Do the newcomers faced any kind of problems, when trying to refer to the old history? If so, how did they solve these problems?
4. How does the lack of history impacted on software evolution? Does it placed any burden in understanding and evolving the software?

Thanks in advance for your collaboration,

Gustavo Pinto, PhD
http://www.gustavopinto.org

No face detection in Vision

Hi,
I'm trying to install ACAT in another ciomputer than mine (it works perfectly on my computer) and I have this issue :
When I launch Vision, I don't have any error but the app doesn't calibrate and stay in the face detection process. Usually this process takes something like 2 sec but on this computer it's extremely slow and above all, I can't see any green squares one the cam image that should normally appears to detect faces.

Here's a screenshot :

Any help ?

sensors and trigger system

where can I get trigger system which are shown in video ?
thnx
raj

Fatal error. Error setting word prediction engine to [Presage Word Predictor]

installed on windows 10 pro

Restarting the Presage engine

Presage seems to die fairly frequently and I can't figure out how to restart the engine. If I run the installer again to uninstall and then reinstall it, it seems to start working again. Running the service tray program in the bin directory seems to put the icon in the service tray but ACAT won't connect to it. Is there a good way to manage the Presage engine?

Blink detection actuator switch

I have a family friend who could use ACAT if it can do blink detection as a "switch". Is there any possibility to add "blink" as a actuator switch.

Support for iOS devices

I understand that this is intel / microsoft project and they would like to advertise it for windows phones but most people are on either android or iOS phones.

Is there any plans to support this great project for iOS platform? If no then how can I convert it into iOS compatible code to use it in my app?

Pressing F12 selects the subsequent word, not the highlighted word

When clicking the highlighted word the software seems to delay and select the next word in the word list. This can be frustrating when one constantly has to delete and try again. Scrolling speeds between screens are on default. Is this a bug or can it be adjusted?

Enter Key freeze program due to Presage timeout

When press Enter Key multiple times in Talk window, the program freezes due the Presage timeout. Here is the log:

DEBUG:[ACAT.Extensions.Default.WordPredictors.PresageWCF][PresageWordPredictor.predict] Presage Predict Exception System.TimeoutException: La operación de solicitud enviada a net.pipe://localhost/PresageService/v1/presage no recibió una respuesta en el tiempo de espera configurado (00:01:00). El tiempo asignado a esta operación puede haber sido una parte de un tiempo de espera mayor. La razón puede ser que el servicio sigue procesando la operación o que no pudo enviar un mensaje de respuesta. Considere la posibilidad de aumentar el tiempo de espera de la operación (convirtiendo el canal/proxy en IContextChannel y estableciendo la propiedad OperationTimeout) y asegúrese de que el servicio pueda conectarse al cliente.

Server stack trace:
en System.ServiceModel.Dispatcher.DuplexChannelBinder.SyncDuplexRequest.WaitForReply(TimeSpan timeout)
en System.ServiceModel.Dispatcher.DuplexChannelBinder.Request(Message message, TimeSpan timeout)
en System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
en System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
en System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
en System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
en System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
en ACAT.Extensions.Default.WordPredictors.Presage.PresageService.Presage.predict(String previous_words, String current_word)
en ACAT.Extensions.Default.WordPredictors.Presage.PresageService.PresageClient.predict(String previous_words, String current_word)
en ACAT.Extensions.Default.WordPredictors.PresageWCF.PresageWordPredictor.predict(String prevWords, String currentWord, Boolean& success)
DEBUG:[ACAT.Extensions.Default.WordPredictors.PresageWCF][PresageWordPredictor.Predict] Prediction error. Will initialize again

When press Enter Key with mouse in alphabet scanner window, this no ocurr.

Any idea?

Thanks in advance