This project is used in the study "It is Okay to be Distracted: How Real-time Transcriptions Facilitate Online Meetings with Distractions (CHI 23)," which explored how real-time transcription can help distracted peopled during meetings. OPARTS features three different levels of real-time transcriptions; full-transcription, summary, and keyword.
The video conferencing feature is built over Mediasoup video conferencing.
- Install requirements in
summarizer/requirements.txt
(change directory tosummarizer/
then runpip3 install -r requirements.txt
. You may want to usevenv
). It takes minutes to complete. - Check if there is
summarizer/wav
directory. If not, makewav
directory. - Run summarizer, keyword, and silent detector server in the
summarizer
directory.
-
- Run
echo [PORT] | npm run summarizer
in thesummarizer
directory. It also takes some time to start listening. It will listen for summary generate request from Moderator server.
- Run
-
- Run
echo [PORT] | npm run keyword
in thesummarizer
directory. It will listen for keyword generate request from Moderator server.
- Run
-
- Run
echo [PORT] | npm run audio
in thesummarizer
directory. It will listen for silent detection request from Moderator server.
- Run
- (required) Change name of
moderator/config.example.js
intoconfig.js
, then add Microsoft Azure STT's subscription key(subKey) and service region(servReg), PORT informations in themoderator/config.js
file. - run
npm install
inmoderator
directory. - Check if there is
moderator/webm
directory. If not, makewebm
directory. - Check if there is
moderator/delays
directory. If not, makedelays
directory. - run
npm run moderator
inmoderator
directory. It will start listening for transcription requests from clients accessing through Media server.
- make
logs
folder inmedia-server
directory. - run
npm install
inmedia-server
directory. - run
npm run media-server
inmedia-server
directory, then finally open your browser athttps://localhost:8000
or your own defined url/port inmedia-server/config.js
file. - A page is rendered requiring you to enter names for a room and your own. Enter them and click
CREATE
button. Invite
others.
(optional) Edit the media-server/config.js
file according to your needs and replace the ssl/key.pem ssl/cert.pem
certificates with your own.
There are special users who have permission to some function.
cpsAdmin
: Manage user invite, start meeting function.*-[odd number]
,*-[even number]
: Can set a default mode of each user. Full-transcript mode when odd number, and summary mode when even number.
To enable video conferencing from other than localhost
, replace announcedIP
in media-server/config.js
with your public ip address of the server and modify the port you want to serve it in.
Add firewall rules of the port of the webpage (default 8000) and the RTC connections (default UDP 10,000-10100) for the machine.
Best to run the project on a linux system as the mediasoup installation could have issues by installing on windows. At least Media server should run on a linux system. If you have a windows system consider installing WSL to be able to run it.
You may want to run the three servers in separate machines. Or at least it is better to run the summary server on another high-performance machine with GPU. If so, edit moderator/config.js
for the summary server host. The three servers are able to run on separate machines if you edit config.js
file properly.
Still, it is better to run summary server first because it takes long time to start listening.