mitchwadair / mthebot

An easy-to-use chatbot for your Twitch channel

Welcome!

My name is Mitch and I am a software engineer from North Carolina.

What do I do?

Check out some of my cool projects!

TESjs

TESjs is a module for Node.js that streamlines usage of Twitch's EventSub system.

MtheBot_

MtheBot_ is a chatbot for Twitch which aims to make it easy for streamers to have an interactive experience with their viewers.

mthebot's Issues

Timers Not Working at Certain Intervals

Whenever a timer has an interval that is greater than 5 minutes, a chat which meets the message threshold requirements for the timer, but has been inactive for more than 5 minutes will not have the timer trigger. This has to do with the bot's session pool being 5 minutes, meaning that the channel gets removed from the chat session pool before the timer goes off.

Upgrade TESjs to Version 0.7.x+

TESjs has now been updated to support external auth handling in v0.7.0, which MtheBot would greatly benefit from. I intended to include this upgrade as part of #87 but forgot. Should be as easy as getting the initial auth token on startup and passing that into the TESjs config, along with getNewAppAccessToken

Switch to Async/Await Syntax

I think there are quite a few areas where async/await syntax will clean up the code quite a bit. It should make it more maintainable and more readable to use this instead of .then and .catch blocks etc.

Prime example:

mthebot/API Server/publicAPIs/auth.js

Lines 10 to 62 in 9d67f56

	const post = (actions, sessionPool, req, res) => {
	const code = req.body ? req.body.code : null;
	const redirectURI = process.env.NODE_ENV == 'development' ? 'http://localhost:8081/auth' : 'https://bot.mtheb.tv/auth';
	if (code) {
	httpsRequest(
	`https://id.twitch.tv/oauth2/token?client_id=${process.env.CLIENT_ID}&client_secret=${process.env.CLIENT_SECRET}&code=${code}&grant_type=authorization_code&redirect_uri=${redirectURI}`,
	{method: 'POST'}
	).then(r => {
	let headers = {
	'Client-ID': process.env.CLIENT_ID,
	'Authorization': `Bearer ${r.access_token}`
	}
	httpsRequest(`https://api.twitch.tv/helix/users`, {headers: headers, method: 'GET'}).then(user => {
	const createSession = _ => {
	actions.refreshChannelData(user.data[0].id);
	let sessionId = crypto.randomBytes(20).toString('hex').slice(0, 20);
	while (sessionPool[sessionId]) {
	sessionId = crypto.randomBytes(20).toString('hex').slice(0, 20);
	}
	sessionPool[sessionId] = {
	channel_id: user.data[0].id,
	timeout: setTimeout(_ => {
	delete sessionPool[sessionId]
	}, r.expires_in * 1000)
	}
	res.status(200).json({user_data: user.data[0], session_token: sessionId});
	}
	DBService.getChannel(user.data[0].id).then(data => {
	if (data) {
	DBService.updateTokensForChannel(user.data[0].id, r.access_token, r.refresh_token).then(_ => {
	createSession();
	}).catch(err => {
	res.status(500).send(encodeURIComponent(err.toString()));
	})
	} else {
	DBService.initChannel(user.data[0].id, user.data[0].login, r.access_token, r.refresh_token).then(_ => {
	createSession();
	}).catch(err => {
	res.status(500).send(encodeURIComponent(err.toString()));
	});
	}
	}).catch(err => {
	res.status(500).send(encodeURIComponent(err.toString()));
	});
	}).catch(err => {
	res.status(500).send(encodeURIComponent(err.toString()));
	});
	}).catch(err => {
	res.status(500).send(encodeURIComponent(err.toString()));
	});
	} else
	res.status(400).send('Missing authentication code in request body');
	}

See how lines 51-59 are all just duplicate code (there are more instances elsewhere too)? That could all be solved with one try/catch block instead. And there would be less indenting.

Update DB Library

Currently MtheBot uses the mysql nodejs library but there is a newer, better, library: mysql2

Create API Server Tests

Create a small set of tests that are intended to test the functionality that is contained within the API server and not on individual endpoints

Provide API Doc

Because this bot provides a custom API server, we need to document the APIs it provides and how they are used.

Default Follow Event Datatag is Wrong

The follow event in defaultEvents has {{username}} instead of {{user}} in its message

Users Should be Indexed by UID Rather than User/Display Name

Currently users are indexed in the DB by their username. This could cause issues if a user ever changed their username on Twitch. Because of this, users should be indexed by their user ID numbers instead, which will stay consistent throughout username changes.

Implement Unit/API Testing

In order for MtheBot_ to remain scale-able and stable there should be tests implemented to ensure the bot works correctly.

Use Prepared Statements in DBService

Right now, DBService is using placeholder strings and the query function, but this does not use prepared statements. It should instead use execute which will prepare and run the query. This should help further prevent SQL injection attacks.

Will also need to update the initChannel queries, as they are not using placeholders either, which is very dangerous (even though no user input is being used here).

mthebot/dbservice.js

Lines 48 to 54 in ae6596c

	let query = `INSERT INTO channels (id, name, token, refresh_token, enabled) VALUES (${id}, "${name}", AES_ENCRYPT("${authToken}", "${process.env.CLIENT_SECRET}"), AES_ENCRYPT("${refreshToken}", "${process.env.CLIENT_SECRET}"), false);`;
	let eventsQuery = `INSERT INTO events (channel_id, name, message, enabled) VALUES`;
	Object.keys(defaultEvents).forEach((k, i) => {
	eventsQuery = `${eventsQuery} (${id}, "${k}", "${defaultEvents[k].message}", ${
	defaultEvents[k].enabled
	})${i === Object.keys(defaultEvents).length - 1 ? ";" : ","}`;
	});

To do a prepared statement for this dynamic data, this issue comment should help.

Add Follow Event

Using TESjs, add support for a 'follow' event similar to an event like 'subscribe' etc.

Default Commands

The bot must support data tags for default commands.

Example

{{commands}} would present a list of available commands for a channel

Use mysql2 Promises in DBService

One of the nice things about mysql2 is that is has support for promises. Using these instead of creating my own promises and resolving from the callbacks should improve code readability and reduce code length some.

Command Arguments

The bot currently separates command chats into arguments, there should be support for additional functionality for these arguments.

Example

!myCommand arg would be split into a command myCommand with argument arg and data tags for myCommand would be handled with arg in mind.

Create Table Schema Template

In order for people that want to set up their own instance of MtheBot or want to do any local development of MtheBot to easily get up and running, I should create a template sql script file to create a database/tables. This way people do not have to look into the code and reverse-engineer the schema and stuff to set it up

Additional Channel Point Reward Actions

Include different channel point reward actions.

• say something
• chat alerts
  • for something like a shoutout, etc

Improve MtheBot_ API

Currently the API is pretty near-sighted and brute-forced. Any update to data is done with a POST request of the entire collection of data. We should be able to use an UPDATE call to update single items. In addition, improve DB queries for DELETE to use JSON modification rather than 2 queries to get and set.

Support PubSub for Channel Points

Channel points present a unique opportunity for chat to have engaging messages or alerts that are triggered by a redemption of a channel point reward. We should add support for subscribing to PubSub channel points topics.

Advanced Data Tags for Channel Data

In order to support simple command creation for advanced concepts, we must include data tags that correspond with ways to fetch certain data. Providing as many data tags as possible by default is ideal. This issue will cover the beginning of these advanced data tags by including channel related data from the Twitch API

Example

{{uptime}} would be replaced with data from an API fetching the uptime data for the channel the command was executed from

Add YouTube API Support

Users should be able to connect to their YouTube account and have datatags like {{latestvideo}} {{ytchannel}} etc.

External Data API Improvement

Currently building a request for each API. Why not make one method for that, return all data then process it method-by-method rather than building the whole request for each method.

Create Contact API Test

Test the functionality of the contact API. This will be a little tricky because we'll need to mock nodemailer

Twitch Subcount Datatag not Working

With recent updates to Twitch auth, can no longer use client_credentials to get sub count data. Update token auth/refreshing

Encoding Contact Endpoint Inputs Not Clean

Currently to sanitize input from the contact API, encodeURIComponent is being used. This is causing the emails sent to not look good. This should be fixed. The input still needs to be sanitized somehow though

Make All Config Items Environment Variables (or JSON)

There are a few places where things are still hard-coded into MtheBot. Most notably the TES base URL off the top of my head. It should be ensured that everything like that should be an env var or have some kind of configuration file to source from. The config file route may have some positive advantages but could be difficult to set up for some production instances

Custom Alert Types for Point Redemptions

Provide custom alerts for users for point redemptions. This would require the bot to serve people a url with their alert which is subscribed to a point redemption topic. They could put this url into a browser source on their streaming app and an alert would show for that redemption.

Use node-fetch

Currently we're using a custom request function. This should just be replaced with node-fetch, as that is better tested and more robust

Add Timer Queue

Add a queue for timers so they do not get sent at the same time for timers with similar lengths or which overlap. This will allow timers to be staggered which reduces chat saturation of timed messages.

Links in Message Outputs are Mangled

This is probably due to the sanitization I'm doing, so we'll have to get that fixed. With the DB upgraded to use prepared statements, we actually shouldn't need to sanitize that heavily

Upgrade REST server to Express

While having a home-brew REST server is great and all, Express should make a lot of this much easier to write/maintain

Improve API design

the API probably needs a bit of a rewrite for improvement. Should also consider zod for validation and sanitize for sanitization

Include Moderator Actions for Point Redemptions

There are several moderator actions that could be allowed for point redemptions. Some safe examples include:

• Making a user VIP
• Enabling a chat mode (emote only, subcriber only, followers only) for some time

Improve MtheBot_'s Buildability/Packagability

I feel like, with this project being open-source, MtheBot_ should probably be written in a way that it can be packaged up and run locally with relatively minimal setup should people want to. Currently it is pretty near-sighted in terms of its architecture (only connecting to a remote DB, among other things).

This will likely require a lot of refactoring, but should be useful should people want to run their own instance of MtheBot_ (with their own credentials, etc)

Clean Up/Improve Code Efficiency in app.js

I've noticed a few areas where there are some inefficiencies in app.js. This task should be used to comb over it and make some improvements

Create Users API Test

Test the functionality of the users API

Automatically Add as Channel Mod When Activating Bot

With the add channel moderator endpoint now available/in beta, MtheBot should add itself as moderator to the user's channel upon enabling MtheBot for their channel. This will eliminate a manual requirement for the user to add as moderator themselves

Bump tmi.js and TESjs versions to latest

Need to bump versions for security reasons (relatively low severity). The biggest hurdle for this is needing to recreate all TESjs subscriptions. This will require a script to unsubscribe to the current ones, and re-subscribe to them again with the updated TESjs configuration.

Update Followers EventSub/API

Twitch has updated their follows API endpoint and EventSub topic, and has deprecated the previous versions. MtheBot_ needs to upgrade to use the new versions

https://discuss.dev.twitch.tv/t/follows-endpoints-and-eventsub-subscription-type-are-now-available-in-open-beta/43322

Add Creator Goals Datatag

Need to add a datatag to display active goals for the channel. This datatag would list the type and the progress.
For example, the datatag should look like:

{{goals}}

And would output something like:

followers: 24/40, subscriptions: 6/10, new subscriptions: 2/5

Auto Refresh Tokens on 401 Errors

Right now, we are validating access tokens for Twitch API calls before each request. This slows data fetching down a lot, due to a lot of unnecessary requests. Instead, we should just refresh tokens when needed and validate every hour as required by Twitch. Can take a similar approach to what I did with TESjs

Improve API authentication

The current design for API auth and sessions is very basic and definitely needs improvement. Consider not handling sessions at all, and verifying each request with Twitch instead.

For example, each API call should come in with a channel ID and an access token (currently, the session ID which expires after 5 mins of inactivity). In the new design, we should instead use the Twitch access token directly and verify that the access token matches the channel ID by verifying with Twitch.

This will create a challenge with refreshing the token after some time away from the UI, as the bot itself will refresh UATs itself. We will need to store a last known refresh token or something so the UI can properly refresh the token even after some time away.

V3

Twitch has made some major improvements to EventSub and the ability for sending chat messages through the API. These things should allow for MtheBot_ to evolve in a positive way.

This said, it will mean an almost entire rewrite of the bot

Create Auth API Test

Test the functionality of the auth API

Set Prettier Standards Across Project

Set up prettier and reformat files using it

Update Sub Count for New API

Twitch added a subscriptions total amount to the get subscriptions endpoint. Should update to this instead of calculating to increase efficiency

Create Data Service for Better DB interaction

There should be a data service type file which handles DB interactions. Should make APIs more readable

Improve API Validation

I've done this already the contact API in #67, but it should probably be done elsewhere. I already have a validation util, but using express-validator is probably better, as I can also sanitize. Sanitizing should also improve readability as I can remove a bunch of encodeURIComponent calls

Custom API Data Tag

It would be nice for a user to be able to provide an API endpoint to fetch data from if it is not included in the product by default. This would be a feature reserved for the more advanced users of the app, but should still be accessible enough that it is easy enough to understand for less-experienced people.

The usage could look something like this:

{{customapi|https://someurl|returned.data.path}}

We have two parameters to the data tag:

• Endpoint URL
  • This would have to be an open API, not needing an API key
• Path to the desired data value
  • The API would be required to return data in JSON format or plaintext
  • If the response data is plaintext, this parameter should be blank
  • For JSON responses, the path should look like a normal JSON data accessor path (keys separated by dots) and should lead to a primitive data type that can be printed as a string

Add Twitter API Support

Users should be able to connect with Twitter to have related datatags like {{latesttweet}} {{twitter}} etc