pmonks / futbot Goto Github PK

Description of Problem:

When a new match is starting soon, I want a simple reminder message to be posted to a Discord channel, so I can start viewing the match on time if I wish to do so.

Potential Solutions:

Timer-triggered logic that pulls match details from the football-data.org API match resource (/v2/matches/{id}) and posts a short summary message to the most relevant channel for that match (note: this will require a mapping from football-data.org competitions to Referee Discord's various "League Discussions" channels - see issue #8 for details).

Potential Complications:

• Messages need to be kept short and sweet, given that multiple matches may all be starting at the same time, and the risk of the bot becoming spammy as a result.
• Matches can be delayed, postponed, or canceled, and the logic needs to account for that as best it can.

It appears that there can be empty or blank entries in the referees element (e.g. the referee with id 98555 has no name), which results in odd results. Empty and blank entries should be replaced with some other value before being added to the message.

Daily schedule PDFs that span more than one page are being truncated after the first page.

Description of Problem:

When the football-data API returns a referee without a name, I want the bot to lookup up the name in a local directory (using the referee's id, as returned by the API), so I can see the referee's name instead of the default ([unnamed referee]).

Potential Solutions:

Add a map of referee-id → referee name to the config.edn, and use that map if and only if the API response is missing a name (i.e. the API always has precedence).

With the introduction of Toonbot (specifically its "scoreboard" feature), the daily schedule PDF is redundant and can be removed.

Description of Problem:

The word master has problematic connotations, and in the spirit of inclusion it should not be used in this repository.

Potential Solutions:

Rename the master branch to main.

Description of Problem:

When a large block of reminders are all sent at once, I want to be able to distinguish each one, so I can more easily visually parse those reminders and home in on the ones of interest.

Potential Solutions:

Start each reminder with the ⚽️ emoji, and bold the first line. e.g.:

⚽️ Stoke City FC vs Middlesbrough FC starts in 14 minutes.
Referees: Matt Donohue, Samuel Lewis, Darren Blunden, Darren Bond
⚽️ Birmingham City FC vs Hull City AFC starts in 14 minutes.
Referees: Dean Whitestone, Akil Howson, Geoff Russell, Thomas Bramall

Description of Problem:

When I am developing code, I want to be able to access the football-data.org API, so I can learn the API and figure out how to meet the bot's requirements.

Potential Solutions:

Register as a football-data.org developer, and obtain a token.

Description of Problem:

When a "featured" referee is officiating a match, I want there to be some form of highlight, so it's more visually apparent that one of these featured referees is assigned to the match.

Potential Solutions:

We already have a :mikedean: emoji, so perhaps a map of referee names -> emojicodes in the configuration file would be a good solution for this. e.g.

{ "Mike Dean" ":mikedean:" }

Description of Problem:

When I see an ambiguous league name (e.g. "Serie A" (Italy) vs "Série A" (Brazil)) in the daily schedule, I want to see a clear visual indicator of which country the league is played in, so that I can more easily find my league(s) of interest.

Potential Solutions:

Use the competition's "area name" (provided by football-data.org) to look up the flag emoji of that country (keeping in mind that not all competition's area names are countries: UEFA Championship = Europe, FIFA WC = World, etc.).

This table may be useful in constructing an appropriate lookup map in the code.

Description of Problem:

When I, as an admin, want to double check the operation of the bot, I want to request the bot's current runtime configuration, so I can ensure it's what I expect it to be.

Potential Solutions:

Look for !status messages from an admin (in any chat), and reply with the current configuration of the bot (with sensitive information i.e. API tokens [REDACTED]).

From @IGJoshua just now:

as a part of the new dev ToS you need to publish a privacy policy which is accessible to anyone using your bot (so maybe a privacy command which dms it to them, or a link in your presence which gives them the privacy policy).

Given the current design of the bot, this is probably best achieved via a responsive chat function e.g. !policy or !privacy or similar. It may also be worth including additional information (like a link to this GitHub repository) in that message too, for added transparency.

Discord allows a bot to declare "intents", which basically just controls which types of event will be sent by the Discord server to the bot, thereby improving performance. See this section of the discljord README for more details.

Description of Problem:

When an unexpected issue occurs with the production instance of the bot, I want to be able to change the logging level dynamically, by sending the bot a direct message, so I can triage the issue without having to redeploy the bot.

When I have triaged the issue, I want to "reset" the logging level to its configured defaults (also by sending the bot a direct message), so that the bot returns to its pre-triage logging state.

Potential Solutions:

Implement two new "secret" commands:

1. "!setlogging loglevel [logger]"
2. "!resetlogging"

See this code for how to modify logging levels.

Description of Problem:

When I'm in a chat, I may want to ask the bot what leagues / competitions it has access to on football-data.org, so I can understand what it can and can't show me.

When a match has concluded, I want to see a summary of the match (teams, league, final score, number of red and yellow cards, etc.), so I can see a consolidated result rather than having to look back through a series of potentially overlapping ToonBot "match event" notifications.

Potential Solutions:

In the match reminder job, schedule another job in 90+${reminder_duration} minutes' time to post the match summary. That job should check if the match has finished, and if not, reschedule itself to run in another max(90 - ${time_played_so_far}, ${configurable_small_duration}) minutes' time.

The message itself should be an embed, perhaps something like:

Team A vs Team B has finished. Final score: X-Y [LOGO OF LEAGUE TOP-RIGHT]
Match highlights:

Time	What	Team	Who
31'	:YC:	Team A	John Smith
37'	Goal	Team B	James Brown
63'	:YC:	Team A	John Smith
63'	:RC:	Team A	John Smith

Description of Problem:

When a match is upcoming, it would be ideal if the bot renamed an available Voice Chat (VC) channel for that match, so that the admins don't have to manage this manually.

When that match is over, it would be ideal if the bot were to rename the VC channel back to something generic (e.g. unused-1), so that the admins (and the bot, perhaps) can quickly determine which VC channels are in use vs those that aren't.

Complications

• Determining a descriptive but short name for the VC channel may be difficult to automate. It may be that we need a map of full-team-name to abbreviated name will be necessary (e.g. "Manchester United" -> "ManU", "Sheffield United F.C." -> "Sheffield"), so that the VC chat can be called something like "ManU vs Sheffield".

• There are many instances where there are more matches running concurrently than there are VC channels, so the bot would need some rule in order to determine which ones are the most important to assign to the (limited) VC channels.

• Determining when a match is over (in order to rename it back to something generic) is quite tricky - either the football-data.org match API would need to be polled (which raises the risk of quota violations), or a hardcoded time limit on matches (e.g. 2.5 hours) would need to be chosen.

Reminders for Premier League matches are showing up in #other, when they should be appearing in League Discussions > #england.

Description of Problem:

When I see something provocative or ridiculous in the Discord server, I want to ask futbot to produce a fake IST headline, so I can highlight the incident.

UI:

Me: !IST
Futbot: CUP MATCH GETS UGLY!! DIRTY TACKLES, KNEE TO THE TEMPLE, CHEAP SHOT INJURY & HEARTLESS SMACK TALK!!

Potential Solutions:

Read IST's YouTube titles via this API call, process the headlines into a Markov chain and then, upon request, use it to generate fake IST headlines.

Implementation Notes:

• This will require a YouTube API key.
• Because of the bot's limited runtime environment (Heroku hobby dyno), it would be best to pre-generate the Markov chain and persist it to disk (i.e. as an EDN file), rather than regenerate it each time on startup.
  • There is a decision to be made about whether the Markov chain EDN file gets checked into source control, or is generated each time the bot is built (including by Heroku). I'm leaning towards the former, not only for cost/performance reasons but also because it means the Youtube API call / generate Markov chain code could be a separate project (and so that code would not deployed to Heroku and would not consume runtime resources).
• The logic would be approximately the same as fake-tweets, except that for performance reasons, the Markov chain is generated in a separate batch job, rather than each time the bot starts
• One improvement over and above fake-tweets would be to explicitly include an unambiguous "end of title" marker, so that the Markov chain is aware of where titles typically end. The logic that generates a fake title would then look for the last "end of title" marker in the generated text, and truncate the message there.

Description of Problem:

When the matches for today are posted as a message, I want the option to be able to refer back to that message later on without having to scroll all over the place, so I can plan my viewing schedule in chunks throughout the day.

Potential Solutions:

Have futbot pin the daily schedule message to the channel, then unpin it before it sends the next day's schedule.

Potential Complications:

• Because one of the goals of futbot is to be stateless, it ideally shouldn't "remember" anything about the pinned message in order to unpin it the next day (having to remember anything means that if the bot crashes or is upgraded or whatever during the day, it will lose that state). It's not clear if there's a way for a bot to request all of the pinned messages in a channel, and if so whether there's enough state available to know which of those messages to unpin.

Description of Problem:

When the bot connects to a Discord server, I want to see it have the bot role, so I can manage its behaviour based on its "botedness".

Potential Solutions:

From chat: Btw the "bot" role is something specifically created by dyno or mee6, can't remember. but either way, you can't actually give it to others. i.e. somehow other bots set this role on themselves, without human intervention. Futbot should do the same thing.

football-data.org recently updated the referee data structure for matches to include the referee role (CR, AR1, AR2, 4O, VAR, etc.). In doing so, the list of referees is no longer in role-order; instead it appears to be alphabetical by name.

futbot needs to be updated to interrogate the new structure, and ensure referee names are shown in order of role (perhaps also with their role displayed).

Description of Problem:

When I am developing code, I want to be able to access the Discord APIs, so I can learn the API and figure out how to meet the bot's requirements.

Potential Solutions:

Register a new Discord application, and obtain a token for it.

Description of Problem:

When the code changes, I want to have the code automatically tested, so I can be confident the change didn't break things.

Potential Solutions:

GitHub Workflow

Description of Problem:

When a user posts an unacceptable piece of content (a problematic link, for example), I want the bot to automatically delete the problematic post and notify the user of the deletion via DM, so that the admin team don't have to manually monitor and delete such posts. The bot should also notify the admin team of the deletion, by posting a message to the #admin-commands channel and including (at least) the user whose post was deleted.

Potential Solution:

Add a configuration map of regex -> message, and have the bot check every message sent against the set of regexes. If there's a match, delete the message and send the associated message to that user via DM.

Options:

1. Post one mega-message to a single channel (e.g. #announcements or #general) per day, containing all matches on that day

2. Post a message per competition, to the most appropriate channel for that competition (e.g. EPL -> League Discussions/#england)

Comparison Matrix

*depending on how we sort entries in the table. I was assuming we'd sort them strictly by time, but we could instead sort by competition and then by time.

Example:

Description of Problem:

When configuring the bot, I want to use the user-visible Discord channel names (and not channel ids, which are quite difficult for a user to determine), so that I can rapidly reconfigure the bot.

Potential Solutions:

This will require that the bot capture and manage a channel id ↔︎ channel name map, so that lookups can be done within the code (the Discord APIs only support channel ids). This example code captures this information from Discord, caches it, and keeps that cache up to date as channels are modified via the UI. It may be worth looking at, at least until discljord adds this capability itself.

Title says it all...

Description of Problem:

When certain high-quality Youtube refereeing channels post a new video, I want to see a link to that video in a channel on the Discord server, so I can click through and watch that video at my leisure then discuss it with other server participants.

Description of Problem:

When the CNRA releases a new video quiz, I want to see a link to that video quiz posted, so I can participate in it.

Note: the message associated with the link should warn Discord users that the quiz requires PII (name and email address).

Potential Solutions:

This is basically a variant of issue #102, but the implementation will be more tricky due to the way CNRA post new quizzes (i.e. they add a link to this web page).

The monthly cadence of new quizzes may be complicated too, given that the bot is restarted at least once per day and (currently) stores no state.

Also need to add a new emoji for the CNRA logo, in keeping with how other data sources (IST, Dutch Referee Blog) are visually identified.

When a match is communicated to the community (either in the daily schedule PDF or a match reminder message), I want the emoji associated with that match's league to be used, rather than a generic national flag, so that I can distinguish between different leagues in the same country (e.g. EPL vs ECL).

Description of Problem:

When there are a lot of matches starting all at one time, the bot sends a lot of reminders all at once, which tends to be spammy. I want to mute reminders for some less interesting leagues (e.g. Championship), so I can help reduce the bot’s spamminess.

Description of Problem:

When the code changes and has been successfully tested (see #2), I want to have the code automatically built, packaged, and deployed to a hosted runtime, so I can always have the latest and greatest version of the code running.

Potential Solutions:

Deployment Machinery:

GitHub Workflow
TravisCI

Runtimes:

Heroku
OpenShift
AWS Elastic BeanStalk
...others to be evaluated based on cost and capabilities...

Tricky Parts:

Ensuring tokens (Discord and football-data.org) remain secret.

Description of Problem:

When I, an admin, notice that certain leagues are not popular, I want to be able to tell the bot to mute that league, so I can immediately reduce how "noisy" the bot is.

Conversely, when I, an admin, discover that certain muted leagues are becoming more popular, I want to be able to tell the bot to unmute that league, so that community members can be reminded of matches in that league.

UI/UX

This might look something like:

@futbot mute <league name>
@futbot unmute <league name>

sent to the bot in an admin-only channel.

Description of Problem:

When I, an admin, is unsure of what reminders are currently scheduled, I want to ask the bot, so I can find out.

Potential Solutions:

Look for !reminders messages in an admin or 1:1 (DM) chat, and reply with the list of currently scheduled reminders.

Description of Problem:

When I see a reminder message, especially in the #other channel, I want to know what league the match is in, so I can decide whether I want to watch it or not.

Potential Solutions:

Preferred design:

🇵🇹 Primeira Liga: FC Famalicão vs Portimonense SC starts in 15 minutes.
Referees: Fábio Costa, Paulo Soares, Pedro Fernandes, João Pinto, Manuel Oliveira

Steps to Reproduce:

1. Start the bot

Expected Result:

When a typical reminder is sent, it says "Team A vs Team B starts in 15 minutes" (or whatever the configurable reminder duration is).

Actual Result:

"Team A vs Team B starts in 14 minutes" (1 minute less than the configured reminder duration).

Description of Problem:

When the Dutch Referee Blog posts a new quiz, I want a link to the quiz to be posted to the #training-and-resources channel, so that the community knows a new quiz has been posted and can participate in it.

Furthermore, I want the bot to add reactions to that link, representing the question #s that folks got wrong. That way anyone can see which question(s) caused widespread trouble, and we can have a prioritised discussion about them.

Potential Solutions:

The blog offers an RSS feed which is likely to be the most computer-readable source for this information. Note however that it includes all of the Dutch Referee Blog Posts, not just quizzes, so some filtering will need to be done (e.g. looking for the word "quiz" in each post's title).

To (continue to) avoid the need for the bot to store persistent state, the bot should check the feed once a day (since it gets restarted by Heroku every day), and look for a new quiz blog post in the feed in the last 24 hours. It should only perform this processing via a timed job at a specific time each day, to ensure it doesn't post a quiz more than once.

Description of Problem:

When an upcoming match is shown to me, I want to see options for viewing that match on my local TV / cable channels, so I don't have to search around to figure out how to watch the match.

Potential Solutions:

Use football-data match information to construct a livesoccertv match URL (example), and place those links in a new column in the daily match schedule table and/or include them in the 15 minute reminder messages.

If this proves to be too difficult, just place a generic link to the livesoccertv homepage at the bottom of the daily match schedule PDF.

When the daily schedule is sent out, I want to see the two teams' head-to-head stats, so I can get a sense of how they've fared vs each other in the season to date.

Potential Solutions:

Within the PDF generation code, call the football-data "match" endpoint in order to obtain the head2head data (which is not included in the "list of matches" endpoint's response) for that match, then include that data in the generated PDF.

Note that this will case an "N+1 SELECTs" problem, which may be an issue especially on days with large schedules (the free tier can only make 10 API calls per minute). There is code present that detects and sleep/retries when the quota is exceeded however.

Description of Problem:

When the football-data API call limit is exceeded, I want the bot to retry later on (once the call limit has reset), so that calls don't just outright fail when the call limit is exceeded.

Complicating Factors:

• Call limits are defined by subscription level, so this cannot be hardcoded; the bot needs to be dynamic / responsive to whatever call limit the API is forcing upon us. This probably means that the code should respond to API call failures due to the call limit being exceeded, rather than attempting to meter itself.
• The retry logic needs to be aware of the risk of "stampeding herd" problems (e.g. if a bunch of reminders have all failed, then retry at the same time). The standard ethernet approach of adding some random delay, coupled with exponential backoff is probably a good way to avoid this.

Requirement:

When a user identifies that a conversation in a channel has gone off-topic, they want to be able to easily direct those participating in the conversation to the correct channel, to help keep channels focused.

Proposed Solution:

Have a !move #channel responsive command that the bot responds to by:

1. deleting the !move #channel message
2. posting a message in the mentioned channel, with the content Continuing the conversation from #original-channel...
3. posting a message in the original channel, with the content Moving this conversation to #channel (<link to message created in step 1>)...
4. editing the message created in step 1 to read Continuing the conversation from #original-channel (<link to message created in step 2>)...

This creates a targeted cross-linkage between the two channels.

When @BadLotGTakes posts a new tweet, I want futbot to automatically post a link to it to the #memes-and-junk channel on the Referee Discord channel, so that hilarity can ensue.

Description of Problem:

When the matches for today are posted as a message, I want to be able to read all of the information associated with all of those matches all in one place, so I can plan my viewing schedule for that day.

Context

Discord only provides the bare rudiments of formatting control, and, specifically problematic for this bot, doesn't provide any way to display tabular data. This is problematic because a table is the natural way to present a potentially long list of soccer matches, each of which has multiple data elements (date, time, competition / league, home team, away team, link to match, etc.).

Potential Solutions:

1. Post an image containing the match table.

2. Post a rich text (PDF, HTML) attachment containing the match table.

3. Post each match as a separate plain text message.

4. Consider a different design, whereby "match reminders" are posted a fixed interval (e.g. one hour) before a match, rather than posting a full "match day schedule" once per day.

Comparison Matrix

*a hack for this might be to include a QR code of each match link in the image

Description of Problem:

When it is a new day, I want the list of available soccer matches occurring that day (across all available competitions) to be posted to a Discord channel, so I can plan my viewing schedule accordingly.

Potential Solutions:

Timer-triggered logic that pulls matches from the football-data.org API matches resource (potentially using a filter to restrict to today's matches only, though that appears to be the default behaviour of the matches endpoint), formats that data into a pretty chat post, then sends that post to the (configurable) Discord server & channel.

Potential Complications:

• Discord's message formatting capabilities are (being as polite as possible...) "rudimentary", so this message will be a file attachment in a format that supports rich formatting options (specifically, tables) (see decision #7 for the full explanation of this). HTML will be the initial format of this file attachment, but if that proves to be unacceptable we'll try PDF next.

Example:

Discord embeds provide much better formatting control vs regular messages, and because most of the other bots in the referee server use them, futbot stands out as different and unusual. This should be remedied, notably for the match reminder messages, but potentially for other messages (help, privacy, video and quiz posts, etc.) as well.

ToonBot scrapes http://www.flashscore.com/ for all of its soccer information, which is both comprehensive and free. We should evaluate whether doing the same thing makes sense for futbot, especially as the higher levels of football-data.org subscription (which include leagues of interest to our community) are pretty expensive (€99 / month).

Description of Problem:

When a match is upcoming, it would be ideal if the bot renamed an available Voice Chat (VC) channel for that match, so that the admins don't have to manage this manually.

When that match is over, it would be ideal if the bot were to rename the VC channel back to something generic (e.g. unused-1), so that the admins (and the bot, perhaps) can quickly determine which VC channels are in use vs those that aren't.

Complications

• Determining a descriptive but short name for the VC channel may be difficult to automate. It may be that we need a map of full-team-name to abbreviated name will be necessary (e.g. "Manchester United" -> "ManU", "Sheffield United F.C." -> "Sheffield"), so that the VC chat can be called something like "ManU vs Sheffield". Given the massive number of team names worldwide, a fallback mechanism that attempts to reasonably shorten team names will also be needed (e.g. take the first word of the name? Take first letter of each word in the name? Take N characters from the name? Take at least N letters, then stop at the next word break or at M letters if no word break is reached?).

• There are many instances where there are more matches running concurrently than there are VC channels, so the bot would need some rule in order to determine which ones are the most important to assign to the (limited) VC channels.

• Determining when a match is over (in order to rename it back to something generic) is quite tricky - either the football-data.org match API would need to be polled (which raises the risk of quota violations), or a hardcoded time limit on matches (e.g. 2.5 hours) would need to be chosen.