mozilla / testdaybot

Mozilla QA Test Day IRC Bot

Test Day Bot

This is an IRC bot that supports Mozilla QA Test Days.

Installation

Run npm install to install the bot and dependencies.

Configuration

Settings are provided via environment variables:

• IRC_HOST - host of IRC server to connect to
• IRC_PORT - port of IRC server to connect to
• IRC_NICKNAME - nickname to use on IRC
• IRC_PASSWORD - NickServ registered password
• IRC_CHANNELS - comma separated list of IRC channels to join
• IRC_SECURE - use a secure connection to IRC (default: true)
• IRC_AUTO_REJOIN - auto rejoin if disconnected from IRC (default: true)
• ADMINS - comma separated IRC nicks of default admins
• HELPERS - comma separated IRC nicks of default helpers
• AD_CHANNELS - comma separated IRC channels to advertise in
• AD_MESSAGE - message to use when advertising the test day

Running

Run node bot.js to start the bot.

Usage

Send ':help' via a direct message to the bot to receive a list of commands.

testdaybot's Issues

Ability to log, gather metrics data

I would like to have the ability to retrieve logs of activity from testday bot. For example, it would be useful if I could see the number of unique testers who signed on that day.

Update node-irc to version 0.3.7

The code here looks like v.0.3.3.

0.3.5 was released by the original maintainer 2013-01-01. https://github.com/martynsmith/node-irc/releases

Current node-irc 0.3.x branch shows 0.3.7 in package.json

A possible replacement, mentioned in node-irc issues conversations: https://github.com/oftn/oftn-bot

Run testdaybot for the next Test Day on 2015-01-23

@ashughes1 as far as I can remember the next testday is this Friday. I would like to have the testday bot running through the day. Please let me know about details so we can get it setup together.

I will get the bot running on mozqa.com under my personal account. If all is working fine we can make it a service on that host.

Firebot bugs are not counted

The stats do not show any bug as mentioned by firebot in the testday channel. The format is of the following:

https://bugzil.la/1120673 — FIXED, %email% — Fx 35 installer crashes on XP x86 SP3 at the end (creating shortcuts) if the xp firewall service is

handle Test Day helpers per Test Day

No default helpers should be set. Test Day helpers should get set for each test day, starting with default Test Day admins.
PR #26

pull the channel topic message into the welcome note

it'd be awesome if testbot could dynamically pull the channel topic into the welcome message. Admin moderators can update the channel topic, and when they do, new users that join would see the new note.

A lot of times, the topic includes the link to the testday announcement. also, if the etherpad link ever changed during the day, testbot would pick that up.

bot has to update channel topic when testday starts and ends

The topic for the testday as set with the :next command should be reflected in the channel during the testday. So the bot has to update the topic if permissions exist. Also it has to restore the old topic once the testday is over.

Keep in mind that the bot will have OP in the #qa channel.

Testdaybot should be able to be queried for help

I should be able to issue the Testdaybot a 'help' command and be treated to a manual of commands.

Be able to ask Testdaybot who is available to help

As a testday participant, I should be able to easily identify who is available to help me get started.

• I should be able to query the bot and get back a list of helpers
• the bot should be able to ping mentors on my behalf without prompting
• I should be notified of who to ask for help when I join the event

Move config.json to config-example.json

Announce in the channel when the test gets started or ends

Right now the bot is only updating the topic of the channel. But it would be good to send a simple message to the channel itself to announce that the testday has been started or ends. It should show the topic and the link to the etherpad like:

The %topic% testday starts now. For details see %etherpad%
[..]
The %topic% testday ended. Thank you all for your participation!

@ashughes1, would this wording be fine with you.

Configuration is not always saved

I get a ping from @FlorinMezei this morning regarding his admin privileges. Those were gone again since we restarted the bot yesterday. I have had a look into the code and have seen that the configuration for at least admins is not saved when the list is changing. So if the bot is restarted by the system daemon due to updates the configuration is gone.

So for all the data stored in the config file and we are able to change inside the bot, a saveData() call has to happen.

:next command should print start and end times in UTC

Currently the times are shown in PST:

Next Test Day's start is Thu Jan 22 2015 23:00:00 GMT-0800 (PST)
Next Test Day's end is Fri Jan 23 2015 17:00:00 GMT-0800 (PST)

Lets use UTC as format given that this is used everywhere else.

Produce CSV data to be put into spreadsheet

Have data processed like below

Event Testers Greeted Bugs Triaged <2 Messages 2-5 Messages >5 Messages 0 Messages

Add a metric for most active times of the day

The general idea is to count the number of messages posted in the channel per hour. This way I can determine the most "active" times.

Thanks

add to :help response with a single message in channel noting PM response

It's easy for people monitoring lots of channels to miss the PM :help output.

Advertise an active event across multiple channels

The testdaybot should be able to advertise an active event across multiple IRC channels. This should be done at regular intervals or when queried by the event organizer.

Log active participants

Before we were putting out greeted people in the stats. Given that we do not greet people anymore, we should change that to active people. That means we should log who was active during a testday.

@ashughes1 I think that is something you absolutely wants to have for the first testday?

add user verification for each admin command

probably best to write a function to check for admin nick and logged in user.
client.whois crashes the bot if the supplied nick is not currently available on IRC
updates to the irc library are available and may address this crash, but adding test to individual admin commands on execution would be more secure and less likely to cause the crash
PR #26

Identify itself against nickserv for a registered nick

The testdaybot should identify itself on connection to nickserv. That way we can let him get ops automatically via chanserv. @galgeek, is that easy to fix?

Setup a new testday

An admin needs to be able to setup a new testday via the bot. We need:

• date and time range via UTC
• Topic
• Etherpad
• Set Test day organizer and Anthony as helpers

Setting testday topic needs a prefix welcome message

Right now we have the following line:

client.send('TOPIC', testDay.channel, testDay.topic);

It should be changed so it fulfills the following:

• A welcome text
• It includes the link to the etherpad + some prefix text

@ashughes1 can you please give us the wording you would like to have here? @galgeek, this is an issue we also have to fix before Friday.

Enhancement: Post a message to channel shortly after a bug is filed with the days whiteboard

Sometimes in #firefox, Firebot will post messages to channel of recently filed bugs. Not everyone comes to channel during the testday but they will file bugs using the whiteboard for the day. It would be interesting to get the bot to post recently filed bugs to channel for everyone to see.

:stats command should send content as a single message

:stats is sending a dozen of messages which is kinda annoying. Instead please construct the content on the bot side, and send a single message to the user.

store Test Day start and end and etherpad with other metrics

Start and End should be saved with metrics, as well as the etherpad.

:helper add should support addition of multiple helpers per command

Currently only one nick can be added per command.

add :QMO and :SUMO details to the html site!

This is to match Aaron's patch

Put small lag on channel entry greeting

The bot greets people as soon as they enter and it's a bit freaky.

Put a second or two random lag on it to make it a bit less aggressive :)

update main listener to use switch / else if for bot commands

The main listener should use a switch or if / else if block for public bot commands, like :helpers, :schedule, and :optout.

See conversation in PR #46 and handling of admin commands in bot.js.

Testdaybot should be able to provide schedule information

The testdaybot should be able to provide the following schedule information:

• schedule of events/activities for the current day
• schedule of events/activities for the next event

update ReadMe

ReadMe still refers to greeting new users

Investigate emulating functionality of other IRC bots

There are a few different bots on Mozilla's IRC channels which provide functionality we may want to emulate in the Testdaybot. This issue should track cataloging the feature sets of the various bots to see if there's anything we can poach to supplement Testdaybot's feature-set.

Two bots that come to mind are the Fennecbot and WebQAbot.

Improve output of stats command

Right now the entries are kinda unordered and it is hard to find the appropriate information. I would propose we output the information ordered the following way:

• Topic
• Etherpad
• Start
• End
• OptoutTotal
• Active participants
• Active hours

:stats should sort active people from most active to least active

From issue #52:

When running ":stats" sort the active people by "most active" to "least active" - currently randomly listed.

Testdaybot should linkify bugs when mentioned

I can't remember if the bot has this feature already, but I wanted to file it just in case. When someone mentions a bug number in channel, it should be linkified.

Improve time handling in :next command

From issue #52:

1. When setting the ":next" testday make "time" errors more explicit:
   a) start time in the past
   b) start time after end time
   Currently we get the same generic error that time is invalid and have to hunt down where the issue is.

@FlorinMezei it would be good if you can give some examples here. Thanks.

:help command should print help for all privileged commands for admins

Currently :help does not print help for admins for the helpers-also command :advertise.

Be smarter about detecting someone who should be greeted

The bot should greet under the following conditions:

• if the person has joined for the first time
• if the person has not been greeted by someone else
• if the person has rejoined after a certain time away

Add config.json to .gitignore

On mozqa.com the newly created service resets the local checkout of the git repo to ensure that we can successfully apply the latest changes. By doing that we also remove the config.json file. So lets put this file into .gitignore to make it persistent.

:stats command should show timezone in its output

The times as output by the :stat command are in UTC but we do not show that anywhere at the moment:

<_TestDayBot> The following hours were active in the channel:
<_TestDayBot> 21: 1
<_TestDayBot> 22: 1

Lets add this info to the description.

Testdaybot should provide information on joining the community

The testdaybot should be able to provide a person information on how to join the community. This information should be offered to anyone who has contributed during the event (if they aren't already a Mozillian). This information should also be queryable so I can ask the testdaybot how to join.

Don't re-welcome on re-join

The bot does not discriminate against people joining for the first time, people rejoining after being away, or people automatically signing back in after timeout disconnect. The bot should be smarter.

add :help so that people know how to use the bot!

Allow operators of the testday channel to set the etherpad url via an IRC command

As of now you have to restart the bot with an updated etherpad URL to set the new topic. We should simply allow that for any operator in the #testday channel to update it via an irc command like ´:etherpad url´.

Not spam welcome messages on days that arent test days

How would the bot know that there isnt a test day on because currently test day topics are left until the next test day. I like this so will split it into a different bug.

This is from comment #6 (comment) in #6

Investigate data collection for outreach and metrics purposes

The testdaybot should be able to collect certain information for the purposes of reaching out to event attendees and to generate metrics so we can make improvements to the program.

Some initial ideas include:

• IRC names
• Bugs
• Tests
• Tasks
• Check-ins

A couple of requirements:

• This information should only be queryable by event organizers.
• This information should be opt-in for event participants
• A participant should be able to query the bot to scrub data collected on them
• A participant should be able to opt-out of data collection for their activity
• All data should be stored privately or in some way that is not publicly accessible
• No data should be kept indefinitely - we'll need to determine a reasonable time limit

save and reload test day data across bot restarts

The bot does not currently save its test day schedule to reload after a bot restart. Instead, a previously schedule test day must be rescheduled.

We should save a test day schedule, and probably some other details like updated helpers, and reload when the bot restarts.

Please move repository to the Mozilla account

David, it would be great if you could move the whole repository to the official Mozilla account. That way we could more easily maintain it and enhance the features of the bot.

I hope it will work for you. CC'ing @davehunt and @ashughes1.

testdaybot should tell users which data is being collected prior to asking them to opt out

Steps:

Joining #qa on a Test Day

Result:
testdaybot PMs users the following:

11:19 <_TestDayBot> Welcome to today’s Test Day, alex_mayorga!
11:19 <_TestDayBot> To opt out of data collection, use the command :optout.

Expected results:
testdaybot PMs users the following:

11:19 <_TestDayBot> Welcome to today’s Test Day, alex_mayorga!
11:19 <_TestDayBot> We collect data as described at https://allizom.org/some/path/data-collected-by-testdaybot.html
11:19 <_TestDayBot> To opt out of data collection, use the command :optout.

refine commands and add :rmAdmin and :rmHelper

Hello @whimboo and @ashughes1!

Anthony provided some quick feedback on his testing today, including...

• refine the :etherpad command, so that it's smarter about what it says, like :schedule
• refine the :addAdmin/:addHelper commands, so that they don't add duplicate nicks
• refine the :optout command, so that it adds advice on opting back in
• refine the :advertise command, so that it provides feedback on what it does
• add :rmAdmin and :rmHelper commands