rainbow-mind-machine / rainbow-mind-machine

🌈 The extensible framework for Twitter bot flocks. https://pages.charlesreid1.com/rainbow-mind-machine

rainbow-mind-machine

rainbow mind machine is an extensible framework for running Twitter bot flocks in Python.

rainbow mind machine uses the mind machine framework to provide the user with a simple set of building blocks for building bot flocks: a Keymaker, a Shepherd, and a Sheep.

The philosophy is to help users get started quickly, and build on simple behaviors to create complex, rich bot behaviors.

See rainbow-mind-machine in action!

All the shields

Links

rainbow mind machine (rmm) links:

• rmm source code on Github: https://github.com/rainbow-mind-machine/rainbow-mind-machine
• rmm documentation: https://pages.charlesreid1.com/rainbow-mind-machine/
• rainbow mind machine organization on Github: https://github.com/rainbow-mind-machine

releases:

• rmm releases on Github: https://github.com/rainbow-mind-machine/rainbow-mind-machine/releases
• rmm on pypi: https://pypi.org/project/rainbowmindmachine/
• rmm on dockerhub: https://hub.docker.com/r/rainbowmindmachine/rainbowmindmachine/

rainbow-mind-machine's Issues

Write SocialSheep documentation

add documentation for the SocialSheep class

• example of how to create
• example of how to extend/redefine
• toilet
• burd hurd
• favorite
• parameters
• sleep
• what is expected, what is automatic

make it... really weird

In case anybody thought they had any idea what was happening, dispel that notion. Things are about to get more weird.

Quotes from Michel Foucault's Security, Territory, Population: Lectures at the Collége de France:

The theme of the king, god, or chief as the shepherd (berger) of men, who are like his flock, is frequently found throughout the Mediterranean East. It is found in Egypt, Assyria, Mesopotamia, and above all, of course, in the Hebrews...

On his coronation, for example, the Pharaoh receives the insignia of hte shepherd. The shepherd's crook is placed in his hands and he is declared the shepherd of men. The title of shepherd (pâtre) or pastor (pasteur) of men, is one of the royal titles for the Babylonian monarchs...

"Oh Ra who keeps watch when all men sleep, who seeks what is good for your flock..."

Also, what you are about to read is the best.

quote.

ever.

The shepherd's (pasteur) power manifests itself, therefore, in a duty, a task to be undertaken, so that - and I think this is also an important characteristic of pastoral power - the form it takes is not first of all the striking display of strength and superiority. Pastoral power initially manifests itself in its zeal, devotion, and endless application. What is the shepherd (berger)? Is he someone whose strength strikes men's eyes, like the sovereigns or gods, like the Greek gods, who essentially appear in their splendor? Not at all. The shepherd is someone who keeps watch. He "keeps watch" in the sense, of course, of keeping an eye out for possible evils, but above all in the sense of vigilance with regard to any possible misfortune.

SocialSheep base class

The SocialSheep implements functionality to interact with other Twitter users.

Our SocialSheep class will use tweets as an entrypoint into the twitterverse.

So, to begin, the SocialSheep needs a way to get a pool of tweets.

• The SocialSheep defines a call_plumber() method that creates a Toilet (#10)
• The Toilet provides a small pool of tweets
• The Toilet is a private member of the SocialSheep class

Once a SocialSheep has a Toilet, it should be able to do the following:

• favorite tweets in the toilet (subject to terms and conditions)
• retweet tweets in the toilet (subject to terms and conditions)
• extract a list of users, or BurdHurd (#12), from the toilet
• flush the toilet. aaaahahahahahaha, ha ha, ha, hahaha, flush the toilet, get it?

With a BurdHurd (#12) the SocialSheep should be able to:

• follow each user
• add each user to a twitter list

Other functionality should be put off, for now. For example, if we are now creating lists, we want to be able to leverage lists, instead of BurdHurd. So it would be better to create a method to tweet at each user in a given list, rather than a method to tweet at each user in a BurdHurd.

Exception handling

Start implementing our own exceptions.

This is one way of solving the problem of how to run bots in "production mode" - change how they handle exceptions.

• Shepherd can set the flock in "debug mode" or "production mode"
• If a Sheep is in debug mode, it dies on an exception

Mailing list

Something ought to be made of that...

Write boring-mind-machine tests

Also see #28 make boring-mind-machine usable

• BoringKeymaker tests
• BoringOAuthKeymaker tests
• BoringShepherd tests
• BoringSheep tests

Branching workflow

Implement a gitflow-type branching workflow, but replace their "develop" with our "master" and replace their "master" with our "stable":

our master (their develop) is the main branch for development work

• feature branches come off of the develop branch, and are merged back in

our stable (their master) is the main branch for stable versions of things to go

• rare case: critical bugfixes go on a hotfix branch, are applied directly to stable

our releases/v1.0 (same as theirs) contains all files for the v1.0 release

• commits on this branch are tagged v1.0.0, v1.0.1, v1.0.2, etc.

Document the data model (what data is passed where by whom and when)

We have a couple of loose ends flailing about that make this library too confusing:

• Keymaker stores Sheep key info in a json file
• That JSON file is passed along to the sheep
• We also have parameters we pass in to the Sheep (from the Shepherd)

Maybe a diagram to clarify all of this? This is related to the confusing mix of self.params and json_params and &c.

Problem with figuring out what's going on

I retweet at a SocialSheep bot flock.

It just sits there doing nothing.

Sleep interval should be 60 seconds.

Also it keeps uselessly telling me it's following the same people over and over.

Also it does not seem to notify us of things that actually happen.

Why does this always seem to be the case? Logging rarely seems to work for me.

Write SocialSheep tests

Now that #9 and #10 are finished, it's time to write a simple test suite and a simple example.

make embarcadero mind machine usable

setup.py to install it

ensure we are properly importing boring mind machine and extending its classes

keymaker strategy is clear

write keymaker to be cranked out, and i mean cranked out

shepherd

sheep

integrating bot flocks with other bot flocks

A somewhat mind bending curveball...

If we have a Slack bot flock, and we have a Twitter bot flock, and we have a Github bot flock... how do we tie all of these together?

Side question: our Shepherd is creating the same kind of Sheep each time, but should we be building it to create all kinds of different Sheep? (For example, one Sheep will do issues, another Sheep will do pull rquests etc.)

Side answer: No, we want the Shepherd to create the same kind of Sheep each time. We want individual Sheep to behave differently based on parameters, not based on class. (Write the Sheep to fit your flock.)

Strategy: an integrated flock. Slack bot flock and Sheep bot flock both start up, but now they need to communicate, so each sheep has various communications channels.

calendar bot flock

do we even have a mind machine that will interface with the google calendar API?

what would we call it?

Better tests

Tests were not catching some issues. Most surprising: _key_is_valid() instead of self._key_is_valid(). Also some issues with the photo bot, crashing when raising exceptions.

Note 1: need to remove that raise Exception from PhotoADaySheep. see #6

Note 2: is there a better way to have a production mode that tries not to die? see #5

This raises the obvious question: how do we improve our tests to catch this? The main challenge here is trying to write tests of primary functionality, but the primary functionality requires a valid key.

The problem of running different sets of tests, some requiring authentication and some not, has surely been "solved" before.

Check out the python-twitter tests directory and make some notes.

Specifying a complicated footer in mkdocs

We need to add a mailing list signup form to the bottom of the documentation. That requires a complicated, multi-line string for the footer (copyright). How to define multi-line strings in yaml?

holy cow. look at this stack overflow answer. it inspired its own website, yaml-multiline.info (link to github repo: wolfgang42/yaml-multiline).

Makefile for pypi, docker image, dockerhub, deploy docs

there are other things we'd like to do, perhaps custom docs generation with pandoc, but for now just get these tasks automated:

• deploy docs to gh-pages branch
• build docker image
• [] deploy docker image to dockerhub (happens via webhook)
• deploy package to pypi
• [] bump version (do manually)

Toilet base class

A toilet is a small pool of tweets, containing possibly useful things.

To create a Toilet, pass it a Twitter API object, and a set of parameters (TBD).

This will then use the API object to find tweets matching hashtags/search terms/users/replies into a small pool that the Sheep can do things with.

When you're all done with the tweets, you flush your Toilet and get a fresh batch.

Used by SocialSheep (see #9).

Kill the Lumberjack. Log to stdout/stderr

no more logging module. this logging module is a pain. it duplicates output, it doesn't work right, it's not intuitive. just kill it.

OAuth Keymaker abstraction for boring mind machine

User needs to specify a token and a secret

Making this part of the constructor, rather than part of the method to get API keys, is much smarter

Now we have a generic OAuth base class that must be created with a token/secret label

Overarching philosophy

overarching philosophy of rainbow mind machine is to keep things dead simple.

don't be afraid to try things out.

start here: make a test account. use it to learn how to use rmm. when things ok, move on to the flocks.

examples to support this:

• novice
• technician
• general
• extra - einstein sheep + frankenstein shepherd. sheep.insert_brain()

more notes on examples:

• each example can have requirements.txt
• need to be able to get a twitter flock up and running with echo/fake tweets like *SNAP*

Improve examples in documentation

Why improve the examples? Too much documentation can make a user sick. Not enough documentation can make a user angry. To keep them moderated, use examples, and offload the details to the examples. Just document the big picture.

usability examples

To improve usability examples, we basically just want to convert tests to examples, but use more complicated interlinking and better names than "test"

extensibility examples

See #3

travis ci? circle ci? tox? nose?

Welp... check out banana-octo-py.

make boring mind machine installable and usable

import boringmindmachine as bmm

Neener, how do i make a bot, neener neener

teacher I hafta make a bot where is the code that i copy and paste to do that

all the shields!

Add shields.io shields.

Like... a lot of them.

perform surgery to remove watergate.txt

perform a git-commit-ectomy

watergate.txt in the examples dir

they're just examples, they only need 10 lines, not 30 mb

Illustrate git workflow

Illustrate git branching workflow using latex

improve the current Keymaker OAuth disaster

Wow, plumbing the depths of the Keymaker again, having implemented several other APIs, is helping me understand what went wrong and how to fix it. Note that #20 (boilerplate mind machine) will help with this.

Basically:

• Every API service worth anything provides an OAuth authentication system.
• The OAuth can work in online mode (webapps) or offline mode (command line clients); we insist on offline mode.
• Offline doesn't really mean offline, the app still needs to request a login token from the third-party service. It just means the third-party service can't reach the app.
• The Keymaker performs the offline OAuth authorization with the user and gets an OAuth credential which it places in a key file. Those are the magic keys the Shepherd and Sheep use.

For other services, we want to have a similar flow/framework. Desirable features:

• No web server required if possible plz ktnx (if you really have to, maybe oauth-dance would work)
• Everybody use same framework, and NOT oauth2client

There are a loooot of examples, and it's extremely confusing how they all interrelate.

• requests-oauthlib
• oauth2client examples (we're not using oauth2client)
• flask-dance github api example
• pseudocode for any http handling lib
• requests.post

This one makes the most sense:

• requests-oauthlib

Here is a nice example that finally makes some sense:

• requests-oauthlib github example

Release process

• readme and docs/index
• badge versions
• setup.py description
• updating tag commit
• chicken and the egg problem w multiple libs, dependencies on one another

Prime number versioning system

Need to:

• Write up what the prime number versioning system is
• Write a script to bump the version

start a hashtag

#releasethemindmachines

Apollo bots

Now have Apollo 14 - 17

Also have both lunar and surface journals

More complex behavior? Or use both? Or randomly pick one or the other?

Update bot page

Release 23

executive summary: stable release, emm and rmm only

prepare for changes:

• branch for pre-release on all 3 (bmm, emm, rmm) from dev
• update mkdocs

major changes:

• move keymakers to respective libraries
  • rainbow mind machine
  • embarcadero mind machine
• logging, quick integration with existing
  • bmm:
    • keymaker
    • sheep
    • shepherd
  • rmm:
    • keymaker
    • sheep
    • shepherd
  • emm:
    • keymaker
    • sheep
    • shepherd
• exception handling and information

test procedure:

• split up
• test out with test scripts
• clone on blackbeard
• stop container currently running on blackbeard
• start a branch in pod-bots
• run containers on blackbeard

when tests pass:

• if containers ok, stop pod-bots
• set up service
• test service

integration and release:

• on maya, integrate branch into release for rmm

new bot who dis:

• update apollo and integrate changes into pod-bots
• on maya, integrate changes into release of pod-bots
• on blackbeard, check out release of pod-bots
• run forever

bot maintenance:

• write and run follow flock script
• update flock lists
• update flock pages

Extensibility: examples

Now that we're doing a deep clean of each class, we have documented a few ideas/situations where you would want to extend the Keymaker or the Shepherd. Now we need to turn these into examples.

Link the examples/ dir with the documentation.

May need to spit documentation for each class into two parts:

• Keymaker
• Extending the Keymaker
• etc.

Bot maintenance

bot maintenance:

• write and run follow flock script
• update flock lists
• update flock pages

training video: toilet training for your social sheep

a video covering social sheep and how they work

Improve/integrate tests and examples

most of the functionality requires interactivity or secrets, and should not be done in a nose test suite anyway.

tests/ should still contain smoke tests, tests of offline functionality. But in general tests should be more like examples that you verify are working. manual testing. most appropriate test setup for rmm. Makes it possible to get 100% coverage.

Deep clean: Sheep class

We've gone through the deep clean of the Keymaker and (mostly) the Shepherd, now it's time to do a deep clean of the Sheep.

Remove exception from PhotoADaySheep

here

Move Keymakers to respective libraries

exactly what it says

dictionary of parameters vs keyword args

Noticed today in the Sheep class that our use of the dispatcher pattern implies that all parameters will take the form of a dictionary of parameters:

    def perform_action(self,action,extra_params):
        # Dispatcher
        if hasattr(self, action):
            method = getattr(self, action)
            method(extra_params)

extra_params is a single item, and therefore must be a dictionary. Instead we should do:

    def perform_action(self,action,**kwargs):
        # Dispatcher
        if hasattr(self, action):
            method = getattr(self, action)
            method(**kwargs)

so that method can take keyword args. Everybody needs to use kwargs. Don't pass dictionaries around.

training video: using a bot master and bot account, create app and two-bot twitter bot flock

training video:

• start with two (or three) twitter accounts
• create an app using the bot master account
• create a two-bot twitter bot flock

brief overview of the process before it starts, then run the process so each step is familiar.

BurdHurd class

If we need a class to represent a list of twitter users, we can use a BurdHurd.

This is a super simple, barebones wrapper around a Python list of users.

Plumber base class

A Plumber installs (creates) a Toilet.

To create a Plumber:

• Give it hashtags, terms, users, mentions
• Timeframe?
• The Plumber then makes a Toilet (small pool of tweets)

Phone class

private class belonging to a new type of sheep. phone sheep. (dunno, make an issue for that too.)

phone class will use named channels on a given host/port (assumes everyone is coordinating correctly - see store for details).

might even belong in boring mind machine.

Fix logging

use these instructions: https://charlesreid1.com/wiki/Logging

if they don't work, fix them.

pypi markdown description

https://dustingram.com/articles/2018/03/16/markdown-descriptions-on-pypi

training video: POOP with rainbow mind machine

Python Object Oriented Programming

"When Should I POOP?" A guide to deciding when the complexity of object oriented programming is right for your project.

"How Do I POOP?" Get started with your first class.

"Can I Use Someone Else's POOP?" Extensibility is one of rainbow mind machine's biggest selling points. In this tutorial we teach you about extending existing classes.

boilerplate-mind-machine

boring-mind-machine

For boring (boilerplate) mind machine libraries

BoringKeymaker, BoringSheep, BoringShepherd (all non-functional)

Other infrastructure common to all mind machine components

Documentation fixes

Generic mind machine documentation section is ok

communicate those changes and images to the rest of the rainbow mind machine documentation

ditto for embarcadero mind machine documentation