chillen / minecraft-gene-expression-programming-ai

GEP based evolutionary AI using Microsoft's Malmo platform, thesis project for my course on evolutionary computation and artificial life

Minecraft GEP

Hey everyone! This was my thesis project for a course I took in grad school, Evolutionary Computation and Artificial Life. For the project, I had to go off and expand on a topic that we reviewed in class and apply it to a different domain. I decided to try using gene expression programming to develop behaviour trees in isolated environments, and then treat those isolated behaviours as endpoints within new, more generalized situations.

The tests weren't overly successful, mostly limited by computing power. It would have been far easier in a simulated environment, but not nearly as cool.

Updated

• Updated since last submission; now properly parses unigenic chromosomes.
• Has stagnation penalties.
• Reduced number of terminals
• In reducing terminals, they now all mean different things depending on where they are in the operator. This follows Mwuara's solution, but it sucks for this model because it limits the blocks that can be checked, forces a number of view angles, etc. Need to find a better solution, but this one is at least proven to be capable of evolving better behaviour.
• Now evolves better behaviours; within one generation, a rudimentary wall following behaviour was found in one of the chromosomes. Tests are continuing to run to see how well it can improve in future generations.
• Updater: Added serious optimizations to runtime by causing the agent to quit if it stagnates at all and relling the world it doesn't have to recreate each time. Warning: It goes so fast, you should have at least two or three clients rocking at a time just in case, even for one agent. Or increase the delay between reporting fitness, but then that affects every single test whereas having a second client is just safer. Eerily enough, it can't find the best; the best fitness disappears. At this point it's tweaking the GEP params.

Dependencies

Just some quick info in case you didn't feel like installing the dependencies and running yourself :)

It's pretty straightforward to get running, so feel free to try it out on your own.

I've included pygep in the submission, unmodified from the original. I've also included MalmoPython.lib and MalmoPython.pyd unmodified from the Malmo github repo.

Note: After this asignment, I removed those files so I didn't step on any licensing toes, even though they're probably fine. Go pickem up if you want to run this :)

To install dependencies, check out the official malmo repo: https://github.com/Microsoft/malmo

About

Most of the about info is in missionRunner.py, the main source.

• missionRunner.py: The main driver of everything. Originally based on some Malmo tutorial code, but there's only a few lines remaining of that.
• missions.py: Simple loading code to load up the xml. Nothing special, just a tiny cleanup
• programming.py: The main parser for the sim to run. missionRunner is primarily Malmo and PyGEP setup, programming handles the actual parsing of the incoming program and sends out the commands appropriately.
• XML: All xml files are different environments for the AI to run.
• out.txt: Some output with 2 agents during 2 generations

I've also included a screenshot of it running (not overly interesting)

Issues

It took a long time to figure out a nice way to get PyGEN working with my setup, and once I did it broke my multi-client distribution setup and I need to find a new way to do that. As shown in out.txt, 2 agents for 2 generations takes over 100 seconds to run, and I'd like to run this with 50 agents over 100 generations many, many times to get the parameters tweaked nicely, adjusting fitness score, etc.

I may leave this running an incredibly long time, just to see if it can manage any better than a 2 fitness after a few dozen generations with a large population and see what happens, but I'm not entirely hopeful.

Even when this was distributed, however, there were some timing issues; agents wouldn't retry the same client more than once when trying to connect and would just quit altogether, so if one of the clients was hanging at the end of a run it wouldn't try. Additionally, it would sometimes fail to kill the mission and just sit at 0 seconds; the agent would be done, but the client remained busy and wouldn't accept connections.

Future

I'd love to get this distributed again and get it distributed right. I want to see if it can actual learn increment 0 and if it can, introduce all increments and all the original test cases and let it go, once the parameters have been tweaked to optimize with increment 0.