alexthem8 / aiai-ai Goto Github PK

For usage on other machines, a requirements.txt would be useful

The current description for setup in the README is a bit clunky. Re-evaluate and design it as a list of ordered steps rather than a full paragraph of information.

To better facilitate usability, there can be a check for the window location each iteration. However, this may cost execution time. Further investigation is needed.

Current system is AMD GPU, Windows, Python 3.9. If this library is updated for these specs, it could be useful.

Alternatively, could look into shifting to lower Python version on Windows. Or look into Linux usage for simplicity.

Write up of process and systems used

The 0mph check terminates genomes that aren't moving for X amount of time. This is not consistently triggering, likely due to image-similarity being too low.

Investigate how low the similarity can go without creating false positives

Occasionally, the NEAT reproduction will fail after removing stagnated species. The crash happens at the following code in genome.py line 110, get_new_node_key:

    def get_new_node_key(self, node_dict):
        if self.node_indexer is None:
            self.node_indexer = count(max(list(iterkeys(node_dict))) + 1)

        new_id = next(self.node_indexer)

        assert new_id not in node_dict

        return new_id

One fix would be to loop as follows:

        if self.node_indexer is None:
            self.node_indexer = count(max(list(iterkeys(node_dict))) + 1)

        new_id = next(self.node_indexer)
        while new_id in node_dict:
            new_id = next(self.node_indexer)

        assert new_id not in node_dict

        return new_id

But this is editing the library and will not remain as a global fix. Investigate why this happens before asserting the library needs to be fixed.

To speed up the evolution process, there needs to be a feature that will time out a genome if it is no longer moving/not gaining fitness. This could be done via fitness-tracking or reading the MPH (if zero).

Additionally, this should be a flag to set depending on the level being trained. For example, some levels require the player to wait. While this could be too complex for the system, the possibility exists.

Occasionally, the next genome will register as the end-state of the previous genome instantly. Might have the game reset, take a screenshot, pause, then reset again.

Some cases exist where it seems like the goal is not properly detected. More information is required, but could be cause for retraining the model

Currently, the game screen capture is hard-coded in place, but to allow a more flexible user experience, have it find the actual location and take a screenshot that way

To facilitate an easier setup for the controller, dummy controller config functions should be made. The load_state() function already handles this, but the joystick config requires a "reset" to execute properly. See the below code example for setting UP:

controller = Controller()
sleep(0.1)
while True:
    controller.do_movement(0, 1)
    sleep(0.1)
    controller.do_movement(0, 0)
    sleep(0.1)

Occasionally, when getting a goal, the next genome is counted as also getting a goal almost instantly. This was originally thought to be a timing issue, but could be another cause.

Playback is not configured currently

Longshot, but we'll include it

Currently, the end-state check of [Goal, Time Over, Fall Out] are done sequentially in the following code block:

   if img_similarity(img, time_over, to_shape):
        g_max -= 25  # [-25, 25]
        done, info = True, 'Time Over'
    elif img_similarity(img, fall_out, fo_shape):
        g_max -= 50  # [-50, 0]
        done, info = True, 'Fall Out'
    elif img_similarity(img, goal, g_shape):
        g_max = 30 + (1.25 * (60 - (perf_counter() - st)))  # [30, 105]
        done, info = True, 'Goal'

This is an inefficient manner of resolving this issue and could be done simultaneously. Another potential boost could be packaging the Goal state check with the goal-detection (which could also be done at the same time, if not detrimental to execution time.

There's a potential that the image used to run the model is widely outdated by the time the AI acts. To mitigate this, a potential solution could be to pause after capturing an image and unpause when the action is ready. This could cause a stutter to the live-play, but could create a more accurate model.

This is for testing purposes and should not be the final outcome. Results from this experiment could lead to a better understanding of why this isn't training properly. This could ultimately be a hardware problem, potentially being solved by issue #9

Certain stages require the player to wait some amount of time. Therefore, it would should be customizable whether monkeys in a population get timed out when not moving. This would be as simple as a runtime argument.

To create a more-accurate evolution model, the theoretical max fitness should be a variable based on the given level instead of the theoretical of 105 where the goal is achieved instantly.

This could be a challenge as the "theoretical max" is the true max, while any value manually set could not be the max for the level. Maybe TASing would help with this issue.

To reduce amount of logs in the console, implement a flag based on how much should be logged.

Potential options

• full: full available logs
• none: no logging
• partial: include logs of StdOut & non-timed out runs COULD USE REEVALUATION

Note: For partial, there could be a split up of what "partial" means

Relating to issue #5, there could be an upper-limit to how close a run can get to the goal without actually crossing it. There have been cases where a monkey touches the goal tape, but does not break it. This should theoretically be the highest possible non-goal score of 50 (before penalty reductions).

Investigate what the system detects the goal-size to be when almost crossing the goal and adjust fitness function accordingly.

To better analyze the evolution process, the system should harness the stat tracker and gather data to be represented graphically.