mbaske / robot-ants

Robot Ants Demo made with Unity Machine Learning Agents

Robot Ants

This is a reinforcement learning demo made with Unity Machine Learning Agents.
It was insprired by ants following pheromone trails. The robot ants have an energy level which decreases over time. As long as its energy is above zero, each ant leaves a slowly evaporating trail, with individual trail point strengths matching the current energy level. Ants have to balance their behaviour between walking large distances and recharging their batteries at power-ups, spread randomly across the terrain.
The ants are controlled by two reinforcement learning agents that were trained sequentially with PPO:
The lower-tier agent ("walker") controls leg rotations and observes the ant's physics state as well as given a target direction. Training the policy was aided by imitation learning, with demonstrations generated from an oscillator driven heuristic. The agent is rewarded for its velocity in the target direction and penalized for facing away from it.
The upper-tier agent ("searcher") observes the ant's energy level and surroundings (trails, power-ups, other ants' energies) using a grid sensor. The policy's single output value serves as the target direction for the "walker" agent. The agent is rewarded for its current energy level. A waypoint queue stores a given number of agent positions over time. The squared distance between the first and last queued positions is divided by the maximum squared distance the agent can travel, the resulting ratio serves as an additional reward.

Ant Design: Ergin3D (modified) via turbosquid.

robot-ants's Issues

Python error when training

I'm really loving all your demos for ML-Agents and I'm going through all of them right now. I can't seem to train the ants though, since I'm getting the following error every time after running
mlagents-learn config/trainer_config.yaml --run-id=anttest1 --train

I've tried both your project with ML-Agents version 0.6 and the one from the pull request that's updated to the latest. Inference works for your .bytes files (the PR ones don't seem to be trained). But as soon as I'm trying to learn the model myself this happens after a few steps. I have not changed any settings and I'm definitely using your config file.

Any idea what the problem could be?

can you update this project to work with ml-agents 0.10+?

this is a nice demo, but when I try to run it with ml-agents 0.11, there is no brain loaded.