nickswalker / arduinorl Goto Github PK

I think you mean to just write 'gamma' here.

Helpful feedback on the report. Need to do a better job communicating the representation.

The final policy in the video seems like it always leaves the LED on. In the discussion, you say the agent lacks features that describe the interaction between the LED and joint position, but from Section 3.1, it seems like there is a separate value for each (joint1, joint2, LED) configuration.

It might have been due to the action space, which wasn't entirely clear to me either. Is an action a joint instantiation of (joint 1 movement direction, joint 2 movement direction, led activate/deactivate), or does an action just modify one of those 3 elements at each time step? The former is a much larger space to explore, which may have made it harder to learn to avoid leaving the LED on.

Memory

• Eligibility traces
• More features

Compute

• Approximate generalization, a la RBFs. Can use extra cycles to do the math. No additional features. Should work well because value is actually spatially local in this domain

Project is a mess because Arduino IDE doesn't allow subdirectories

• Atmel datasheet
• RL: An Intro

The reward formulation is in terms of time steps, but a more useful reward would be tied to the actual time taken. This would allow different representations to be compared more fairly.