Concept inquiry about pygta5

[codesidian]

Charles does see what we see, but doesn't process it the same. It could in theory process it like us but then it'd probably take over the world. As of now, charles doesn't mess with the game. It just watches and then makes decisions of what it sees, like how we would drive in GTA. I think (i might be wrong) the plan with this project is to avoid delving into gta itself, as much as possible. That way Charles is free the play other games too with not much effort. Preferably a game with more water. I think a tesla sensor inspired setup could be implemented. Multiple camera angles. That way if charles could see what's the right, left, behind, and infront at the same time he should be able to make better decisions. However, that's also quadruple the amount of frames to be processed. I like your idea better.

[Orinion]

It's true that Teslas autopilot and Human drivers have multiple cameras and bc of that 3d vision. However when you play a videogame, you will also only have 2d vision and no problems with driving.
Also it would be hard to generate multiple pictures from gta each frame, instead have a look at this

[Sentdex]

Yeah, I am undecided on this and haven't seen any research to suggest that we see the pseudo 3D in games any different than AI would.

In real life, to actually render distance, you do need 2 cameras, but actually a single camera via some algorithms can do a decent job of general distance detection. Further, some models have even been trained JUST for this purpose (detecting distance in images) and they perform quite well. I really would be surprised if the convnet in this case doesn't have some sort of crude depth perception modeled. It clearly doesn't swerve to miss things far away, it waits til they're closer. Maybe it only memorized specific sizes, no idea.

Anyway, I am going to close this, feel free to re-share updates you find on this topic though.