Cannot connect to X server about gym-pybullet-drones

[JacopoPan]

Hi @alexvaca0

I suppose this happens when creating an env with gui=True at this line?

gym-pybullet-drones/examples/learn.py

Line 78 in 333ec90

env = TakeoffAviary(gui=True,

I don't think you can start PyBullet's GUI from Colab, you should be able to avoid the error by setting gui=False

If you want to implement rendering in Colab you can look at this example exploiting p.getCameraImage and IPython.display

Camera images are actually saved in BaseAviary on this line (when gui == False and record == True)

gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py

Line 301 in 333ec90

[w, h, rgb, dep, seg] = p.getCameraImage(width=self.VID_WIDTH,

[avacaondata]

Hi @JacopoPan , thanks a lot for the quick response, you're being really helpful here :)

Yes, I'm leaving gui=True at that line, my intention was to watch the video dynamically as it's rendered from the env. Okay so first thing I'll do is set that argument to False.
I'm checking your example and it seems understandable, therefore instead of using the PyBullet's GUI, what I'd need to do is load the environment and my agent in the notebook, and then run some episodes, saving the images in the environment object, and accessing them to save all the frames with which I can make the video, right?

[JacopoPan]

Happy to (try to) help @alexvaca0,

admittedly, the current implementation is not intended for graphical display/rendering with Colab.

What I do in BaseAviary is using p.getCameraImage to save frames when gui == False and record == True
These images are taken at self.CAPTURE_FREQ and saved under self.IMG_PATH

gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py

Line 310 in 333ec90

(Image.fromarray(np.reshape(rgb, (h, w, 4)), 'RGBA')).save(self.IMG_PATH+"frame_"+str(self.FRAME_NUM)+".png")

What I'm suggesting you could do is to append these frames in an array, make a video, and show it in Colab as in the example linked above (with the caveat that you are rendering it after the experiment, not during it)

print("creating animated png, please about 5 seconds")
%time write_apng("example6.png", frames, delay=100)
%time Image(filename="example6.png")