GUI rendering does not work properly about gym-pybullet-drones

[GPrathap]

I tried one of the AirSim simulators and it also crashes immediately after it starts, I guess something wrong with graphics drivers

[JacopoPan]

I wish I had a 2080Ti to replicate the issue but it really seems more likely an OpenGL/graphics problem.
Do you have OpenGL installed? Can you try this minimal PyBullet example and see if the problem persists?
If so, try to replace line p.connect(p.GUI) with p.connect(pybullet.GUI, options="--opengl2")
If nothing changes, let me know which version of PyBullet you are using.

import pybullet as p
import pybullet_data
from time import sleep

def main():
    PYB_CLIENT = p.connect(p.GUI)
    p.setGravity(0, 0, -9.8, physicsClientId=PYB_CLIENT)
    p.setRealTimeSimulation(0, physicsClientId=PYB_CLIENT)
    p.setTimeStep(1/240, physicsClientId=PYB_CLIENT)
    p.setAdditionalSearchPath(pybullet_data.getDataPath(), physicsClientId=PYB_CLIENT)
    p.loadURDF("plane.urdf", physicsClientId=PYB_CLIENT)
    p.loadURDF("duck_vhacd.urdf", [0, 0, 1], physicsClientId=PYB_CLIENT)
    for _ in range(240*10):
        p.stepSimulation(physicsClientId=PYB_CLIENT)
        sleep(1/(240*10))

if __name__ == "__main__":
    main()

[GPrathap]

Thank you for the quick reply, Yep this example works fine,

Yes, I have OpenGL

jobs@gdnpc:~/sim/gym-pybullet-drones/files/videos$ glxinfo | grep "OpenGL version"
OpenGL version string: 4.6.0 NVIDIA 460.73.01

jobs@gdnpc:~/sim/gym-pybullet-drones/files/videos$ glxinfo | grep 'version'
server glx version string: 1.4
client glx version string: 1.4
GLX version: 1.4
OpenGL core profile version string: 4.6.0 NVIDIA 460.73.01
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL version string: 4.6.0 NVIDIA 460.73.01
OpenGL shading language version string: 4.60 NVIDIA
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 460.73.01
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
 GL_EXT_shader_group_vote, GL_EXT_shader_implicit_conversions, 

How can I modify the GUI client here https://github.com/utiasDSL/gym-pybullet-drones/blob/master/examples/fly.py#L123

I see it is constructed here: https://github.com/utiasDSL/gym-pybullet-drones/blob/master/gym_pybullet_drones/envs/BaseAviary.py#L180 yet not clear how to override this

I have installed pybullet 3.1.6

[GPrathap]

I modified (https://github.com/utiasDSL/gym-pybullet-drones/blob/master/gym_pybullet_drones/envs/BaseAviary.py#L180 replace p.connect(p.GUI) with p.connect(p.GUI, options="--opengl2") ). However, still having the same problem (:

jobs@gdnpc:~/sim/gym-pybullet-drones$ /usr/bin/python3 /home/jobs/sim/gym-pybullet-drones/examples/fly.py
pybullet build time: Apr 25 2021 07:44:46
[INFO] BaseAviary.init() loaded parameters from the drone's .urdf:
[INFO] m 0.027000, L 0.039700,
[INFO] ixx 0.000014, iyy 0.000014, izz 0.000022,
[INFO] kf 0.000000, km 0.000000,
[INFO] t2w 2.250000, max_speed_kmh 30.000000,
[INFO] gnd_eff_coeff 11.368590, prop_radius 0.023135,
[INFO] drag_xy_coeff 0.000001, drag_z_coeff 0.000001,
[INFO] dw_coeff_1 2267.180000, dw_coeff_2 0.160000, dw_coeff_3 -0.110000
argv[0]=--opengl2
startThreads creating 1 threads.
starting thread 0
started thread 0
argc=3
argv[0] = --unused
argv[1] = --opengl2
argv[2] = --start_demo_name=Physics Server
ExampleBrowserThreadFunc started
X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
4

    visual 0x24 selected

GL_VENDOR=NVIDIA Corporation
GL_RENDERER=GeForce RTX 2080 Ti/PCIe/SSE2
GL_VERSION=4.6.0 NVIDIA 460.73.01
GL_SHADING_LANGUAGE_VERSION=4.60 NVIDIA
pthread_getconcurrency()=0
b3Printf: Selected demo: Physics Server
startThreads creating 1 threads.
starting thread 0
started thread 0
MotionThreadFunc thread started
viewMatrix (-0.8660253882408142, -0.2499999701976776, 0.4330126941204071, 0.0, 0.0, 0.8660253286361694, 0.4999999701976776, 0.0, -0.4999999701976776, 0.4330126643180847, -0.75, 0.0, -0.0, 5.960464477539063e-08, -2.999999761581421, 1.0)
projectionMatrix (1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, -1.0000200271606445, -1.0, 0.0, 0.0, -0.02000020071864128, 0.0)
/home/geesara/.local/lib/python3.6/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))

[GPrathap]

I tried on inside a docker image, it works fine, I guess something wrong with my graphics, if you have time will have look. If not it's fine, I will continue with docker image. Thanks for the help

[JacopoPan]

I am not sure, the following 2 snippets are where all the PyBullet GUI options are set, but they essentially do the same things that the minimal example above does too. You could try to comment out as many of the p.methodName(..) lines to see if any one of the them is the root of your issue.

gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py

Lines 178 to 197 in 7bec934

	if self.GUI:
	#### With debug GUI ########################################
	self.CLIENT = p.connect(p.GUI) # p.connect(p.GUI, options="--opengl2")
	for i in [p.COV_ENABLE_RGB_BUFFER_PREVIEW, p.COV_ENABLE_DEPTH_BUFFER_PREVIEW, p.COV_ENABLE_SEGMENTATION_MARK_PREVIEW]:
	p.configureDebugVisualizer(i, 0, physicsClientId=self.CLIENT)
	p.resetDebugVisualizerCamera(cameraDistance=3,
	cameraYaw=-30,
	cameraPitch=-30,
	cameraTargetPosition=[0, 0, 0],
	physicsClientId=self.CLIENT
	)
	ret = p.getDebugVisualizerCamera(physicsClientId=self.CLIENT)
	print("viewMatrix", ret[2])
	print("projectionMatrix", ret[3])
	if self.USER_DEBUG:
	#### Add input sliders to the GUI ##########################
	self.SLIDERS = -1*np.ones(4)
	for i in range(4):
	self.SLIDERS[i] = p.addUserDebugParameter("Propeller "+str(i)+" RPM", 0, self.MAX_RPM, self.HOVER_RPM, physicsClientId=self.CLIENT)
	self.INPUT_SWITCH = p.addUserDebugParameter("Use GUI RPM", 9999, -1, 0, physicsClientId=self.CLIENT)

gym-pybullet-drones/gym_pybullet_drones/envs/BaseAviary.py

Lines 483 to 499 in 7bec934

	#### Load ground plane, drone and obstacles models #########
	self.PLANE_ID = p.loadURDF("plane.urdf", physicsClientId=self.CLIENT)
	self.DRONE_IDS = np.array([p.loadURDF(os.path.dirname(os.path.abspath(__file__))+"/../assets/"+self.URDF,
	self.INIT_XYZS[i,:],
	p.getQuaternionFromEuler(self.INIT_RPYS[i,:]),
	physicsClientId=self.CLIENT
	) for i in range(self.NUM_DRONES)])
	for i in range(self.NUM_DRONES):
	#### Show the frame of reference of the drone, note that ###
	#### It severly slows down the GUI #########################
	if self.GUI and self.USER_DEBUG:
	self._showDroneLocalAxes(i)
	#### Disable collisions between drones' and the ground plane
	#### E.g., to start a drone at [0,0,0] #####################
	# p.setCollisionFilterPair(bodyUniqueIdA=self.PLANE_ID, bodyUniqueIdB=self.DRONE_IDS[i], linkIndexA=-1, linkIndexB=-1, enableCollision=0, physicsClientId=self.CLIENT)
	if self.OBSTACLES:
	self._addObstacles()