bradenhurl / deepgtav-presil Goto Github PK

Does your plugin support multiple cameras?

Furthermore, does this run in real time. Meaning that can I pipe the output depth map from your code into another algorithm while I drive in real time

Hello, I am doing computer vision research, in which I want to create a simulation of people walking around the street. To be more specific, can we get:

• The head pose (yaw, pitch, roll) of the person (compared to camera view)
• The normalized position (x,y coordinate) of the person
  If we can get that, where should i start?

Could you tell me how to get the stereo images and depth image?
Thanks a lot

Thanks for this great dataset.
I’ve downloaded the dataset and I have 2 questions:

1. I try to match the Entity_ID label (from label_aug directory) with the color code in the segImage/instSegImage .png files. Is there any relation/equation between these 2 numbers?

2. GT_Pose - Is there any recorded information about the egomotion. I.e. the relative 6DOF transformation between 2 consecutive camera positions?

Any answer/hint/code-reference would be helpful.

How can I install the GTA game with an old version? as the README says Make sure GTAV is on version 1.0.1180.2 or below.

Hi!
Thanks for sharing your work. It's super interesting! I have been tinkering with it over the past month to better understand it. I'm wondering why you record secondary perspectives from different vehicles on the road? I can only think of 2 things:

1. To somehow improve the bi-linear interpolation from the depth maps from different perspectives? Do you use these secondary perspectives to improve the final point cloud collected?

2. To collect more data in a single scene, so that we have more samples without moving the cars around or continuing the game. If this is the case, do you see any big problems with this approach?

Thanks for clearing this up in advance!

Hi,
Thanks for sharing your work! WHile trying to run the code, I'm facing the following issue (error from Client side) once i run python dataset:

$ python dataset
Trying to connect to DeepGTAV
Successfully connected to DeepGTAV
Connecting to client...
Message sent to client to begin.
Traceback (most recent call last):
  File "dataset", line 188, in <module>
    message = client.recvMessage()
  File "C:\Users\account\Downloads\DeepGTAV-PreSIL\VPilot\deepgtav\client.py", line 52, in recvMessage
    frame = self._recvall()
  File "C:\Users\account\Downloads\DeepGTAV-PreSIL\VPilot\deepgtav\client.py", line 66, in _recvall
    packet = self.s.recv(4 - len(data))
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

Once I run python dataset, the game freezes and is forcefully closed. The deepgta.log file shows:

Initializing Winsock...Initialized.
Socket created.
Bind done.
Listening...

The last log entry I can get from the code by inserting my own log comments is in the void Server::checkClient() function.

Any help would be much appreciated!

Error messages :

PS D:\work\DeepGTAV-PreSIL-master\DeepGTAV-PreSIL-master\VPilot> python .\dataset
Trying to connect to DeepGTAV
Successfully connected to DeepGTAV
datasetPath : D:\data\object\dataset.pz
Connecting to client...
Message sent to client to begin.
dict_keys(['vehicles', 'peds', 'reward', 'throttle', 'brake', 'steering', 'speed', 'yawRate', 'location', 'time', 'frame'])
received message !
Traceback (most recent call last):
File ".\dataset", line 201, in
index = message['index']
KeyError: 'index'

Hi,

I want to extract the occlusion level of a target inside a bounding box, say a person partially occluded by a tree. I found the following info from the code. But from the message obtained through the python script, this occluded filed is not Integer as said, but a value between 0 and 1. Is it correct? if yes, how can I translate from a value 0 to 1 to the following documented values?

1 occluded Integer (0,1,2,3) indicating occlusion state:
0 = fully visible, 1 = partly occluded
2 = largely occluded, 3 = unknown.

Another question, I found the following code, is this function used to calculate the occluded filed above? If yes, how can I pass the value of e->occlusion to the message passed to python script.

Thank you in advance!

Hello,
I am wondering if in this repo the file "DeepGTAV.asi" shouldn't be presented inside Release folder.
Because I am not seeing this mode being loaded by asiloader and I cannot connect via TCP while the game is running.

Thanks!

Hi, I used your publised data but find several problems. Listed below:

1. The provided depth map looks like disparity map.
2. Instance Map around Frame 3056 is incorrect.
3. Insufficient description in provided code from github repo PreSIL-tools "https://github.com/bradenhurl/PreSIL-tools" . In this repo, you provided python file "vis_presil.py" which seems to do the conversion between depth and raw depth data, however, it is not. The descrpition is insufficient and misguiding.

It is a great work but with some improvment it will be recognized and utilized better, thanks!

I just bought GTA5 for this, but I can't figure out how to downgrade to this version

In Readme:
To download pre-generated data see: https://uwaterloo.ca/waterloo-intelligent-systems-engineering-lab/presil-dataset-downloader
The link doesn't work anymore.:(
Could I get access to your dataset somewhere?
Thanks!:)

Hello Braden @bradenhurl.
I am extending your projetct to include multiple cameras,
however, I came to a problem where when the segmentations and detected objects are being exported, it doesn't take into account the camera rotation and it breaks the exports and detection boxes - I guess it just takes the collector car direction since it consideres only one camera facing directly on the hood. Can you please point me into direction what should I look into when fix this issue and take the actual rotation from the camera?
Thank you so much in advance!

Hi! Thanks for sharing the codes!
My subject is to collect a semantic segmentation dataset from a pedestrian perspective, not a car.
But I don’t have any basics about game rendering.
I want to know if I can expand this code to get a complete semantic segmentation map, including buildings, roads, and sidewalks, etc.
If the above information can be obtained from the game rendering, can you tell me what to do next?
Thanks!

Hi all,

In dataset.py I get the error below.

Trying to connect to DeepGTAV
ERROR: Failed to connect to DeepGTAV
Connecting to client...
ERROR: Failed to send message. Reason: [WinError 10057]

Any help would be appreciated.

Hello,
when I'm in the game, and send Start command to the server to begin collecting data,
the game freezes for about 45 seconds, and than it teleports the player to the spawned and begin.
Is this long delay normal? Is it possible to speed it up?

Also the capturing one frame (depth+stencil+annotations) takes around 6 seconds.
Is this a normal behaviour? Are there any possibilities to speed it up? I know reducing the game resolution helps a bit, but I guess it is not possible to collect the data close to the real-time?
Maybe using RAM cache for storing buffers and than save to HDD using another cpu thread?

Thanks,
Jan

Hi,
Thanks a lot for the code it's been helping me a lot with my work! I just noticed something in the data I collected using your code. If the ground/road is tilted at an angle, the bounding boxes have a offset from the real object. The farther that object is, the greater this offset. I have attached an image below that shows this. Is this something that you noticed as well? What might be causing this? Is there a way to fix it?

My version of a game is 1.0.2245.0. Does it mean I can't use your code to do my own implementations?

Hello, I left an inquiry after reading your paper and have a question. When I download the data attached to the reference, the label format is shown as follows, and I wonder which part here means the radial velocity.

Hello!
Is it possilble for you to host the paths.xml in some file host provider different from the access protected google drive it resides now?
I've tried to download it and asked permission from the prompt but as of today (a week after) I wasn't able to download it yet

Thank You!

Hello, I'm using your dataset(PSIL), It's a great work. I find depth files are organized in bin file, and the value is between in 0 and about 0.2, furthermore, I found it is more likely disparity, i.e.,value is going to be lower with the distance growing. Can you help me fix this problem? Thank you very much!

Hi!
I noticed that there are some file missing in the repository (DeepGTAV.asi, DeepGTAV.iobj, DeepGTAV.ipdb) and i built the project with visual studio after solving some issue with dependecies. The results are those files with other files .obj. I copied all the files in the GTAV directory but i noticed that the DeepGTAV files were not loading. I checked asiloader log file and in fact DeepGTAV.asi failed to load.

// GTA V ASI LOADER (build May  2 2015)
//     (C) Alexander Blade 2015
LIB: "C:\Windows\system32\dinput8.dll" => 00007FFFAF290000
API: "DirectInput8Create" => 00007FFFAF292230
API: "DllCanUnloadNow" => 00007FFFAF2A5220
API: "DllGetClassObject" => 00007FFFAF2A5330
API: "DllRegisterServer" => 00007FFFAF2B4740
API: "DllUnregisterServer" => 00007FFFAF2B49C0
LOADER: Loading *.asi plugins
ASI: Loading "C:\Program Files\Rockstar Games\Grand Theft Auto V\DeepGTAV.asi"
     "DeepGTAV.asi" failed to load
ASI: Loading "C:\Program Files\Rockstar Games\Grand Theft Auto V\GTAVLauncherBypass.asi"
     "GTAVLauncherBypass.asi" => 00007FFFC8B50000
ASI: Loading "C:\Program Files\Rockstar Games\Grand Theft Auto V\NativeTrainer.asi"
     "NativeTrainer.asi" => 00007FFFC6C70000
LOADER: Finished loading *.asi plugins

The problem is that if i use DeepGTAV.asi from the original DeepGTAV repository it works but not with the one i built.

There don't seem to be any problems with dependecies or missing libraries path.

Any help would be much appreciated!

Hi! Thanks for sharing the codes!
Semantic segmentation task on 3D point clouds is attracting more and more researchers nowadays. However, the annotation of real world point clouds is expensive. The Semantic-KITTI dataset is the only one with semantic point cloud sequences. I'm curious of if this tool could get the a synthetic dataset like semantic-kitti. Thanks for giving hints to me! Discussions are welcomed!

I'm getting this error when I execute the "python dataset" command in VPilot. I've downgraded GTA V to recommended version (1.0.1180.2) and I have also tried the latest version (1.0.1868.0). I run this command go to the game and try to resume the game (get out of pause menu) and suddenly the game freezes and this error comes up.

Is there any workaround or solutions? Thank you very much.

Dear authors, thank you for such great work! I really like that you generate LiDAR data, which is definitely useful for my current work on dense mapping quality.

In the paper, you stated that currently only 90 degrees FoV is supported, is there a new version with 360 FoV support? If not, what are the limitations or is there an easy way to get full view point cloud?