timostoff / esim_config_generator Goto Github PK

Hi T. Stoffregen, thanks for your hard work.
My question is: the bags created are videos in grayscale, is that correct? or has something gone wrong at my end, because I thought that ESIM suppose to output events, like their example: https://github.com/uzh-rpg/rpg_esim/wiki/Simulating-events-from-a-video
Second, is there a way to make color bags? We are inputting RGB images, and I was hoping that we could output an RGB bag, and then maybe use bag2vid on that.
Thanks so much,
RazRamon

What is the difference between 'image_raw' and 'image_corrupt' in the output .bag file? Which one should I choose to train my network? I mean, as the ground truth for reconstruction. @TimoStoff, could you please help me answer this question? Thank you so much for your assistance and support in this matter. I truly appreciate your help.

Hi, T. Stoffregen, first thank you for providing me the esim_config_generator which is new to the research field. I follow the guidance in the README and generate some bags using your code. But I am confused with the content and size of the generated bags. Why the size of these bags are just 4.1KB while the first bag have the size 600MB+? Are these bags dependent on the first bag in some way? Accoring to the code, it seems that all these bags are independent of each other.

No error during the execution, below is the warning output. ALL generated bags provide the same warning output.

Log file created at: 2020/10/14 18:59:25
Running on machine: ubuntu
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
W1014 18:59:25.173624 12009 signal_handler.cpp:46] Signal handler w
as called with signal 2
W1014 18:59:25.972116 12009 adaptive_sampling_benchmark_publishe
r.cpp:57] Empty benchmark folder string: will not write benchmark files
W1014 18:59:25.972132 12009 synthetic_optic_flow_publisher.cpp:41] E
mpty output folder string: will not write synthetic optic flow files

Looking forward for your rely.

Yours,sincerely.
From Alan Ng

Hi TimoStoff,

I am using your existing configs to generate bags. But when I ran python scripts/generate_preset.py /tmp, I got the following output (see the end):

I only get the first bag, but the following bags cannot be generated. But when I stop the program by ctrl+C, the code can generate the next new bag but failed the generate the following.

I guess there's something wrong with "launch.shutdown()" in generate_preset.py, which did not stop the ros node cleanly. But I am not familar with stoping ros launch by this way.

Any ideas to solve this problem? Many thanks.

(the full output)

(==> /tmp/000000000_config2d.txt')
...
publish image from callback
publish image from callback
publish image from callback
I0714 20:16:56.509268 8885 rosbag_writer.cpp:62] Finalizing the bag...
I0714 20:16:56.634008 8885 rosbag_writer.cpp:64] Finished writing to bag: /tmp/000000000_out.bag
[esim_node-1] process has finished cleanly
log file: /home/larrydong/.ros/log/e5db78ac-e497-11eb-a8f1-54b2039462bb/esim_node-1.log
[WARN] [1626265016.706444]: esim_node-1 died with code 0
[optic_flow_viz-3] killing on exit
[dvs_renderer-2] killing on exit*
(==> /tmp/000000001_config2d.txt')
Saving launch file to /tmp/esim.launch
... logging to /home/larrydong/.ros/log/e5db78ac-e497-11eb-a8f1-54b2039462bb/roslaunch-NUC-8863.log
started roslaunch server http://NUC:36919/
SUMMARY
PARAMETERS
/optic_flow_viz/arrows_scale: 0.07
/optic_flow_viz/arrows_step: 7
/optic_flow_viz/arrows_upsample_factor: 1
/optic_flow_viz/publish_rate: 100
/rosdistro: melodic
/rosversion: 1.14.10
NODES
/
dvs_renderer (dvs_renderer/dvs_renderer)
esim_node (esim_ros/esim_node)
optic_flow_viz (esim_visualization/optic_flow_converter.py)
ROS_MASTER_URI=http://localhost:11311
process[esim_node-4]: started with pid [8932]
process[dvs_renderer-5]: started with pid [8933]
process[optic_flow_viz-6]: started with pid [8934]
I0714 20:16:57.325270 8932 imp_multi_objects_2d_renderer.cpp:44] width = 256, height = 256
I0714 20:16:57.325460 8932 imp_multi_objects_2d_renderer.cpp:45] tmax = 10
I0714 20:16:57.325490 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/coco/000000425702.jpg.
I0714 20:16:57.328163 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.329393 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5a019a827ca233f48ba62757.png.
[optic_flow_viz-6] killing on exit
[dvs_renderer-5] killing on exit
I0714 20:16:57.330996 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
[esim_node-4] killing on exit
W0714 20:16:57.331701 8932 signal_handler.cpp:46] Signal handler was called with signal 2
I0714 20:16:57.332190 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Jersey-PNG-Transparent.png.
Traceback (most recent call last):
File "/home/larrydong/sim_ws/src/rpg_esim/event_camera_simulator/esim_visualization/src/py/optic_flow_converter.py", line 9, in
import rospy
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/init.py", line 47, in
from std_msgs.msg import Header
File "/opt/ros/melodic/lib/python2.7/dist-packages/std_msgs/msg/init.py", line 1, in
from ._Bool import *
File "/opt/ros/melodic/lib/python2.7/dist-packages/std_msgs/msg/_Bool.py", line 6, in
import genpy
File "/opt/ros/melodic/lib/python2.7/dist-packages/genpy/init.py", line 34, in
from . message import Message, SerializationError, DeserializationError, MessageException, struct_I
File "/opt/ros/melodic/lib/python2.7/dist-packages/genpy/message.py", line 48, in
import yaml
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/init.py", line 8, in
from loader import *
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/loader.py", line 4, in
from reader import *
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/reader.py", line 47, in
class Reader(object):
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/site-packages/yaml/reader.py", line 147, in Reader
NON_PRINTABLE = re.compile(NON_PRINTABLE)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/re.py", line 194, in compile
return _compile(pattern, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/re.py", line 249, in _compile
p = sre_compile.compile(pattern, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 576, in compile
code = _code(p, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 561, in _code
_compile(code, p.data, flags)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 107, in _compile
_compile_charset(av, flags, code, fixup, fixes)
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 232, in _compile_charset
flags & SRE_FLAG_UNICODE):
File "/home/larrydong/anaconda3/envs/py27/lib/python2.7/sre_compile.py", line 274, in _optimize_charset
r = range(av[0], av[1]+1)
KeyboardInterrupt
I0714 20:16:57.339816 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.341230 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Gabriella-Wilde-Transparent-PNG.png.
I0714 20:16:57.349253 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.351073 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5c6404aee4b8dd029ff25b55.png.
I0714 20:16:57.352191 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.352732 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Human-PNG-Free-Download.png.
I0714 20:16:57.367199 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.369637 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Secretary-PNG-Transparent.png.
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: cHRM chunk does not match sRGB
I0714 20:16:57.387706 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.391546 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Abraham-Lincoln-Transparent-Background.png.
I0714 20:16:57.401508 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.402863 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5aead1916554160a79be9faa.png.
I0714 20:16:57.403936 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.404393 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/580b585b2edbce24c47b27b2.png.
I0714 20:16:57.405547 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.406095 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/580b585b2edbce24c47b2923.png.
I0714 20:16:57.407203 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.407754 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Hand-PNG-Transparent-File.png.
I0714 20:16:57.422911 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.425623 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5954bf12deaf2c03413be360.png.
I0714 20:16:57.426824 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.427173 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Walrus-PNG-Clipart.png.
I0714 20:16:57.435050 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.435981 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Lawyer-Transparent-PNG.png.
I0714 20:16:57.438027 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.438702 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Tie-Transparent-Background.png.
I0714 20:16:57.442828 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.443619 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/Kingfisher-Background-PNG.png.
I0714 20:16:57.452098 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.454032 8932 object.cpp:12] Loading texture file from file: /home/larrydong/dataset/esim_generator/foreground_objects/5996e4a99c43bb35d25d73a5.png.
I0714 20:16:57.455160 8932 object.cpp:24] Pre-filtering the texture with median filter of size: 3.
I0714 20:16:57.482300 8932 ros_publisher.cpp:37] Initializing ROS
I0714 20:16:57.513383 8932 rosbag_writer.cpp:39] Will write to bag: /tmp/000000001_out.bag
W0714 20:16:57.513398 8932 adaptive_sampling_benchmark_publisher.cpp:57] Empty benchmark folder string: will not write benchmark files
W0714 20:16:57.513438 8932 synthetic_optic_flow_publisher.cpp:41] Empty output folder string: will not write synthetic optic flow files
I0714 20:16:57.715121 8932 rosbag_writer.cpp:62] Finalizing the bag...
I0714 20:16:57.715297 8932 rosbag_writer.cpp:64] Finished writing to bag: /tmp/000000001_out.bag

Hi @TimoStoff,
Thank you very much for this work. I have a confusion regarding the JSON files you provided in generator_config folder. There are three paths you set.

1. "foreground_images": {
   "png_dataset": {
   "path": "/home/path/to/esim_objects",
   "proportion": 0.8
   },
   "coco_dataset": {
   "path": "/home/path/to/coco_png",
   "proportion": 0.2
   },
   And then
2. "background_images": {
   "coco_dataset": {
   "path": "/home/path/to/coco"
   }
   Here I replaced "path": "/home/path/to/coco_png", this with png images that i extracted using tools/jpg_png.py script and "path": "/home/path/to/coco" with the original COCO training dataset that has jpg images.
   What is this path "path": "/home/path/to/esim_objects", about? Which folder I should specify in this path?

Please let me know this
Thanks!!