tactile_optical_simulation's Introduction

Tactile optical simulation

Requirements

Python>=3.8

Setup

The project uses Mitsuba3 renderer(v3.3.0) for generating all the images.

For installing python dependencies

pip3 install -r requirements.txt

Quick Start

python3 generate_image.py
To choose the configuration file see files inside file_list_cfgs
To choose the render configurations see files inside render_cfgs

Expected Results


Real Sensor Image	Rendered Image

Usage

User can render a new object by providing its mesh in Wavefront .obj format and passing its name as a param to render function. new_mesh_render.py provides an example. Please refer to the script for more details.

Execute python new_mesh_render.py to render a sensor view without any object pressed against it.

Output Visualization

tev is a EXR viewer and comparison tool. You can download the latest executables from here
qt4Image is another viewer for exr images across different platform. It can also generate low-dynamic range images with gamma encoding. You can download the utility for linux from this link.

Advanced

The model files for GelSight are in Mitsuba xml format. The Mitsuba3 documentation has details on the parameters and how to set them.
The important file which are used in rendering for flatgel is or models/flatgel_with_mesh.xml

Not supported

Bidirectional Path tracing
Heightfields

Citation

@article{agarwal2020simulation,
  title={Simulation of Vision-based Tactile Sensors using Physics based Rendering},
  author={Agarwal, Arpit and Man, Tim and Yuan, Wenzhen},
  journal={arXiv preprint arXiv:2012.13184},
  year={2020}
}

tactile_optical_simulation's People

Contributors

Stargazers

Watchers

tactile_optical_simulation's Issues

Could not instantiate texture plugin of type "bitmap":

I'm getting the following error when trying to run the sample script of generate_image.py

`Traceback (most recent call last):
File "new_mesh_render.py", line 41, in scene = load_file(model_fn, **render_params, **params)

RuntimeError: \u200b[xml.cpp:1047] Error while loading "models/components/flat_elastomer_material.xml" (near line 6, col 8): could not instantiate texture plugin of type "bitmap":

[xml.cpp:1047] [FileStream] "textures/flatgel_surface_texture.png": I/O error while attempting to open file: No such file or directory
`

Any idea how to resolve this? Thanks!

Virtual Computer or Local

Hello. I was just wondering for this project, if it would be better to install everything to a local machine or a virtual environment. If the former, I am on windows, and running into trouble installing mitsuba2.

Specifically, this part:
source setpath_linux.sh

It gives me an "'source' is not recognized as an internal or external command,
operable program or batch file" error.

Thank you

cmurobotouch / tactile_optical_simulation Goto Github PK