cucapra / approx-vision

Designing image processing hardware and software for computer vision

The Approximate Vision Project

This is the public release of code developed for the paper "Reconfiguring the Imaging Pipeline for Computer Vision" by Mark Buckler, Suren Jayasuriya, and Adrian Sampson. It contains the Configurable & Reversible Imaging Pipeline (CRIP) described in the paper, documentation on how to run and edit the CRIP for your own use, and both Dockerfiles and instructions for how to run our supported computer vision benchmarks.

License and citation

All code in this git repository is released under the MIT license. If you use this code in your research, please our ICCV 2017 paper:

@inproceedings{buckler-iccv2017,
    author = {Mark Buckler and Suren Jayasuriya and Adrian Sampson},
    title = {Reconfiguring the Imaging Pipeline for Computer Vision},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    year = {2017},
}

Compiling, running, and general usage

All available documentation for this code can be found in this GitHub repo's Wiki. Those of you who just want to run a simple example or are just getting started will find our FAQ particularly helpful.

Learning and using other camera models

Our pipeline is an augmented version of this reversible imaging pipeline. The page that used to host additional models as well as the code needed to learn new models is here but unfortunately the site appears to be down. This is possibly because the PI on the project Michael S. Brown has moved universities. If you contact them you may gain access to the original code, but we don't have access.

Contributors

• Mark Buckler ([email protected])
• Suren Jayasuriya ([email protected])
• Adrian Sampson ([email protected])
• Taehoon Lee ([email protected])

approx-vision's Issues

Have no permission to access camera model

The link for camera model
http://www.comp.nus.edu.sg/~brown/radiometric_calibration/
seems not open to public.

Can you give another link for camera model?

models

Is it possible to find other camera models online? Other than the nikon one you provide?

not having docker

Hi,
I don't have docker. Instead, I have singularity (I am not root on the machine). I tried to look at the singularity document to run the docker image files but I couldn't. Do you have any advice on how to run the docker image with singularity?
Thanks

Linker errors when running tests

Running make version=5 in pipelines/common_test (which is invoked by the test script on @taehoonl's branch) leads to these errors:

g++ pipeline_V5.cpp PipelineUtil.cpp ../common/pipe_stages.cpp ../common/LoadCamModel.cpp ../common/MatrixOps.cpp -g -I ~/packages/Halide/include/ -I ~/packages/Halide/tools -L ~/packages/Halide/bin -lHalide `libpng-config --cflags --ldflags` `pkg-config opencv --cflags --libs` -lpthread -ldl -lm -o pipeline_V5.o -std=c++11
/tmp/cc3UGN9n.o:(.data+0x0): multiple definition of `demosaiced_image'
/tmp/ccIWvHoH.o:(.data+0x0): first defined here
/tmp/cc3UGN9n.o: In function `operator halide_type_t':
/root/packages/Halide/include/Halide.h:1353: multiple definition of `jpg_image'
/tmp/ccIWvHoH.o:/root/packages/Halide/include/Halide.h:3184: first defined here
/tmp/cc3UGN9n.o: In function `operator halide_type_t':
/root/packages/Halide/include/Halide.h:1363: multiple definition of `cam_model_path'
/tmp/ccIWvHoH.o:/root/packages/Halide/include/Halide.h:3184: first defined here
/tmp/cc3UGN9n.o: In function `operator halide_type_t':
/root/packages/Halide/include/Halide.h:1363: multiple definition of `wb_index'
/tmp/ccIWvHoH.o:/root/packages/Halide/include/Halide.h:3184: first defined here
collect2: error: ld returned 1 exit status
Makefile:7: recipe for target 'all' failed
make: *** [all] Error 1

Those are linker errors, indicating that the same symbols are defined (not just declared!) in multiple C++ files. The confusing bit is that the linker seems to be referring (erroneously) to Halide.h—I don't know why it's doing that, but the error comes from somewhere else.

Specifically, it looks like jpg_image is defined here in ImgPipeConfig.h. And that header file is included both in pipeline_V5.cpp and in PipelineUtil.cpp (indirectly via PipelineUtil.h). That means both .o files have a string named jpg_image, which creates the conflict.

In general, the problem is that there's something actually defined in a header file. Headers should usually only contain declarations, like this:

char jpg_image[];

While the implementation (.cpp) file should contain the actual definition:

char jpg_image[] = "temp.png";

Or, if you want these things to work like constants, you can use static to give them internal linkage:

static char jpg_image[] = "temp.png";

That way, each compiled .o gets its own copy of jpg_image that isn't exposed to the others. That's morally equivalent, although not exactly the same, as using macros:

#define jpg_image = "temp.png";

(Statics are nice for constants cuz they don't have all the same weird pitfalls as #define macros. They have types, for example.)

I hope that helps!

Fail with an error for some images

For instance:

root@167545c2c5e4:/approx-vision/pipelines/common# ./pipeline_V2.o /datasets/000000431848.png /datasets/ 
Error at ./pipeline_V2.cpp:153:
Input buffer b0 is accessed at 2, which is beyond the max (-1) in dimension 2
Aborted (core dumped)

There should be check for number of channels while conversion to RAW format. Raise en error if gray-scale image is passed

Can't figure out a script to feed my own image directory

I have a directory full of 346 images in /approx-image/datasets folder that I created. I wanted to write a script file to have the first (reverse) pipeline to iterate over every image and reverse it but I can't seem to figure out the way it's done in other datasets. how can I do that please?

How to generate parameters for different number of control points in gamut mapping

Hi,

I noticed that the default number of control points for the gamut mapping stage is set to 3702, which maybe a bit too large for me to test. Is there any way to generate the control points, weights, coefficients for gamut mapping?

MOD