project-10 / dgm Goto Github PK

Hi,

Given pairwise potentials for a dense CRF (DCRF) I want to know if it is available to train bilateral and smoothing kernel parameters. I've checked the "Demo Dense" but as far as I understand the training learns unary node potentials. In "Demo Train", the edge trainer learns edge potentials between any two nodes of a bidirectional graph. The for loops to add edge vectors in Demo Train process only nodes (x,y),(x,y-1),(x-1,y), where x is the row index and y is the height index of an image/matrix. However, in a dense graph all nodes (pixels) are connected to each other. Also, in Demo Train the x and y indices start from 1. Then, the loops skip some edges such as those between the nodes in the first row indexed by x=0, e.g., (0,0) and (1,0). Briefly, I could not understand the adjacency for the class "CGraphPairwiseExt". However, I think I should write such a loop to add edge vectors between a selected pixel and the others to train potentials for a DCRF. Can you please help me to understand how to use training DGM module for DCRFs?

Thank you!

In CTrainNodeBayes, the 2D PDF special case is handled in addFeatureVec(const cv::Mat &, byte), but it is not taken into account in calculateNodePotentials(const cv::Mat &, cv::Mat &, cv::Mat &).

CTrainNodeBayes::getPDF2D(byte) is called nowhere.

The CTrainNodeCvRF::save() / CTrainNodeCvRF::load() seems to not working.
When loading the training data (saved before with CTrainNodeCvRF::save() function) with the CTrainNodeCvRF::load() function, the random forest appears to be empty and the first call of the CTrainNodeCvRF::getNodePotentials() function causes an exception.

However, it is possible to successfully load the training data, which was saved by the DGM v.1.5.0.

In order to reproduce, please insert into the Demo Train the code:
nodeTrainer->save("D:\\");
nodeTrainer->reset();
nodeTrainer->load("D:\\");

after the line
nodeTrainer->train();

Thank you for your outstanding work on the library.

This not a bug report, just a question.

The training example shows the case when only one image is used for training: http://research.project-10.de/dgmdoc/a01846.html

Is there possibility to train the dense model using several (or many) training images with ground truth. Similar to what we do in training neural network. I have checked the original code DenseCRF, and several derivative projects - no such functionality included. Is it possible as all?

With Ubuntu 18.04, OpenCV 3.3

CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: In function `main':
Demo FEX.cpp:(.text+0xf8): undefined reference to `DirectGraphicalModels::fex::CCoordinate::get(cv::Mat const&, DirectGraphicalModels::fex::coordinateType)'
Demo FEX.cpp:(.text+0x231): undefined reference to `DirectGraphicalModels::fex::CCommonFeatureExtractor::invert() const'
CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: In function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getIntensity(cv::Scalar_<double>) const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor12getIntensityEN2cv7Scalar_IdEE[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor12getIntensityEN2cv7Scalar_IdEE]+0x66): undefined reference to `DirectGraphicalModels::fex::CIntensity::get(cv::Mat const&, cv::Scalar_<double>)'
CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: In function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getSaturation() const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv]+0x42): undefined reference to `DirectGraphicalModels::fex::CHSV::get(cv::Mat const&)'
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv]+0x73): undefined reference to `DirectGraphicalModels::fex::CCommonFeatureExtractor::getChannel(int) const'
CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: In function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getNDVI(unsigned char) const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor7getNDVIEh[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor7getNDVIEh]+0x4e): undefined reference to `DirectGraphicalModels::fex::CNDVI::get(cv::Mat const&, unsigned char)'
CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: In function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getVariance(DirectGraphicalModels::fex::SqNeighbourhood) const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor11getVarianceENS0_15SqNeighbourhoodE[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor11getVarianceENS0_15SqNeighbourhoodE]+0x5b): undefined reference to `DirectGraphicalModels::fex::CVariance::get(cv::Mat const&, DirectGraphicalModels::fex::SqNeighbourhood)'
collect2: error: ld returned 1 exit status
demos/CMakeFiles/Demo_FEX.dir/build.make:91: recipe for target 'bin/Demo Feature Extraction' failed
make[2]: *** [bin/Demo Feature Extraction] Error 1
CMakeFiles/Makefile2:513: recipe for target 'demos/CMakeFiles/Demo_FEX.dir/all' failed
make[1]: *** [demos/CMakeFiles/Demo_FEX.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

In DGM/3rdparty/densecrf/edgePotentialPotts.cpp, I got the error:
error: invalid initialization of non-const reference of type 'cv::Mat&' from an rvalue of type 'cv::Mat'

Cause of line 34:

Odroid C2, installed OpenCV 3.2.0 but I changed cmake. I commented:

When running CMake, all dependencies seem to be satisfied. Running make from the build directory complains about "Zi" directory not being found. What is this directory?

Scanning dependencies of target DGM
[  1%] Building CXX object modules/DGM/CMakeFiles/DGM.dir/AveragePrecision.cpp.o
clang: error: no such file or directory: '/Zi'
make[2]: *** [modules/DGM/CMakeFiles/DGM.dir/AveragePrecision.cpp.o] Error 1
make[1]: *** [modules/DGM/CMakeFiles/DGM.dir/all] Error 2
make: *** [all] Error 2

System information (version):

• DGM: 1.7.0
• OpenCV: 3.4
• OS: Windows 10 64bit
• Compiler: Visual Studio 2017

Describe the bug
The bug is located in:

Steps To Reproduce
Run the Demo Dense project in Debug mode.

DGM: 1.7.0 && OpenCV 4.0.0 && Ubuntu 18.04 && Compiler g++

First, I can't use CEdgeModelPotts because it's unidentified;
Next, I find "DGM.h" doesn't include "EdgeModelPotts.h" so I include it explicitly.
But it reports

In file included from /usr/local/include/DGM.h:5:0, from /home/base/Desktop/XX.cpp:2: /usr/local/include/DGM/EdgeModelPotts.h:6:10: fatal error: permutohedral/permutohedral.h: No such file or directory #include "permutohedral/permutohedral.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.
I'm now confused. Is it a bug? I expect to use CEdgeModelPotts directly in my code.

In the "CTrainNodeGMM::addFeatureVec()" function the notations:

#ifdef OPTIMIZATION_MODE
    pGauss->addPoint(point);            // update the nearest Gauss
#else
    *pGauss += CNDGauss(point);         // update the nearest Gauss
#endif

missmatch.
Running the classification with OPTIMIZATION_MODE and without produces very different results.

I'm excited to try this library!

I see the forum posts and Github issues asking for this. I don't have Windows, so my options are limited.

Compiling manually with

clang++ -std=c++11 -libstd=libc++ -fms-compatibility -fms-compatibility-version=19.00.23506 -fms-extensions

helps but is not enough.

Please port this to run on other operating systems :-)

This error comes in various parts in make. Is there a spesific flag to show isnan is an std function?

Hi,
are there any plans to port DGM library to Ubuntu?

• DGM:1.6.0
• OpenCV: 3.3.1
• OS: Ubuntu 18.04
• Compiler: Gnu g++

I changed cmake to accept opencv 3.3.1 only ( OPENCV 3 REQUIRED, instead of 3.4). Without any problem build went okay. For a simple #include program I included "DGM.h" and build. This error came out:
[build] In file included from /usr/local/include/DGM.h:43:0, from /home/aliyasineser/Desktop/columnDetection/modules/module_column/include/module_column.h:6, from /home/aliyasineser/Desktop/columnDetection/modules/module_column/src/module_column.cpp:1: /usr/local/include/DGM/GraphDense.h:6:10: fatal error: densecrf/edgePotentialPotts.h: No such file or directory #include "densecrf/edgePotentialPotts.h" compilation terminated.

My Cmake to build:
CMakeLists.txt

I tried to locate how 3rdparty libraries included but I am just new to Cmake and build with cmake stuff. Unfortunately I couldn't find that part. I can try to give any info if necessary, these are the things came to my mind.

System information (version):

• DGM: master
• OpenCV: 4.2.0
• OS: Ubuntu 20.04
• Compiler: gcc 9.3.0

Describe the bug
Compiling the library via the standard CMake procedure (mkdir build && cd build && cmake .. && make -j) fails due to the missing visibility of some symbols, e.g. DirectGraphicalModels::fex::CCoordinate::get(cv::Mat const&, DirectGraphicalModels::fex::coordinateType).

CI would probably help to catch these problems early.

Steps To Reproduce

1. mkdir build && cd build && cmake .. && make -j
2. error message:

/usr/bin/ld: CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: in function `main':
Demo FEX.cpp:(.text+0x118): undefined reference to `DirectGraphicalModels::fex::CCoordinate::get(cv::Mat const&, DirectGraphicalModels::fex::coordinateType)'
/usr/bin/ld: Demo FEX.cpp:(.text+0x229): undefined reference to `DirectGraphicalModels::fex::CCommonFeatureExtractor::invert() const'
/usr/bin/ld: CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: in function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getIntensity(cv::Scalar_<double>) const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor12getIntensityEN2cv7Scalar_IdEE[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor12getIntensityEN2cv7Scalar_IdEE]+0x6a): undefined reference to `DirectGraphicalModels::fex::CIntensity::get(cv::Mat const&, cv::Scalar_<double>)'
/usr/bin/ld: CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: in function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getSaturation() const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv]+0x46): undefined reference to `DirectGraphicalModels::fex::CHSV::get(cv::Mat const&)'
/usr/bin/ld: Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor13getSaturationEv]+0x77): undefined reference to `DirectGraphicalModels::fex::CCommonFeatureExtractor::getChannel(int) const'
/usr/bin/ld: CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: in function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getNDVI(unsigned char) const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor7getNDVIEh[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor7getNDVIEh]+0x52): undefined reference to `DirectGraphicalModels::fex::CNDVI::get(cv::Mat const&, unsigned char)'
/usr/bin/ld: CMakeFiles/Demo_FEX.dir/Demo_FEX.cpp.o: in function `DirectGraphicalModels::fex::CCommonFeatureExtractor::getVariance(DirectGraphicalModels::fex::SqNeighbourhood) const':
Demo FEX.cpp:(.text._ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor11getVarianceENS0_15SqNeighbourhoodE[_ZNK21DirectGraphicalModels3fex23CCommonFeatureExtractor11getVarianceENS0_15SqNeighbourhoodE]+0x5f): undefined reference to `DirectGraphicalModels::fex::CVariance::get(cv::Mat const&, DirectGraphicalModels::fex::SqNeighbourhood)'
collect2: error: ld returned 1 exit status
make[2]: *** [demos/CMakeFiles/Demo_FEX.dir/build.make:92: bin/Demo FEX] Error 1
make[1]: *** [CMakeFiles/Makefile2:413: demos/CMakeFiles/Demo_FEX.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Expected behavior
The library does compile without errors.

Additional context
Add any other context about the problem here.

Do you want a PR? I'm not sure I can't even use the library but I made the changes first, thought about it later. So I can send if you want them.

Hi Sergey,

I wrote a custom dense graph similar to GraphDenseKit. When I test that code, the decoding throws the following exception:

terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.5.1-dev) /home/baer/opencv/modules/core/src/arithm.cpp:669: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'arithm_op'

I attached my header and cpp files.

Thank you !

my code.pdf