jiaxiang-wu / quantized-cnn Goto Github PK

您好，没有在您开源的代码中找到Error Correction的实现代码，请问是怎么实现的

Hello, will your home page open source code Q-CNN is also about quantifying acceleration? If I would like to ask how to use it, if you can write a description of the use I would be appreciate it,thanks.

Hello!!
I have two questions about the implementation.

1. How many training data would be used in error correction step? And did FC layer and Conv layer use the same amount of training data to do error correction?
2. In the paper, you mention fine-tune after quantization. How did fine-tune work? Will fine-tune maintain the structure of D and B?

Thanks!!

Hi, @jiaxiang-wu , thanks for your demo, I set bool kEnblAprxComp = false or true, to test the time of 1000 images.

bool kEnblAprxComp = false; the result is :
ACCURACY@1: 542, 54.20%
elapsed time: 143.5600 (s)

bool kEnblAprxComp = true; the result is :
ACCURACY@1: 545, 54.50%
elapsed time: 116.8000 (s)

the result is weird:

1. The approximation is better than approximation.
2. the time is almost the same. why?

I am not able to read binary file which are in Alexnet/Bin.Files directory, which i downloaded from the url mentioned in README. Getting following error:

static bool FileIO::ReadBinFile(const string&, Matrix*) [with T = float; std::__cxx11::string = std::__cxx11::basic_string]: Assertion `rtnVal == dataCntInBuffer' failed.

It would be great help if you could tell me the cause for this.

Hi! In your paper with respect to quantizing the FC layers,you divide weight matrix into M subspaces represented by a product of D and B.And during the test-phase, you store the results given by the computation of the inner products between S(m) and every sub-codeword in D(m) int a look-up table. For inputs as images which are different， they give different inputs S(m),so how's a look-up table working? Thank you!

I tried the links ILSVRC12.227x227.IMG AlexNet/Bin.Files from README.md, but the corresponding file did not exist

hello,thank you for your contribution to the model quantification model
but I have some problems when I read your paper,
when you quantification the conv kernel and FC,what do you mean by pre-computation?
I can not understand it,the input can not be sure during the test phase ,how to realize the pre-computation

Hello,
Thanks for code reproducing the paper,.
however,when i set the mode "ENUM_CompMethod::Prec",it is the mode that caffe used. it will see [ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.convKnl.01.bin
[ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.convKnl.05.bin
[ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.convKnl.09.bin
[ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.convKnl.13.bin
[ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.fcntWei.16.bin
[ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.fcntWei.19.bin
[ERROR] could not open file at .//AlexNet/Bin.Files/bvlc_alexnet_aCaF.fcntWei.22.bin
etc..
it seems you have forget to upload it .
if i hadn't that file ,i only can get the test time using Q-CNN,and can't make a contrast between Q-CNN and caffe .
I hope you can upload it ,
Thanks you very much~

hello!
Thanks for the code! I was following the steps of installation, the command is
make run
It seems alright at the start but soon it came out with this error:
...
swIndvLayerLst #13: 0.5279 (s)
swIndvLayerLst #14: 0.0028 (s)
swIndvLayerLst #15: 0.0038 (s)
swIndvLayerLst #16: 0.4414 (s)
swIndvLayerLst #17: 0.0003 (s)
swIndvLayerLst #18: 0.0003 (s)
swIndvLayerLst #19: 0.1936 (s)
swIndvLayerLst #20: 0.0003 (s)
swIndvLayerLst #21: 0.0002 (s)
swIndvLayerLst #22: 0.1903 (s)
swIndvLayerLst #23: 0.0024 (s)
elapsed time: 7.9943 (s)
*** Error in `bin/QuanCNN': double free or corruption (!prev): 0x00000000019072c0 ***
make: *** [run] Aborted (core dumped)

Well that looked to me is an invoking problem of internal memory?
Did that ever come to you? Or maybe i just made any steps wrong?

https://github.com/jiaxiang-wu/caffe-QCNN
You github above did not contain the quantized convolution layer ？

In CaffeEva::~CaffeEva(void) , I think it is inappropriate that calling ~Matrix() manually. It causes double release problem due to matrix allocated in stack will be released automatically.
This should be enough ^_^:
CaffeEva::~CaffeEva(void) {
// release dynamically allocated memory
delete [] featMapLst;
for (int layerInd = 0; layerInd < caffeParaObj.layerCnt; layerInd++) {
FeatBufStrLst& featBufStrLst = featBufStrMat[layerInd];
for (std::size_t bufInd = 0; bufInd < featBufStrLst.size(); bufInd++) {
if (featBufStrLst[bufInd].pFeatBuf != nullptr) {
delete featBufStrLst[bufInd].pFeatBuf;
} // ENDIF: featBufStrLst
} // ENDFOR: bufInd
} // ENDFOR: layerInd

// destory objects:
}

Dear @jiaxiang-wu ,
I noitced that the input data size of all example nets are fixed. What if test image size are diffrient from each other? For example, some pixel-wise classification tasks run on FCN. Theoretically, Can the quantized technology apply on FCN? If yes, any modification on this project needed?
Best regard

hi @jiaxiang-wu ,
You teach us to quantize fully-connected layer weights via codebooks. But There is not any instruction in the paper for how the input S(m) could be quantized, So I am not every clear that how the LUT could be pre-computed. Would you pls to explain this more?

If I want to test the origial model, what should I do? And I put the origial model file in the dir , there is no difference.

Sorry to trouble you! But I could not download the two files in the onedrive, so could you give another link? Thanks

Hi,
Thanks for code reproducing the paper, I was able to reproduce the test phase computation following the directions. I was however unable to find any code to generate compact representation (BIN) files for Caffe models, the AlexNet is provided in the repo but there is no code to create the BIN files for models such as VggCnnS or CaffeNet. I reviewed the code, it is basically loading precomputed code files from BIN files and then performing optimized layer operations on it.

Do you have any instructions or code on how to use pretrained Caffe models and generate the corresponding BIN files?

I try to implement the error correction for the FC, but I got a problam that as updaing D(m) and B(m) goes by, some subspces maping table B(m) trend to consuct by only a few sub words, that's leads to updating abort. ?

I try to figure out the reason, it seems that i miss uderstading the definion of residual ? the folling snippet is my implement of coumute the residual of one subspace on all the input.

def conpute_all_residual_on_one_dim(self, sub_dim_index): construct_ouptput = np.zeros(self.N_resudial.shape) for i in xrange(self.num_sub_dims): if sub_dim_index == i: continue table_B = self.from_asmt_data_get_index_table(i) #print table_B.shape, self.centeroid_data[i].shape res_dot = np.dot(table_B, self.centeroid_data[i].T) # dot([4*64], [64*6916]) # sum up residual construct by different subspace # N*6915 = dot([N*4], dot([4*64], [64*6916])) construct_ouptput += np.dot(self.feat_in[:, i*self.len_sub_dim:(i+1)*self.len_sub_dim], res_dot.T) self.N_resudial = self.feat_out - construct_ouptput

@jiaxiang-wu

first thanks for you paper.
I want to know ,how to quantization the weights and input about about the result ??

can you tell me some code about quantization ?

Wondering which folder should I direct to for OpenVML installation?
When I run make, it return

mkdir -p bin
mkdir -p obj
g++ -I/usr/include/atlas -I/opt/OpenVML/include -Wall -std=c++11 -O2 -D ENABLE_ATLAS -D ENABLE_OPENVML -c src/BlasWrapper.cc -o obj/BlasWrapper.o
In file included from src/BlasWrapper.cc:8:0:
src/../include/BlasWrapper.h:84:23: fatal error: openvml.h: No such file or directory
   #include <openvml.h>
                       ^
compilation terminated.
make: *** [obj/BlasWrapper.o] Error 1

Hi,
I still have several question after reading your code and paper.

if the FC network parameter is W (matrix), input is x (vector), so the inference should be y = Wx

• split the W into M sub-matrices, are the M sub-matrices share the same codebook, or they have their own codebook (totol M codebooke should be stored)?
• the test-data-set (labelled data) is needed to compute the D(m) and B(m), is it?
• For each subspace, we compute the inner products between S(m) and every sub-codeword in D(m), and store the results in a look-up table while the S(m) is splitted input layer, whose value should be know only when we need to inference (e.g., a new picture comes to the cnn), how can we compute the inner product in advance?

I may not totally understand the method, could you please explain those questions? Thanks a lot!

Hello -- Will you please share guidance on how to apply the quantization on other model architectures, such as Squeezenet? Thank you in advance.

just as the title, i am wandering howto apply the quantized cnn to an Android phone