kyamagu / matlab-lmdb Goto Github PK

Hi,
Is there an easy fix to read float lmdb (features from caffe)?

Hi,

Sometimes I get the following error when I try to read from a db, following which matlab crashes:

[libprotobuf ERROR google/protobuf/descriptor_database.cc:57] File already exists in database: caffe.proto
[libprotobuf FATAL google/protobuf/descriptor.cc:954] CHECK failed: generated_database_->Add(encoded_file_descriptor, size):

Any ideas as to why this is happening ?

After I run this command :

I cant not find this function caffe_pb.toDatum
Should I rebuild the caffe or what should I do

Hi,

I am wondering how to use this package to load existing .mdb files into matlab ? I tried this:

database = lmdb.DB(<path-to-mdb-file>,'RDONLY',true);

but without success:

Error using LMDB_
Not a directory

Error in lmdb.DB (line 65)
this.id_ = LMDB_('new', filename, varargin{:});

Could you please clarify this ?

Thanks for this great work. However, I wonder if the wrapper supported with single value. I used function ".put() " with single value but get a corresponding integer with function ".get() ". Could you make some instruction about read and write with single value? Thank your!

I have some .mat files which I need to convert into lmdb format.

Could you please tell how can I do it using this wrapper?

I encounter error during compiling

This file requires compiler and library support for the upcoming ISO C++ standard, C++0x. This support is currently experimental, and must be enabled with the -std=c++0x or -std=gnu++0x compiler options.

Please help >.<

Hi,

I am having a problem when I clear a cursor. I have put it as a field in a Matlab struct and, for some reason, when I call clear cursor; and the method below is execute the 'txn_commit' and the 'txn_delete' functions are making the Matlab to crash. If I comment both lines it finishes my script correctly.

Is there any drawback of commenting these lines? I am not sure what why they are being executed after the 'cursor_delete' one.

Do you have any idea about what is wrong with it? Btw, thanks for the nice package.

function delete(this)
%DELETE Destructor.
assert(isscalar(this));
LMDB_('cursor_delete', this.id_);
%LMDB_('txn_commit', this.transaction_id_);
%LMDB_('txn_delete', this.transaction_id_);
end

My bad. Never mind.

Hi Kyamagu
The matlab crashes when I run the following command:
datum = caffe_pb.toDatum(data,label),
where label=1 and data is a 7x7x2048 single or uint8 array.

The error information is as follows:
[libprotobuf ERROR google/protobuf/descriptor_database.cc:57] File already exists in database: caffe.proto
[libprotobuf FATAL google/protobuf/descriptor.cc:1018] CHECK failed: generated_database_->Add(encoded_file_descriptor, size):

However, the file should not already exists in the database.

I am looking forward to your reply : )
best,
Yan

Hi kyamagu,
First at the most, this is a very nice work!!
I want to save some 3D matrix into lmdb database. I was able to do it with database.put('MyMatrix', MyMatrix) without any problem. I was also able to retrieve the data back by using MyMatrix2=database.get('MyMatrix'). However, the problem is the retrieved MyMatrix2 is a array of char while the saved MyMatrix is a matrix of int16. Let's say size(MyMatrix) == 256x256x32 (int16) then size(MyMatrix2) == 1x2097152 (char). Do I need to explicitly tell put() function the size of the data or did I just miss something important? Please help

I got this error with the newest release

/usr/local/MATLAB/R2013a/bin/mex -output +lmdb/private/LMDB_.mexa64 src/LMDB_.cc -Iinclude -Isrc/liblmdb CXXFLAGS="$CXXFLAGS -std=c++11" src/liblmdb/liblmdb.a
cc1plus: error: unrecognized command line option "-std=c++11"

mex: compile of ' "src/LMDB_.cc"' failed.

make: *** [+lmdb/private/LMDB_.mexa64] Error 1

Hi,
Sorry to bother..I'm a windows user and I'm quite new to the terms unix, linux, shell, etc.
I notice this is for UNIX but I thought to give it a try with 'MinGW make' through git bash under the matlab-lmdb-caffe-extension folder and got the following error msg. Just want to confirm if this is something to do with the cross-platform? Is there a quick solution to compile on windows?
Thanks!

$ make
C:/MinGW/bin/make -C src/liblmdb
make[1]: Entering directory 'C:/Projects/matlab-lmdb-caffe-extension/src/liblmdb'
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mdb_stat.o liblmdb.a -o mdb_stat
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mdb_copy.o liblmdb.a -o mdb_copy
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mdb_dump.o liblmdb.a -o mdb_dump
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mdb_load.o liblmdb.a -o mdb_load
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mtest.o liblmdb.a -o mtest
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mtest2.o liblmdb.a -o mtest2
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mtest3.o liblmdb.a -o mtest3
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mtest4.o liblmdb.a -o mtest4
gcc -fPIC -pthread -O2 -g -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized mtest5.o liblmdb.a -o mtest5
make[1]: Leaving directory 'C:/Projects/matlab-lmdb-caffe-extension/src/liblmdb'
C:/MATLAB/R2016b/bin/mex.bat -output +lmdb/private/LMDB_.mexw64 src/LMDB_.cc -Iinclude -Isrc/liblmdb CXXFLAGS="$CXXFLAGS -std=c++11" src/liblmdb/liblmdb.a
Building with 'MinGW64 Compiler (C++)'.
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xe39): undefined reference to mdb_txn_abort' C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xe58): undefined reference to mdb_txn_begin'
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xe63): undefined reference to mdb_strerror' C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xe9e): undefined reference to mdb_txn_abort'
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xecf): undefined reference to mdb_txn_abort' C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xf07): undefined reference to mdb_cursor_close'
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xf1d): undefined reference to mdb_cursor_open' C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xf28): undefined reference to mdb_strerror'
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xf61): undefined reference to mdb_txn_commit' C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xf6c): undefined reference to mdb_strerror'
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xfbe): undefined reference to mdb_dbi_close' C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0xfc7): undefined reference to mdb_env_close'
C:\Users\ding\AppData\Local\Temp\mex_8670067939871759_6064\LMDB_.obj:LMDB_.cc:(.text+0x1007): undefined reference to `mdb_dbi_close'
...
...
...
...
...
...
collect2.exe: error: ld returned 1 exit status

Makefile:18: recipe for target '+lmdb/private/LMDB_.mexw64' failed
make: *** [+lmdb/private/LMDB_.mexw64] Error 127

I was trying to create a LMDB dataset with a huge MAPSIZE and when i insert a datum on it, it fires the following error: Error using LMDB_MDB_MAP_FULL: Environment mapsize limit reached.

What is the MAPSIZE maximum value ?
And is there some work around to overcome this limitation ?