eladhoffer / lmdb.torch Goto Github PK

Hello!

After installing the lmdb luarock module, I wanted to use it, but I have received error message:
/home/usero/torch/install/bin/luajit: /home/usero/torch/install/share/lua/5.1/trepl/init.lua:384: /home/usero/torch/install/share/lua/5.1/lmdb/ffi.lua:175: liblmdb.so: cannot open shared object file: No such file or directory stack traceback: [C]: in function 'error' /home/usero/torch/install/share/lua/5.1/trepl/init.lua:384: in function 'require' test.lua:1: in main chunk [C]: in function 'dofile' ...mojo/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

Thank you very much for your help!

Regards,
Marton

I am using lmdb to store the images, but I am having random MDB_NOTFOUND errors after some epochs. The key is clearly in the database, otherwise it won't even finish one epoch. I inspected the lmdb code, I suspect lmdb.MDB_val function in init.lua is problematic. The "value" is ffi.cast to a "void*" pointer, but the value object is local to the function and it will be garbage collected when out of scope according to http://luajit.org/ext_ffi_semantics.html (see section Garbage Collection of cdata Objects). One possible solution would be return the value together with mdb_val and keep it for a while till transaction complete. This probably also caused the problem when using integer keys.

Hi, i'm trying to enable torch loading lmdb files.I installed this code without any error.
But when running the test.lua, an error occurred:
Error in LMDB function mdb_env_open : Function not implemented.

We tried to install http://symas.com/mdb/, while this is not opensource code.
Also, we installed https://github.com/LMDB/lmdb, but this error still exist....

I think I have installed the lmdb wrapper for torch. However torch is not able to require 'lmdb'. My system is ubuntu 16.04.
The following error is thrown. I can't seem to figure out what the problem is. Please help:
th> require 'lmdb'
lua: parser.c:2059: parse_root: Assertion `lua_gettop(L) == top' failed.
Aborted (core dumped)

When opening a connection in a class

 self.lmdb = require 'lmdb'
  self.db = self.lmdb.env {Path = self.dir,Name = split}

I get the following error message :

$ Warning: cannot write object field <mdb_env> of <lmdb.env>
Do you know why ?

Your example usage seems to be not working...

th> db= lmdb.env()
missing argument: Path
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
DB:open

Initializes a LMDB Database

usage:
DB:open{
Path = string -- Name of DataProvider
[MapSize = number] -- Size of map [default = 1099511627776]
[NOSUBDIR = boolean] -- [default = false]
[NOMETASYNC = boolean] -- [default = false]
[RDONLY = boolean] -- [default = false]
[WRITEMAP = boolean] -- [default = false]
[MAPASYNC = boolean] -- [default = false]
[NOSYNC = boolean] -- [default = false]
[NOTLS = boolean] -- [default = false]
[NOLOCK = boolean] -- [default = false]
[Mode = number] -- [default = 664]
[MaxDBs = number] -- [default = 1]
[MaxReaders = number] -- [default = 3]
[Name = string] -- [default = Data]
}

DB:open{MapSize=number, Path=string}
DB:open(string, ...)

After executing test.lua I'm getting following

{
psize : 4096
depth : 1
entries : 6
branch_pages : 0
leaf_pages : 1
overflow_pages : 126
}
Difference: 0
Segmentation fault (core dumped)

but the created LMDB seems to be correct.What can be the reason for segmentation fault?

On trying to run the example provided along with the Readme file, I keep coming across this error, once I have added some samples to the LMDB and am trying to access them:
Error in LMDB function mdb_get : MDB_NOTFOUND: No matching key/data pair found

Any idea what might be causing it?

It turns out that not all samples are written into the DB, which results in the error while reading from it.

I have tried multiple tensor sizes, and the error keeps occurring with any tensor of more than 1 dimension.

In my code, it seems most of the time is spent in reading from lmdb. Trying to find ways to speed it up.

I believe that this is something LMDB can handle but I can't seem to figure it out for myself.

Could you provide any insight into how to get this working?

Thanks.

I think I have installed the lmdb wrapper for torch. However torch is not able to require 'lmdb'. The following error is thrown. I can't seem to figure out what the problem is. Please help

soham@Aspire:~/torch_codes$ th dl4nlp3.lua
/home/soham/torch/install/bin/luajit: /home/soham/torch/install/share/lua/5.1/trepl/init.lua:384: module 'lmdb' not found:No LuaRocks module found for lmdb
no field package.preload['lmdb']
no file '/home/soham/.luarocks/share/lua/5.1/lmdb.lua'
no file '/home/soham/.luarocks/share/lua/5.1/lmdb/init.lua'
no file '/home/soham/torch/install/share/lua/5.1/lmdb.lua'
no file '/home/soham/torch/install/share/lua/5.1/lmdb/init.lua'
no file './lmdb.lua'
no file '/home/soham/torch/install/share/luajit-2.1.0-beta1/lmdb.lua'
no file '/usr/local/share/lua/5.1/lmdb.lua'
no file '/usr/local/share/lua/5.1/lmdb/init.lua'
no file '/home/soham/.luarocks/lib/lua/5.1/lmdb.so'
no file '/home/soham/torch/install/lib/lua/5.1/lmdb.so'
no file '/home/soham/torch/install/lib/lmdb.so'
no file './lmdb.so'
no file '/usr/local/lib/lua/5.1/lmdb.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'error'
/home/soham/torch/install/share/lua/5.1/trepl/init.lua:384: in function 'require'
dl4nlp3.lua:1: in main chunk
[C]: in function 'dofile'
...oham/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
[C]: at 0x00406670

Hi,

I have a very weird problem.

I tried the example code on this webpage (https://github.com/eladhoffer/lmdb.torch) in TREPL. Everything seems fine.

However, when I copy the code into a lua file (say, test.lua), then I run the command th test.lua, it occurs a segmentation fault.

Could anyone help me on this issue? Thanks!