drycpp / lmdbxx Goto Github PK

In my case, I'm storing key/value pairs as int/string.

The problem arises when I wan't to check if given pair exists in database. Currently I'm only able to check if given key exists via lmdb::cursor::find. Is it somehow possible to do this for key/value pair?

In the example code provided in your Readme.MD, the second transaction uses the "dbi" instance that was opened using the first transaction:

auto dbi = lmdb::dbi::open(wtxn, nullptr);
...
auto cursor = lmdb::cursor::open(rtxn, dbi);

Is this recommended? Or should one open a new database for every transaction?

Right now, catching an lmdb::error exception doesn't give much information to aid in troubleshooting incorrect uses of LMDB or runtime errors. For example, mdb_stat: Invalid argument is particularly uninformative and unhelpful. Error messages in exceptions should ideally contain some information on what the parameters passed to the underlying LMDB API function actually were.

Hi, I am trying yo use lmdb++ to work with LMDB. I have it working via the example you provided. I have a problem when trying to use cursor_put with MDB_MULTIPLE. I cant seem to get it to work right. Here is the code:

`std::vector rawData(1000000, std::rand());
MDB_val value1, value2, key;
value1.mv_size = sizeof(int);
value1.mv_data = const_cast<void*> (static_cast<const void*> (&rawData[0]));;
value2.mv_size = 1000000;
MDB_val mData[2] = { value1, value2 };
int i = 0;

auto env = lmdb::env::create();
env.set_mapsize(1UL * 1024UL * 1024UL * 1024UL); /* 1 GiB */
env.open(path.c_str(), 0, 0664);
auto wtxn = lmdb::txn::begin(env);
auto dbi = lmdb::dbi::open(wtxn, nullptr, MDB_DUPSORT | MDB_DUPFIXED | MDB_INTEGERDUP | MDB_CREATE);
auto cursor = lmdb::cursor::open(wtxn, dbi);
key.mv_size = sizeof(i);
key.mv_data = &i;
cursor_put(cursor, &key, mData, MDB_MULTIPLE);`

I think I am probably misusing the lib or some part of it. So any advice would be good.

I tried to compile your example.cc using the latest lmdb++.h in /usr/include like so:

g++ example.cc

But I get the followin error:

/tmp/ccUhugbK.o: In function `lmdb::env_create(MDB_env**)':
example.cc:(.text+0x14): undefined reference to `mdb_env_create'
/tmp/ccUhugbK.o: In function `lmdb::env_open(MDB_env*, char const*, unsigned int, unsigned int)':
example.cc:(.text+0x5b): undefined reference to `mdb_env_open'
/tmp/ccUhugbK.o: In function `lmdb::env_close(MDB_env*)':
example.cc:(.text+0x8e): undefined reference to `mdb_env_close'
/tmp/ccUhugbK.o: In function `lmdb::env_set_flags(MDB_env*, unsigned int, bool)':
example.cc:(.text+0xc8): undefined reference to `mdb_env_set_flags'
/tmp/ccUhugbK.o: In function `lmdb::env_set_mapsize(MDB_env*, unsigned long)':
example.cc:(.text+0x106): undefined reference to `mdb_env_set_mapsize'
/tmp/ccUhugbK.o: In function `lmdb::txn_begin(MDB_env*, MDB_txn*, unsigned int, MDB_txn**)':
example.cc:(.text+0x14f): undefined reference to `mdb_txn_begin'
/tmp/ccUhugbK.o: In function `lmdb::txn_commit(MDB_txn*)':
example.cc:(.text+0x182): undefined reference to `mdb_txn_commit'
/tmp/ccUhugbK.o: In function `lmdb::txn_abort(MDB_txn*)':
example.cc:(.text+0x1b5): undefined reference to `mdb_txn_abort'
/tmp/ccUhugbK.o: In function `lmdb::dbi_open(MDB_txn*, char const*, unsigned int, unsigned int*)':
example.cc:(.text+0x1e6): undefined reference to `mdb_dbi_open'
/tmp/ccUhugbK.o: In function `lmdb::dbi_put(MDB_txn*, unsigned int, MDB_val const*, MDB_val*, unsigned int)':
example.cc:(.text+0x239): undefined reference to `mdb_put'
/tmp/ccUhugbK.o: In function `lmdb::cursor_open(MDB_txn*, unsigned int, MDB_cursor**)':
example.cc:(.text+0x28b): undefined reference to `mdb_cursor_open'
/tmp/ccUhugbK.o: In function `lmdb::cursor_close(MDB_cursor*)':
example.cc:(.text+0x2be): undefined reference to `mdb_cursor_close'
/tmp/ccUhugbK.o: In function `lmdb::cursor_get(MDB_cursor*, MDB_val*, MDB_val*, MDB_cursor_op)':
example.cc:(.text+0x2ef): undefined reference to `mdb_cursor_get'
/tmp/ccUhugbK.o: In function `lmdb::error::what() const':
example.cc:(.text._ZNK4lmdb5error4whatEv[_ZNK4lmdb5error4whatEv]+0x1c): undefined reference to `mdb_strerror'
collect2: error: ld returned 1 exit status

Looking inside lmdb++.h it seems that mdb_env_create hasn't been defined anywhere, but it IS referenced.

PS: Thanks for this great library, I hope to make use of it.

Not really sure if this is an issue as I don't think there is any fix for this given the available lmdb library, but I think it is worth noting on github wiki.

As per the lmdb documents for commiting a tx "Earlier documentation incorrectly said all cursors would be freed. Only write-transactions free cursors.".

So when a wite tx is committed AND a cursor object live, the underlying MDB_cursor* is freed during the commit, then freed when the cursor object is destroyed resulting in memory violation. As such the ordering of committing and destroying (or closing) below will crash.

wtxn.commit(); cursor.close();

I do not think there is a way for a cursor to know if the tx has been committed, so there seems to be no trivial fix if one at all.

An easy solution from a users perspective is just to make sure you actively close all cursors of a write tx before before committing. Its pretty simple but it took me a while to figure out where the memory violation was coming from. Perhaps this could be added somewhere to your wiki to warn users of this.

cursor.close(); wtxn.commit();

@core-process noticed and fixed this issue in our C++17 fork of lmdbxx:

If an exception was throw by txn.commit() (ie MDB_MAP_FULL), and this transaction was later aborted (because it went out of scope while unwinding the stack), then a double-free would occur.

You can use the following test to observe this (address sanitizer should be enabled, as it is by default in our Makefile):

https://github.com/hoytech/lmdbxx/blob/5223582ebf92a9b14608ce6768535aef0c65910f/check.cc#L329

Describe the bug
This is one single header wrapped in an entire package. Maybe it should just be merged into lmdb? They accept pull requests.

It makes sense that the env and txn classes are moveable but not copyable, since they represent resources that are closed on destruction. But dbi isn't like that — the destructor is a no-op. So I don't see a reason for not having a copy constructor and copying assignment.

Having these would make it cleaner to pass dbi instances as parameters, store them in member variables, etc.

I'd like to use lmdb++ but as it stands I cannot because of it's dependency on pthreads... The real problem is that on Windows it looks like the current code requires pthread_t from mingw and I am using Visual Studio on Windows. Since it requires a modern implementation of C++ anyway, might it be possible to use only standard features?

Error is "lmdb++.h:97:12: error: thread-local storage is not supported for the current target
static thread_local char buffer[1024];"

Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.2.0
Thread model: posix

First of all I want to say thanks, lmdbxx has been very useful to me!

For anyone using C++17 in their projects, std::string_view is a very nice complement to LMDB so I integrated it into lmdbxx. Since Arto hasn't checked in for a while, I just want to let anyone browsing here that I will maintain my fork by answering questions and incorporating bugfixes and so on.

Here's the link to my fork: https://github.com/hoytech/lmdbxx

Weird output from cursor get: I would get both key and value inside the key variable, then the value inside the value variable :

#include <cstdio>
#include <cstdlib>
#include <lmdb++.h>
using namespace lmdb;
int getsize(const lmdb::env& e){
    auto t = lmdb::txn::begin(e.handle(), nullptr, MDB_RDONLY);
    auto d = lmdb::dbi::open(t, nullptr);
    int r=d.size(t);
    t.abort();
    return r;
}


int main() {
  auto env = lmdb::env::create();
  env.set_mapsize(1UL * 1024UL * 1024UL * 1024UL); /* 1 GiB */
  env.open("./example.mdb", 0, 0664);
  {
    auto wtxn = lmdb::txn::begin(env);
    auto dbi = lmdb::dbi::open(wtxn, nullptr);
    char a[6] = "hello";
    dbi.put(wtxn, "email", "hello");
    dbi.put(wtxn, "key", "value");
    dbi.put(wtxn, "user", "johndoe");
    wtxn.commit();
  }
  {
      auto rtxn = lmdb::txn::begin(env);
      auto dbi = lmdb::dbi::open(rtxn, nullptr);
      auto cursor = lmdb::cursor::open(rtxn, dbi);
      lmdb::val k, v;
      while(cursor.get(k, v, MDB_NEXT)){
        printf("We got '%s'\nValue '%s'\n", k.data(), v.data());
      }
  }
  {
    std::printf("size is %d\n", getsize(env));
  }
return EXIT_SUCCESS;
}

Expected Output:

We got 'email'
Value 'hello'
We got 'key'
Value 'value'
We got 'user'
Value 'johndoe'
size is 3

Output :

We got 'emailhello'
Value 'hello'
We got 'keyvalue'
Value 'value'
We got 'userjohndoe'
Value 'johndoe'
size is 3

Why is it that when I do a put with a key that already exists, it doesn't update the value for that key? Do I have to delete the old key/value before adding the new key/value?

I tried to simply delete an entry, and can't really know if I'm wrong or if this is a bug, the only documentation I could find was the api reference, if anyone has better material I'd be glad to take it.

#include <cstdio>
#include <cstdlib>
#include <lmdb++.h>
using namespace lmdb;
int main() {
  auto env = lmdb::env::create();
  env.set_mapsize(1UL * 1024UL * 1024UL); 
  env.open("./example.mdb", 0, 0664);

  auto wtxn = lmdb::txn::begin(env);
  auto dbi = lmdb::dbi::open(wtxn, nullptr);
  dbi.put(wtxn, "key_entry", "value_entry");
  wtxn.commit();
  dbi.del(wtxn, "key_entry");

  return EXIT_SUCCESS;
}

This yields the following error :

terminate called after throwing an instance of 'lmdb::runtime_error'
  what():  mdb_del: Invalid argument

I have already tried to use lmdb++ but I failed.
What I want to do is simple, it is like the example.cc but I want to get an specific value of a key of the data base instead of go over all the values that have been saved on it with cursors.

For example:

int main() {
  auto env = lmdb::env::create();
  env.open("./example.mdb", 0, 0664);

  auto wtxn = lmdb::txn::begin(env);
  auto dbi = lmdb::dbi::open(wtxn, nullptr);
  dbi.put(wtxn, "username", "jhacker");
  dbi.put(wtxn, "email", "[email protected]");
  dbi.put(wtxn, "fullname", "J. Random Hacker");
  wtxn.commit();

  auto rtxn = lmdb::txn::begin(env);

  std::string value;
  dbi.get(rtxn, "username", value);
  std::printf("value: '%s'\n",value.c_str());

  rtxn.abort();

  return EXIT_SUCCESS;
}

But it returns me:
Segmentation fault (core dumped)

How can I do this?

in lmdb::dbi_put there is no way to put an element like :
int data[10]
lmdb::val lmdbData((void*)data, sizeof(int)*10 );
...
dbi.put(wtxn, "data", lmdbData))

Hi,
When I try to save orgin image data using lmdb++, I find that I cannot save images containing piexls with gray-level zero. As I have to transform the image data into a string first, the string usually 'breaks' whenever there exits zero pixels. How can I solve this problem ?
Thanks very much!

Would it be possible to tag a new release?
Thanks!