vorot93 / libmdbx-rs

Rust bindings for MDBX

libmdbx-rs

Rust bindings for libmdbx.

Updating the libmdbx Version

To update the libmdbx version you must clone it and copy the dist/ folder in mdbx-sys/. Make sure to follow the building steps.

# clone libmmdbx to a repository outside at specific tag
git clone https://gitflic.ru/project/erthink/libmdbx.git ../libmdbx --branch v0.7.0
make -C ../libmdbx dist

# copy the `libmdbx/dist/` folder just created into `mdbx-sys/libmdbx`
rm -rf mdbx-sys/libmdbx
cp -R ../libmdbx/dist mdbx-sys/libmdbx

# add the changes to the next commit you will make
git add mdbx-sys/libmdbx

License

The entire code within this repository is licensed under the Mozilla Public License v2.0

libmdbx-rs's Issues

Cannot set page_size

Hello, and thanks for this crate!

I'm trying to set the page_size Geometry attribute in the set_geometry function but it looks like it's only possible to do so with PageSize::Set() to get the proper PageSize type for this value and it is located in a private module, so I can't access it.

So if I try libmdbx::environment::PageSize::Set(...) then I have the module 'environment' is private error obviously.

Thanks in advance for your insight! I'm pretty new to Rust so sorry in advance if I missed something obvious 👀

`libmdbx::cursor::IntoIter` is private

I need to use type libmdbx::cursor::IntoIter to create a custom iterator, but it seems to be a private type. Is it possible to expose it? If yes, can I send a merge request for this?

can give an example how to use?

as the title can give an example how to use? thanks

Update of bitflags to version 2 is backwards-incompatible

As noted in the release notes for bitflags v2 (https://github.com/bitflags/bitflags/releases/tag/2.0.0), the library changed in a backward-incompatible way, and you need to add new trait derives to your publicly exposed bitflags.

In our case, we require the Debug trait, which is now no longer derived automatically in v2. But it would probably be good to add all the previously derived traits, as shown in the above-linked release notes:

In src/flags.rs for both TableFlags and WriteFlags:

- #[derive(Default)]
+ #[derive(Default, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Clone, Copy)]

`map_resize` assertion failure

I'm seeing these in my logs:

mdbx:9268: map_resize: Assertion `size_bytes == env->me_dxb_mmap.current' failed.

This is how I've setup libmdbx:

const GIB: usize = 1024 * 1024 * 1024;
const MIN_DB_SIZE: usize = GIB;
const MAX_DB_SIZE: usize = 10 * GIB;

let env = Environment::new()
            .set_max_dbs(1024)
            .set_geometry(libmdbx::Geometry {
                size: Some(MIN_DB_SIZE..MAX_DB_SIZE),
                growth_step: None,
                shrink_threshold: None,
                page_size: None,
            })
            .open(path.as_ref())?;

Should I be doing things differently? My DB is using about 2GB on disk right now.

[Yocto Kirkstone] build break cross compiling

 |   process didn't exit successfully: `/__w/meta-flutter/zero2w-drm-kirkstone-latest/raspberrypi0-2w-64/tmp/work/cortexa53-poky-linux/isar-core/2.5.11-r0/build/target/release/build/mdbx-sys-885295866be6bada/build-script-build` (exit status: 101)
|   --- stderr
|   /usr/include/pthread.h:657:6: error: 'regparm' is not valid on this platform
|   /usr/include/pthread.h:669:3: error: 'regparm' is not valid on this platform
|   /usr/include/pthread.h:710:6: error: 'regparm' is not valid on this platform
|   /usr/include/pthread.h:657:6: error: 'regparm' is not valid on this platform, err: true
|   /usr/include/pthread.h:669:3: error: 'regparm' is not valid on this platform, err: true
|   /usr/include/pthread.h:710:6: error: 'regparm' is not valid on this platform, err: true
|   thread 'main' panicked at 'Unable to generate bindings: ()', mdbx-sys/build.rs:99:10

Cannot compile on windows -> expected `u32`, found `i32

Hello,

I am new to rust and trying to cargo build the mdbx-rs package on windows 10 x64 with clion ide. Every time i run the cargo build command its giving error : expected u32, found i32 . I have made the same tests on visual studio code and got the same problem.

How can i resolve this?

Best

Complete result :

Compiling cexpr v0.5.0
Compiling thiserror-impl v1.0.29
Compiling derive_more v0.99.16
Checking thiserror v1.0.29
Compiling mdbx-sys v0.9.3 (C:\Users\tarkan\CLionProjects\lmdbx_test_3\mdbx-rs\mdbx-sys)
Checking mdbx v0.1.0 (C:\Users\tarkan\CLionProjects\lmdbx_test_3\mdbx-rs)
error[E0308]: mismatched types
--> src\cursor.rs:384:28
|
384 | IterDup::new(self, ffi::MDBX_NEXT)
| ^^^^^^^^^^^^^^ expected u32, found i32
|
help: you can convert an i32 to a u32 and panic if the converted value doesn't fit
|
384 | IterDup::new(self, ffi::MDBX_NEXT.try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\cursor.rs:394:28
|
394 | IterDup::new(self, ffi::MDBX_FIRST)
| ^^^^^^^^^^^^^^^ expected u32, found i32
|
help: you can convert an i32 to a u32 and panic if the converted value doesn't fit
|
394 | IterDup::new(self, ffi::MDBX_FIRST.try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\cursor.rs:408:28
|
408 | IterDup::new(self, ffi::MDBX_GET_CURRENT)
| ^^^^^^^^^^^^^^^^^^^^^ expected u32, found i32
|
help: you can convert an i32 to a u32 and panic if the converted value doesn't fit
|
408 | IterDup::new(self, ffi::MDBX_GET_CURRENT.try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\cursor.rs:444:76
|
444 | ffi::mdbx_cursor_put(self.cursor, &key_val, &mut data_val, flags.bits())
| ^^^^^^^^^^^^ expected i32, found u32
|
help: you can convert a u32 to an i32 and panic if the converted value doesn't fit
|
444 | ffi::mdbx_cursor_put(self.cursor, &key_val, &mut data_val, flags.bits().try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\cursor.rs:460:51
|
460 | ffi::mdbx_cursor_del(self.cursor, flags.bits())
| ^^^^^^^^^^^^ expected i32, found u32
|
help: you can convert a u32 to an i32 and panic if the converted value doesn't fit
|
460 | ffi::mdbx_cursor_del(self.cursor, flags.bits().try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\cursor.rs:811:43
|
811 | let op = mem::replace(op, ffi::MDBX_NEXT_NODUP);
| ^^^^^^^^^^^^^^^^^^^^ expected u32, found i32
|
help: you can convert an i32 to a u32 and panic if the converted value doesn't fit
|
811 | let op = mem::replace(op, ffi::MDBX_NEXT_NODUP.try_into().unwrap());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\cursor.rs:815:93
|
815 | unsafe { ffi::mdbx_cursor_get(cursor.cursor(), &mut key, &mut data, op) };
| ^^ expected i32, found u32
|
help: you can convert a u32 to an i32 and panic if the converted value doesn't fit
|
815 | unsafe { ffi::mdbx_cursor_get(cursor.cursor(), &mut key, &mut data, op.try_into().unwrap()) };
| ^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\database.rs:37:47
|
37 | ffi::mdbx_dbi_open(txn, name_ptr, flags, &mut dbi)
| ^^^^^ expected i32, found u32
|
help: you can convert a u32 to an i32 and panic if the converted value doesn't fit
|
37 | ffi::mdbx_dbi_open(txn, name_ptr, flags.try_into().unwrap(), &mut dbi)
| ^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0599]: no method named as_bytes found for reference &OsStr in the current scope
--> src\environment.rs:507:64
|
507 | let path = match CString::new(path.as_os_str().as_bytes()) {
| ^^^^^^^^ method not found in &OsStr

error[E0308]: mismatched types
--> src\transaction.rs:298:67
|
298 | ffi::mdbx_put(txn, db.dbi(), &key_val, &mut data_val, flags.bits())
| ^^^^^^^^^^^^ expected i32, found u32
|
help: you can convert a u32 to an i32 and panic if the converted value doesn't fit
|
298 | ffi::mdbx_put(txn, db.dbi(), &key_val, &mut data_val, flags.bits().try_into().unwrap())
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
--> src\transaction.rs:330:36
|
330 | flags.bits() | ffi::MDBX_RESERVE,
| ^^^^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\transaction.rs:330:21
|
330 | flags.bits() | ffi::MDBX_RESERVE,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected i32, found u32
|
help: you can convert a u32 to an i32 and panic if the converted value doesn't fit
|
330 | (flags.bits() | ffi::MDBX_RESERVE).try_into().unwrap(),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0277]: no implementation for u32 | i32
--> src\transaction.rs:330:34
|
330 | flags.bits() | ffi::MDBX_RESERVE,
| ^ no implementation for u32 | i32
|
= help: the trait BitOr<i32> is not implemented for u32

error[E0308]: mismatched types
--> src\flags.rs:168:29
|
168 | const REVERSE_KEY = MDBX_REVERSEKEY;
| ^^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:169:26
|
169 | const DUP_SORT = MDBX_DUPSORT;
| ^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:170:29
|
170 | const INTEGER_KEY = MDBX_INTEGERKEY;
| ^^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:171:27
|
171 | const DUP_FIXED = MDBX_DUPFIXED;
| ^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:172:29
|
172 | const INTEGER_DUP = MDBX_INTEGERDUP;
| ^^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:173:29
|
173 | const REVERSE_DUP = MDBX_REVERSEDUP;
| ^^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:174:24
|
174 | const CREATE = MDBX_CREATE;
| ^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:175:24
|
175 | const ACCEDE = MDBX_DB_ACCEDE;
| ^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:183:24
|
183 | const UPSERT = MDBX_UPSERT;
| ^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:184:30
|
184 | const NO_OVERWRITE = MDBX_NOOVERWRITE;
| ^^^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:185:29
|
185 | const NO_DUP_DATA = MDBX_NODUPDATA;
| ^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:186:25
|
186 | const CURRENT = MDBX_CURRENT;
| ^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:187:25
|
187 | const ALLDUPS = MDBX_ALLDUPS;
| ^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:188:25
|
188 | const RESERVE = MDBX_RESERVE;
| ^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:189:24
|
189 | const APPEND = MDBX_APPEND;
| ^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:190:28
|
190 | const APPEND_DUP = MDBX_APPENDDUP;
| ^^^^^^^^^^^^^^ expected u32, found i32

error[E0308]: mismatched types
--> src\flags.rs:191:26
|

Windows builds failing

For windows builds (locally and on the CI) I always get the following error:

LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library
          mdbx.lib(mdbx.c.obj) : error LNK2019: unresolved external symbol __imp_CharToOemBuffA referenced in function mdbx_strerror_r_ANSI2OEM

Any ideas what could be wrong?

0.1.8 break macos build

"/Users/runner/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-1fc8833aab60d522.rlib" "-framework" "Security" "-framework" "CoreFoundation" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/runner/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "-o" "/Users/runner/work/papyrus/papyrus/target/debug/deps/papyrus_storage-ef13791cab1400dd" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: Undefined symbols for architecture x86_64:
            "___cpu_indicator_init", referenced from:
                _scan4seq_resolver in libmdbx_sys-1ed6b037cfe501ab.rlib(mdbx.o)
            "___cpu_model", referenced from:
                _scan4seq_resolver in libmdbx_sys-1ed6b037cfe501ab.rlib(mdbx.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
          

Re: Unable to compile on windows

Originally posted here meilisearch/heed#107

Currently, https://github.com/Kerollmops/heed with mdbx backend is not compiling on windows. Also, @erthink pointed out that the issue might be in the bindings.

I am still new to rust but I am willing to help ✌🏻