coolzhao / py-leveldb Goto Github PK

Hi,

I've tried to build python-leveldb on Debian, the result is shown below:

python3.2 setup.py build
running build
running build_py
package init file '__init__.py' not found (or not a regular file)
package init file '__init__.py' not found (or not a regular file)
running build_ext
building 'leveldb' extension
creating build
creating build/temp.linux-x86_64-3.2
gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g 
-fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security 
-Werror=format-security -fPIC -I/usr/include/python3.2mu -c leveldb_ext.cc -o 
build/temp.linux-x86_64-3.2/leveldb_ext.o -I./leveldb/include -fPIC -Wall -g2 
-D_GNU_SOURCE -O2 -DNDEBUG
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC 
but not for C++ [enabled by default]
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC 
but not for C++ [enabled by default]
leveldb_ext.cc: In function 'PyObject* initleveldb()':
leveldb_ext.cc:18:80: error: 'Py_InitModule3' was not declared in this scope
leveldb_ext.cc:21:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:24:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:27:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:30:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:36:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:41:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:46:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:52:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:57:3: error: return-statement with no value, in function 
returning 'PyObject* {aka _object*}' [-fpermissive]
leveldb_ext.cc:58:1: warning: no return statement in function returning 
non-void [-Wreturn-type]
error: command 'gcc' failed with exit status 1
make[1]: *** [build-python3.2] Error 1

Would you please add the support for Python 3?
Thanks in advance.

py-leveldb r23 source code was build error and import error under pypy 1.5.0 
and virtualenv.

~/dev/py-leveldb-read-only/test$ python -m unittest test
F....
======================================================================
FAIL: testIterationBasic (test.TestLevelDB)
testIterationBasic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 134, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/utils.py", line 191, in runWithWarningsSuppressed
    result = f(*a, **kw)
  File "test.py", line 188, in testIterationBasic
    self._test_lowercase_iter(db)
  File "test.py", line 170, in _test_lowercase_iter
    self.assertEquals(s, 'mlkj')
  File "/usr/lib/python2.7/dist-packages/twisted/trial/unittest.py", line 270, in assertEqual
    % (msg, pformat(first), pformat(second)))
FailTest: not equal:
a = ''
b = 'mlkj'


----------------------------------------------------------------------
Ran 5 tests in 1.012s

FAILED (failures=1)

The docs for LevelDB.RangeIter read:

    RangeIter(key_from=None, key_to=None, [...])

However, key_from and key_to are required to be buffers if present. This 
complicates things for the caller, and makes for slightly misleading 
documentation.

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.

Really simple - RepairDB doesn't support a custom comparator.

Patch attached.

What steps will reproduce the problem?
1.building the Python extensions using "$python setup.py build" from the 
installation steps
2.Or installing using "sudo pip install leveldb"
3.

What is the expected output? What do you see instead?
Expected output would be a success in building the code, but instead I see:

cc1plus: error: -Werror=unused-command-line-argument-hard-error-in-future: no 
option -Wunused-command-line-argument-hard-error-in-future

error: command 'gcc' failed with exit status 1


What version of the product are you using? On what operating system?
Current version. Mac OS X 10.9.5. 


Please provide any additional information below.
I have tried building the code with Xcode 6.1.1 as well as Xcode 5.0.2. I have 
also tried the temprorary fix provided at 
"http://stackoverflow.com/questions/23144307/installing-pillow-getting-wunused-c
ommand-line-argument-hard-error-in-future", but it still does not work. 

[root@localhost leveldb]# python
Python 2.6.7 (r267:88850, Apr 25 2013, 16:12:30) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> leveldb
<module 'leveldb' from 
'/root/.python-eggs/leveldb-0.1-py2.6-linux-x86_64.egg-tmp/leveldb.so'>

>>> import leveldb
>>> db = leveldb.LevelDB('./db')
>>> dir(db.RangeIter())
Segmentation fault

leveldb version was 1.9

Compilation with the compile_leveldb.sh script fails due to an include error in 
snappy:

c++ -I. -I./include -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX -fPIC -O2 -DNDEBUG 
-DSNAPPY -I../snappy-read-only -c db/builder.cc -o db/builder.o
In file included from db/builder.cc:7:
In file included from ./db/filename.h:14:
In file included from ./port/port.h:14:
In file included from ./port/port_posix.h:46:
../snappy-read-only/snappy.h:45:10: fatal error: 'snappy-stubs-public.h' file 
not found
#include "snappy-stubs-public.h"
         ^
1 error generated.
make: *** [db/builder.o] Error 1

What steps will reproduce the problem?
1. Run the script attached to this report (WARNING: It will exhaust the 
available memory when run for long enough)

What is the expected output? What do you see instead?
I expect memory usage to rise to a certain level and stay at that level.

Instead memory usage keeps rising, even after syncing the WriteBatch to disk, 
and subsequently deleting the WriteBatch object.

What version of the product are you using? On what operating system?
Version 0.19 from here: https://pypi.python.org/pypi/leveldb
On Ubuntu Raring amd64

Please provide any additional information below.

What steps will reproduce the problem?
1. Download pyleveldb
2. Run ./compile_leveldb

What is the expected output? What do you see instead?
It used to run just fine... but now I'm getting 

configure.ac:40: error: possibly undefined macro: AC_DEFINE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.

Another report of this instance: http://blog.plotcup.com/a/68

What version of the product are you using? On what operating system?
I'm on Debian Wheezy 64bit.

Please provide any additional information below.

Even if you don't have full pypi support, being able to use setuptools means 
you can use things like virtualenv without any extra steps. Patch forthcoming.

Happened across this extension when I was looking for a python wrapper for 
leveldb, and without noticing the "POSIX" in the package classifier, I 
attempted building py-leveldb against the current release of leveldb-win. 
(http://code.google.com/p/leveldbwin/)

While the setup script didn't work, VC++10 compiled the code just fine using 
the following command:

cl.exe /nologo /LD /EHsc /I[Python Include Folder] /I. /MD /O2 leveldb_ext.cc 
leveldb_object.cc /link leveldb.lib /out:lib\leveldb.pyd /libpath:[Python Libs 
Folder]

I haven't gotten a chance to try out all the API provided by the extension, but 
the example on the front page runs perfectly. If you're interested, I can throw 
together a quick build script for windows.

What steps will reproduce the problem?
import leveldb
db = leveldb.LevelDB('nana')
db.Put('hh', 'jj')
iterator = db.RangeIter()

What is the expected output? What do you see instead?
an iterator would be assigned into the variable

instead the python interpreter crash with the message:
pthread lock: Invalid argument
Aborted

What version of the product are you using? On what operating system?
fedora19

it would be nice to support CompactRange()

Yes this is an advanced feature, or maybe a developer feature of LDB, but I 
need it to diagnose actual corruption in an LDB from python.

Patch attached.

What steps will reproduce the problem?
1. call leveldb.RepairDB(dir) on any leveldb database <dir>
2.
3.

What is the expected output? What do you see instead?

It should return None, but instead dumps core with a Segmentation Violation. A 
stack backtrace shows that it reaches the leveldb C++ library RepairDB 
function, so I assume the issue is with the formatting of input to that 
function.

What version of the product are you using? On what operating system?

I'm using a py-leveldb snapshot from June 12th, but the RepairDB code is 
unchanged since then. The OS is FreeBSD 8.2 and FreeBSD 9.0, with Python 
version 2.7.3


Please provide any additional information below.

it would be nice to have a reverse keyword argument to the 
RangeIter so that when reverse=True iterator->Prev() is used
in place of iterator->Next()

Hi,

the sources totally lack information about licensing and copyright holders.

Please add a COPYING file (or LICENSE or whatever) including the full text of 
the license with which you want to distribute the sources.

Thanks!

In leveldb_object.cc, PyLevelDB_Get_ switches on whether or not 
PY_LEVELDB_BUFFER_3 or PY_LEVELDB_BUFFER_26 is defined.

Unfortunately, neither of these are getting defined on my system for some 
reason, so I'm getting the wrong implementation for my Python version and my 
process is getting segmentation faults.

Up above where PyLevelDB_Get_ is defined, the PY_LEVELDB_BUFFER macros are 
defined conditional on PYTHON_MAJOR_VERSION and PYTHON_MINOR_VERSION. In my 
copy of the Python header files, patchlevel.h defines these as PY_MAJOR_VERSION 
and PY_MINOR_VERSION. I don't have PYTHON_MAJOR_VERSION and 
PYTHON_MINOR_VERSION.

The upshot is if I build this way:

  ./compile_leveldb.sh
  python setup.py build

I get a faulty Python leveldb module that segfaults, because the wrong macros 
are getting used.

I'm a complete newb to this, so I have no idea what the appropriate behavior 
is, but could it be possible that the macros are defined wrong? Is something 
different with my system, or is py-leveldb actually broken? I would assume it's 
something with my environment due to the successful use of PyLevelDB elsewhere. 
I'm running standard Ubuntu 12.04, so I can't imagine I'm the only one 
experiencing this.

Can we get this fixed?

Its no where documented that py-leveldb requires ez_setup for installation.

$ pip install leveldb
Downloading/unpacking leveldb
  Running setup.py egg_info for package leveldb
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "/Users/vkapoor/.virtualenvs/test/build/leveldb/setup.py", line 14, in <module>
        import ez_setup
    ImportError: No module named ez_setup
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/Users/vkapoor/.virtualenvs/test/build/leveldb/setup.py", line 14, in <module>

    import ez_setup

ImportError: No module named ez_setup


This should be either clearly documented or the dependency should be removed.

What steps will reproduce the problem?
1. follow installation instructions and run commands up to and including 
"python setup.py build", which produces two error messages
2. continuing with the instructions results in an error message when trying to 
import the module in python (v2.6):
>>> import leveldb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libleveldb.so.1: cannot open shared object file: No such file or 
directory


What is the expected output? What do you see instead?
terminal output:
running build
running build_py
package init file '__init__.py' not found (or not a regular file)
package init file '__init__.py' not found (or not a regular file)
running build_ext


What version of the product are you using? On what operating system?
revision 54, Debian Linux 6.0

Please provide any additional information below.
creating an empty file called __init__.py in the main directory fixed the 
problem.

Implement a torture test for the library, using different operations under 
multiple threads.

http://blog.csdn.net/toontong/article/details/8442995
或是下载连接:
http://download.csdn.net/detail/toontong/4933540

What steps will reproduce the problem?
1. install snappy
2. compile leveldb (notice the -DSNAPPY)
3. python setup.py build
4. import leveldb

What is the expected output?
Just works.

What do you see instead?
ImportError: ./leveldb.so: undefined symbol: _ZN6snappy13RawUncompressEPKcmPc

What version of the product are you using? On what operating system?
r35 on gentoo
I installed snappy from gentoo package-management as static-library.
I updated the leveldb source inside the py-leveldb package to 
42fb47f6edab02d4abb814631c6615dd4661d7d2 but still use your make-options.

Please provide any additional information below.
I "fixed" the ImportError with -lsnappy in the extra_link_args but I guess this 
wouldn't work if a user doesn't has snappy.

extra_link_args = ['-L./leveldb-read-only', '-Bstatic', '-lleveldb', '-lsnappy']

Also I was a bit confused with the documentation about how leveldb uses snappy. 
I thought it would be used on the fly, when I install snappy and didn't expect 
that recompiling the python module was required.. but maybe It's just me :)
Maybe you could also had some boolean like LevelDB.hasSnappy or usesSnappy if 
this is possible.

What is the expected output? What do you see instead?

/usr/include/python2.6/longobject.h:51: warning: ISO C++ 1998 does not support 
‘long long’
leveldb_object.cc: In function ‘int PyLevelDB_Write_(PyLevelDB*, PyObject*, 
PyObject*, const leveldb::Snapshot**)’:
leveldb_object.cc:402: error: ‘struct leveldb::WriteOptions’ has no member 
named ‘post_write_snapshot’
error: command 'gcc' failed with exit status 1

What version of the product are you using? On what operating system?
ubuntu10.04 python2.6

Please provide any additional information below.

System:

    * Ubuntu 11.04 2.6.35-28-generic AMD64 
    * gcc-4.4
    * python-2.6

Install Error:

{{{
leveldb_object.cc:394: warning: deprecated conversion from string constant to 
'char*'
g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions 
build/temp.linux-x86_64-2.6/leveldb_ext.o 
build/temp.linux-x86_64-2.6/leveldb_object.o -lstdc++ -o 
build/lib.linux-x86_64-2.6/leveldb.so -Bdynamic -lm -L./leveldb-read-only 
-Bstatic -lleveldb
/usr/bin/ld.bfd.real: ./leveldb-read-only/libleveldb.a(db_impl.o): relocation 
R_X86_64_32S against `vtable for leveldb::DBImpl::RecoverLogFile(unsigned long, 
leveldb::VersionEdit*, unsigned long*)::LogReporter' can not be used when 
making a shared object; recompile with -fPIC
./leveldb-read-only/libleveldb.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: command 'g++' failed with exit status 1
}}}



SUCCESS when:

adding a -fPIC flag to leveldb

$cd leveldb-read-only
$svn diff
{{{
$ svn diff
Index: Makefile
===================================================================
--- Makefile    (revision 45)
+++ Makefile    (working copy)
@@ -8,7 +8,7 @@
 # Uncomment exactly one of the lines labelled (A), (B), and (C) below
 # to switch between compilation modes.

-OPT = -O2 -DNDEBUG       # (A) Production use (optimized mode)
+OPT = -fPIC -O2 -DNDEBUG       # (A) Production use (optimized mode)
 # OPT = -g2              # (B) Debug mode, w/ full line-level debugging symbols
 # OPT = -O2 -g2 -DNDEBUG # (C) Profiling mode: opt, but w/debugging symbols
}}}

Get a lot of errors when try to build with older versions of Python (tested 
with 2.4, 2.5 and 2.6). Is it possible to make it compatible with older python 
versions ?

I detailed the problems I ran into, and what I had to do to solve them.  It 
looks like this is stuff outside of your control though.  It'd be awesome if 
necessary stuff could be pushed upstream as necessary.  Then leveldb and its 
wrappers could all have great build experiences on OSX.

https://gist.github.com/11add51afc4d02f717e3

It would be extremely convenient to be able to install this from pypi with pip.
Please consider making that possible.

and.. Thanks for the library!

I filing this request in order to get an updated setup.py so it can be compiled 
and installed via pip and shows the right version in pip.

Why not use same license as level DB ?

So commercial use and distribution is permitted.

I had emailed about something like this earlier - where you have an existing 
ldb and you want to write the comparator in Python. I actually have a case 
where some code I have uses the bytewise comparator but changes the comparator 
name, presumably to detect changes in the comparator.

I'm attaching a patch which allows you to pass comparator_name to the LevelDB 
constructor - it does it by writing a hacky wrapper comparator, but I'm hoping 
this can help get things started. 

The LevelDB object should have a method to release the lock on the database.
Currently it seems like the only way to close the database is to delete every 
reference to it so it is garbage collected.

What steps will reproduce the problem?
1../compile-leveldb.sh
2.python setup.py build
3.python setup.py install

What is the expected output? What do you see instead?
build outputs :
running build
running build_py
package init file '__init__.py' not found (or not a regular file)
package init file '__init__.py' not found (or not a regular file)
running build_ext

install outputs:
running install
running build
running build_py
package init file '__init__.py' not found (or not a regular file)
package init file '__init__.py' not found (or not a regular file)
running build_ext
running install_lib
running install_egg_info
Removing 
/home/oleiade/Dev/Envs/sandbox/lib/python2.7/site-packages/leveldb-0.1-py2.7.egg
-info
Writing 
/home/oleiade/Dev/Envs/sandbox/lib/python2.7/site-packages/leveldb-0.1-py2.7.egg
-info

when >>> import leveldb ; leveldb.LevelDB()
should shout, 'need a filename argument', got 
"""leveldb.LevelDB()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'LevelDB'"
instead

What version of the product are you using? On what operating system?
I'm using r48. Was working alright three, or four weeks ago...

What steps will reproduce the problem?
# python setup.py build
running build
running build_py
package init file '__init__.py' not found (or not a regular file)
package init file '__init__.py' not found (or not a regular file)
running build_ext
building 'leveldb' extension
creating build
creating build/temp.linux-x86_64-2.6
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall 
-Wstrict-prototypes -fPIC -I/usr/local/include/python2.6 -c leveldb_ext.cc -o 
build/temp.linux-x86_64-2.6/leveldb_ext.o -I./leveldb-read-only/include -fPIC 
--std=c++0x -pedantic -Wall -g2 -D_GNU_SOURCE -O2 -DNDEBUG
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for 
Ada/C/ObjC but not for C++
cc1plus: error: unrecognized command line option "-std=c++0x"
error: command 'gcc' failed with exit status 1


What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?

from SVN

Please provide any additional information below.

This would break API compatibility, though in a relatively minor way:

My application issues a lot of Gets for keys that don't exist, and handling the 
KeyError takes more time than checking for None.

A simple test (one million inserts; one million Gets of a missing key) has 
exception handling taking 65% longer, 5.22s vs 3.39s.

The only change I made was the simple patch:
--- leveldb_object.cc   (revision 48)
+++ leveldb_object.cc   (working copy)
@@ -271,8 +271,8 @@
    PY_LEVELDB_RELEASE_BUFFER(key);

    if (status.IsNotFound()) {
-       PyErr_SetNone(PyExc_KeyError);
-       return 0;
+       Py_INCREF(Py_None);
+       return Py_None;
    }

    if (!status.ok()) {

What steps will reproduce the problem?
1. run "pip install leveldb" from a fresh virtualenv.

What is the expected output? What do you see instead?

I would expect the package to be installed.

But installation bails out with:

Downloading/unpacking leveldb
  Downloading leveldb-0.192.tar.gz (225kB): 225kB downloaded
  Running setup.py (path:/home/t/py/build/leveldb/setup.py) egg_info for package leveldb
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/home/t/py/build/leveldb/setup.py", line 14, in <module>
        import ez_setup
    ImportError: No module named ez_setup
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/home/t/py/build/leveldb/setup.py", line 14, in <module>

    import ez_setup

ImportError: No module named ez_setup

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in 
/home/t/py/build/leveldb
Storing debug log for failure in /home/t/.pip/pip.log


What version of the product are you using? On what operating system?

debian 7.4, x86_64, python 2.7, pip 1.5.2

Please provide any additional information below.