chtd / psycopg2cffi Goto Github PK

View Code? Open in Web Editor NEW

This project forked from mvantellingen/psycopg2-ctypes

177.0 177.0 44.0 929 KB

Port to cffi with some speed improvements

License: Other

Python 100.00%

psycopg2cffi's People

Contributors

Stargazers

Watchers

psycopg2cffi's Issues

bulk_insert() quite faster, but cursor.execute() much slower

Hi, I report an weird for me issue.
I'm inserting test data into PostgreSQL 9.3 (milions of single, simple rows)

When I'm using bulk_create() method:

insert_list.append(myModel), every 10000 times myModel.objects.bulk_create(insert_list)

performance raises when switching to PyPy 4.0.1 instead CPython 2.7.6 from 11k inserts/s to 17k inserts/s, which is fine ...

But when I'm using cursor_execute() method:

query_base = 'INSERT INTO table (id,vg,vn,vb,pt) VALUES '
.... query += '(%i,%i,%i,%i,0)' % (i, 20+i%30, 10+i%20, i%10)
.... and every 50000 times cursor.execute(query)

performance significally drops from 35k inserts/s to only ... 8k inserts/s.

Can someone check if this is repeatable issue or anyone has idea why it's happening?
The only thing which changes between tests is switching psycopg2 to psycopg2cffi (newest from pip)

register_type for unicode is crashing for 'unknown' type

>>> import psycopg2cffi as psycopg2
>>> import psycopg2cffi.extensions as ext
>>> c = psycopg2.connect(database='postgres')
>>> cur = c.cursor()
>>> ext.register_type(ext.new_type((705,), "UNKNOWN", ext.UNICODE))
>>> cur.execute("SELECT '日本語' AS japanese;")
>>> cur.fetchall()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.
py", line 26, in check_closed_
    return func(self, *args, **kwargs)
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.
py", line 39, in check_no_tuples_
    return func(self, *args, **kwargs)
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.
py", line 373, in fetchall
    return [self._build_row() for _ in xrange(size)]
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.
py", line 884, in _build_row
    self._casts[i], val, length, self)
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/typecas
ts.py", line 70, in typecast
    return caster.cast(value, cursor, length)
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/typecas
ts.py", line 37, in cast
    return self.py_caster(value, cursor)
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/typecas
ts.py", line 38, in cast
    return self.caster(value, length, cursor)
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/site-packages/psycopg2cffi/_impl/typecas
ts.py", line 224, in parse_unicode
    return value.decode(cursor._conn._py_enc) if value is not None else None
  File "/Users/amjith/.virtualenvs/testpsycopgffi/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)
>>>

This works correctly in psycopg2 2.5.4 and returns unicode.

pycache Error

When I try to enter first time to app client (tryton), after to restart server, client fail with next error:

ERROR:tryton.common.common:Traceback (most recent call last):
File "/trytond/protocols/jsonrpc.py", line 162, in _marshaled_dispatch
response['result'] = dispatch_method(method, params)
File "/trytond/protocols/jsonrpc.py", line 191, in _dispatch
res = dispatch(*args)
File "/trytond/protocols/dispatcher.py", line 31, in dispatch
Database = backend.get('Database')
File "/trytond/backend/init.py", line 18, in get
import(modname)
File "/trytond/backend/postgresql/init.py", line 4, in
from .database import *
File "/trytond/backend/postgresql/database.py", line 12, in
from psycopg2cffi import compat
File "/psycopg2cffi/init.py", line 4, in
from psycopg2cffi import extensions
File "/psycopg2cffi/extensions.py", line 39, in
from psycopg2cffi._impl import connection as _connection
File "/psycopg2cffi/_impl/connection.py", line 10, in
from psycopg2cffi._impl import exceptions
File "/psycopg2cffi/_impl/exceptions.py", line 12, in
from psycopg2cffi._impl.libpq import libpq, ffi
File "/psycopg2cffi/_impl/libpq.py", line 257, in
ext_package='psycopg2cffi')
File "/cffi/api.py", line 367, in verify
lib = self.verifier.load_library()
File "/cffi/verifier.py", line 95, in load_library
self._write_source()
File "/cffi/verifier.py", line 183, in _write_source
with open(self.sourcefilename, "w") as fp:
IOError: [Errno 2] No such file or directory: '/psycopg2cffi/_impl/pycache/_cffi__x97b0ad0cxb201248a.c'

[Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/psycopg2cffi/_impl/pycache/_cffi__x97b0ad0cxb201248a.c'

But the second tries not fail and client enter fine, on Xubuntu 15.04.

Version: 2.6.1

Installation problems

For now, psycopg2cffi from master branch works pretty well on CPython 3.4, but the release on PyPI has some issues.
If you have few time, it should be useful for new corners to publish a new release, everybody doesn't have the reflex to test also with the development version when you have some issues with latest stable version.

not installing on win32 (pypy400)

Hello.

First, I need to change the _build_libpq.py like that (adding the last 2 lines):
def find_libpq(self):
path = self.query('libdir')
fname = None
if os.name == 'posix':
if sys.platform == 'darwin':
fname = os.path.join(path, 'libpq.dylib')
if sys.platform in ['linux', 'linux2', 'linux3']:
fname = os.path.join(path, 'libpq.so')
if sys.platform.startswith('freebsd'):
fname = os.path.join(path, 'libpq.so')
elif sys.platform == 'win32':
fname = os.path.join(path, 'libpq.dll')

But, after that, I have an error:

Found libpq at:
-> c:/PROGRA~~1/POSTGR~~1/9.4/lib\libpq.dll

running install
Checking .pth file support in c:\usr\pypy400\site-packages
c:\usr\pypy400\pypy.exe -E -c pass
TEST PASSED: c:\usr\pypy400\site-packages\ appears to support .pth files
running bdist_egg
running egg_info
writing psycopg2cffi.egg-info\PKG-INFO
writing dependency_links to psycopg2cffi.egg-info\dependency_links.txt
writing requirements to psycopg2cffi.egg-info\requires.txt
writing top-level names to psycopg2cffi.egg-info\top_level.txt
reading manifest file 'psycopg2cffi.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'psycopg2cffi.egg-info\SOURCES.txt'
installing library code to build\bdist.win32\egg
running install_lib
running build_py
copying psycopg2cffi_impl_build_libpq.py -> build\lib.win32-2.7\psycopg2cffi_
impl
running build_ext
generating cffi module 'build\temp.win32-2.7\Release\psycopg2cffi._impl._libp
q.c'
already up-to-date
building 'psycopg2cffi._impl._libpq' extension
C:\Users\USER\AppData\Local\Programs\Common\Microsoft\Visual C
++ for Python\9.0\VC\Bin\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ic:/PROGRA
~~1/POSTGR~~1/9.4/include -Ic:\usr\pypy400\include /Tcbuild\temp.win32-2.7\Release
\psycopg2cffi._impl._libpq.c /Fobuild\temp.win32-2.7\Release\build\temp.win32-2.
7\Release\psycopg2cffi._impl._libpq.obj
psycopg2cffi._impl._libpq.c
build\temp.win32-2.7\Release\psycopg2cffi._impl._libpq.c(400) : fatal error C108
3: Cannot open include file: 'stdint.h': No such file or directory
error: command 'C:\Users\USER\AppData\Local\Programs\Com
mon\Microsoft\Visual C++ for Python\9.0\VC\Bin\cl.exe' failed with exit st
atus 2

I have no idea for fixing it.
Best regards

Adapting a DateRange object created with a unicode bounds fails.

In the process of making my code python3 compliant, I turned on unicode literals at the top of a test file:

from __future__ import unicode_literals

Then, when attempting to adapt a DateRange object (but not a Numeric Range) object that I was creating manually, resulted in an error. The line of code that was run:

DjangoFieldsModel.objects.filter(date__in=DateRange(lower='2001-01-01', upper='2001-01-03', bounds='(]'))

I believe it is because of the line:

https://github.com/chtd/psycopg2cffi/blob/master/psycopg2cffi/_range.py#L218

This line uses string formatting to inject the bounds, which, when they are a unicode object, results in that object becoming a unicode object.

This then results in the actual error:

  File "/Users/matt/Development/django-postgres/.tox/pypy/site-packages/psycopg2cffi/_impl/cursor.py", line 687, in _pq_execute
    self._pgres = libpq.PQexec(pgconn, query)
TypeError: initializer for ctype 'char *' must be a str or list or tuple, not unicode

Single row mode support

libpq has single row mode support since 9.2 - this can save memory when loading large datasets - http://www.postgresql.org/docs/9.2/static/libpq-single-row-mode.html

Maybe I will try to first see it it really reduces memory usage. Is is currently implemented in psycopg2? @dvarrazzo

Installing requires cffi<1.0

I am on mac and using pyenv-ed pypy 5.3. I didn't have this problem in my linux box, but only in mac.
Even after uninstalling psycopg2cffi, i am getting same error.

 ✗ pip install --upgrade psycopg2cffi                                               
Collecting psycopg2cffi
Requirement already up-to-date: six in /Volumes/UserSpace/Projects/beat/envpypy/site-packages (from psycopg2cffi)
Collecting cffi<1.0 (from psycopg2cffi)
Requirement already up-to-date: pycparser in /Volumes/UserSpace/Projects/beat/envpypy/site-packages (from cffi<1.0->psycopg2cffi)
Installing collected packages: cffi, psycopg2cffi
  Found existing installation: cffi 1.7.0
    Not uninstalling cffi at /Users/<masked>/.pyenv/versions/pypy-5.3/lib_pypy, outside environment /Volumes/UserSpace/Projects/beat/envpypy
Successfully installed cffi-1.7.0 psycopg2cffi-2.7.4

Infinite loop when result size exceeds Cursor.itersize

A copy of mvantellingen#26

If the number of results the query returns is greater than Cursor.itersize (currently 2000), the call to cursor.iter never returns. If I make it large, say 100000, the call finishes ok.
Traceback:

File "/home/kostia/chtd/netdb_test/netdb_test/pypy-env/src/netdb/netdb/stitcher.py", line 306, in __iter__
   for row in cursor :
File "/home/kostia/chtd/netdb_test/netdb_test/pypy-env/site-packages/psycopg2ct/_impl/cursor.py", line 550, in __iter__
   rows = self.fetchmany(self.itersize)
File "/home/kostia/chtd/netdb_test/netdb_test/pypy-env/site-packages/psycopg2ct/_impl/cursor.py", line 22, in check_closed_
   return func(self, *args, **kwargs)
File "/home/kostia/chtd/netdb_test/netdb_test/pypy-env/site-packages/psycopg2ct/_impl/cursor.py", line 35, in check_no_tuples_
   return func(self, *args, **kwargs)
File "/home/kostia/chtd/netdb_test/netdb_test/pypy-env/site-packages/psycopg2ct/_impl/cursor.py", line 336, in fetchmany
   rows.append(self._build_row(self._rownumber))
File "/home/kostia/chtd/netdb_test/netdb_test/pypy-env/site-packages/psycopg2ct/_impl/cursor.py", line 821, in _build_row
   for i in xrange(n):
KeyboardInterrupt

Run the benchmarks in every Travis build

It can be useful to run the benchmarks in every build to ensure no performance regressions have occured.

Running via source checkout fails (especially tests)

It appears that the change to consulting pg_config breaks running from source checkouts, eg via py.test psycopg2cffi, because PG_INCLUDE_DIR in psycopg2cffi._config is None.

I wouldn't think this is a big deal, because it looks like it does run properly when installed (installed _config is correct for system), but where this becomes more troublesome is running tests seem a lot more painful. I'm not too familiar with py.test, but the source checkout always seems to end up in the search path if I try to use the source checkout's tests, even if I run from a different directory.

With some effort, I've gotten the tests to run from the package install location, but I'm not sure this is desirable, since it's painful without using tox, and it doesn't fix the "real" issue. However, if I'm wrong, and that's the best solution or fixing the root issue is too difficult, I'd be happy to submit a PR (the changes are pretty simple).

For posterity, here is the traceback:

__________________ ERROR collecting psycopg2cffi/tests/test_notify.py ___________________
env/pypy/site-packages/py/_path/local.py:608: in pyimport
    __import__(pkgpath.basename)
psycopg2cffi/__init__.py:4: in <module>
    from psycopg2cffi import extensions
psycopg2cffi/extensions.py:36: in <module>
    from psycopg2cffi._impl import connection as _connection
psycopg2cffi/_impl/connection.py:7: in <module>
    from psycopg2cffi._impl import exceptions
psycopg2cffi/_impl/exceptions.py:1: in <module>
    from psycopg2cffi._impl.libpq import libpq, ffi
psycopg2cffi/_impl/libpq.py:220: in <module>
    ext_package='psycopg2cffi')
/usr/lib/pypy/lib_pypy/cffi/api.py:339: in verify
    lib = self.verifier.load_library()
/usr/lib/pypy/lib_pypy/cffi/verifier.py:74: in load_library
    self._compile_module()
/usr/lib/pypy/lib_pypy/cffi/verifier.py:139: in _compile_module
    outputfilename = ffiplatform.compile(tmpdir, self.get_extension())
/usr/lib/pypy/lib_pypy/cffi/ffiplatform.py:25: in compile
    outputfilename = _build(tmpdir, ext)
/usr/lib/pypy/lib_pypy/cffi/ffiplatform.py:50: in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
E   VerificationError: CompileError: command 'cc' failed with exit status 1
------------------------------------ Captured stderr ------------------------------------
psycopg2cffi/_impl/__pycache__/_cffi__ga4db05c3xe4edfa37.c:26:26: fatal error: postgres_ext.h: No such file or directory
 #include <postgres_ext.h>
                          ^
compilation terminated.
___________ ERROR collecting psycopg2cffi/tests/psycopg2_tests/test_async.py ____________

Unreliable test_cancel

https://travis-ci.org/chtd/psycopg2cffi/jobs/171827862#L866:

test_cancel (psycopg2cffi.tests.psycopg2_tests.test_cancel.CancelTests) ... Exception in thread Thread-1:
Traceback (most recent call last):
  File "/opt/python/2.7.9/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/opt/python/2.7.9/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/tests/psycopg2_tests/test_cancel.py", line 62, in neverending
    conn.rollback()
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 43, in check_closed_
    return func(self, *args, **kwargs)
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 84, in check_async_
    return func(self, *args, **kwargs)
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 63, in check_tpc_
    return func(self, *args, **kwargs)
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 191, in rollback
    self._rollback()
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 756, in _rollback
    self._execute_command('ROLLBACK')
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 661, in _execute_command
    raise exc
QueryCanceledError: canceling statement due to user request
FAIL

...

======================================================================
FAIL: test_cancel (psycopg2cffi.tests.psycopg2_tests.test_cancel.CancelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/tests/psycopg2_tests/testutils.py", line 218, in skip_before_postgres__
    return f(self)
  File "/home/travis/build/chtd/psycopg2cffi/psycopg2cffi/tests/psycopg2_tests/test_cancel.py", line 86, in test_cancel
    self.assertEqual(errors, [])
AssertionError: Lists differ: [QueryCanceledError(u'cancelin... != []
First list contains 1 additional elements.
First extra element 0:
canceling statement due to user request
- [QueryCanceledError(u'canceling statement due to user request\n',)]
+ []
----------------------------------------------------------------------
Ran 480 tests in 15.708s
FAILED (failures=1, skipped=47)

ImportError: No module named psycopg2._psycopg

Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "server.py", line 7, in <module>
    from server.connection_pool import *
  File "/home/erick/0.13.x/server/connection_pool.py", line 1, in <module>
    from pypgwrap.pool import ThreadedConnectionPool
  File "/home/erick/pypy-karoo/site-packages/pypgwrap/__init__.py", line 1, in <module>
    from connection import connection
  File "/home/erick/pypy-karoo/site-packages/pypgwrap/connection.py", line 1, in <module>
    from psycopg2._psycopg import OperationalError
ImportError: No module named psycopg2._psycopg

Python 3 type caster is throwing an error for string casting.

>>> import psycopg2cffi
>>> c = psycopg2cffi.connect(database='postgres')
>>> cur = c.cursor()
>>> cur.execute('create table binarydata(c bytea)')
>>> psycopg2cffi.extensions.register_type(psycopg2cffi.extensions.new_type((17,), 'BYTEA_TEXT', psycopg
2cffi.STRING))
>>> cur.execute("insert into binarydata (c) values (decode('DEADBEEF', 'hex'))")
>>> cur.execute('select * FROM binarydata')
>>> cur.fetchall()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/cursor.py",
 line 30, in check_closed_
    return func(self, *args, **kwargs)
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/cursor.py",
 line 43, in check_no_tuples_
    return func(self, *args, **kwargs)
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/cursor.py",
 line 377, in fetchall
    return [self._build_row() for _ in xrange(size)]
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/cursor.py",
 line 377, in <listcomp>
    return [self._build_row() for _ in xrange(size)]
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/cursor.py",
 line 891, in _build_row
    self._casts[i], val, length, self)
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/typecasts.p
y", line 70, in typecast
    return caster.cast(value, cursor, length)
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/typecasts.p
y", line 37, in cast
    return self.py_caster(value, cursor)
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/typecasts.p
y", line 38, in cast
    return self.caster(value, length, cursor)
  File "/Users/amjith/.virtualenvs/py3_pgcli/lib/python3.3/site-packages/psycopg2cffi/_impl/typecasts.p
y", line 88, in parse_string
    if value is not None else None
AttributeError: 'str' object has no attribute 'decode'
>>>

The error is triggered from https://github.com/chtd/psycopg2cffi/blob/master/psycopg2cffi/_impl/typecasts.py#L87 but I'm not sure what's the correct fix here. I presume there won't be a change between PY2 and PY3 for strings, but that seems wrong, somehow.

Error installing Pypy on Amazon Linux (CentOS)

Traceback (most recent call last):
  File "app_main.py", line 75, in run_toplevel
  File "/var/www/karoo.com.br/server.py", line 7, in <module>
    from server.connection_pool import *
  File "/var/www/karoo.com.br/server/connection_pool.py", line 1, in <module>
    from pypgwrap.pool import ThreadedConnectionPool
  File "/opt/python/run/venv-karoo-pypy/site-packages/pypgwrap/__init__.py", line 1, in <module>
    from connection import connection
  File "/opt/python/run/venv-karoo-pypy/site-packages/pypgwrap/connection.py", line 5, in <module>
    from psycopg2.extras import DictCursor, NamedTupleCursor
  File "/opt/python/run/venv-karoo-pypy/site-packages/newrelic-2.34.0.29/newrelic/api/import_hook.py", line 128, in find_module
    __import__(fullname)
  File "/opt/python/run/venv-karoo-pypy/site-packages/psycopg2.py", line 1, in <module>
    from psycopg2cffi import compat
  File "/opt/python/run/venv-karoo-pypy/site-packages/newrelic-2.34.0.29/newrelic/api/import_hook.py", line 128, in find_module
    __import__(fullname)
  File "/opt/python/run/venv-karoo-pypy/site-packages/psycopg2cffi/__init__.py", line 4, in <module>
    from psycopg2cffi import extensions
  File "/opt/python/run/venv-karoo-pypy/site-packages/newrelic-2.34.0.29/newrelic/api/import_hook.py", line 128, in find_module
    __import__(fullname)
  File "/opt/python/run/venv-karoo-pypy/site-packages/psycopg2cffi/extensions.py", line 36, in <module>
    from psycopg2cffi._impl import connection as _connection
  File "/opt/python/run/venv-karoo-pypy/site-packages/psycopg2cffi/_impl/connection.py", line 7, in <module>
    from psycopg2cffi._impl import exceptions
  File "/opt/python/run/venv-karoo-pypy/site-packages/psycopg2cffi/_impl/exceptions.py", line 1, in <module>
    from psycopg2cffi._impl.libpq import libpq, ffi
  File "/opt/python/run/venv-karoo-pypy/site-packages/psycopg2cffi/_impl/libpq.py", line 220, in <module>
    ext_package='psycopg2cffi')
  File "/opt/pypy-2.4/lib_pypy/cffi/api.py", line 340, in verify
    lib = self.verifier.load_library()
  File "/opt/pypy-2.4/lib_pypy/cffi/verifier.py", line 73, in load_library
    self._write_source()
  File "/opt/pypy-2.4/lib_pypy/cffi/verifier.py", line 125, in _write_source
    file = open(self.sourcefilename, 'w')
IOError: [Errno 2] No such file or directory: '/opt/python/run/venv-karoo-pypy/site-packages/psycopg2cffi/_impl/__pycache__/_cffi__g34a829d3xebeb8df8.c'

When I run my App on this OS I get this error. If I try on Ubuntu run fine.
On CentOS I use https://github.com/squeaky-pl/portable-pypy (64bits)

AssertionError: sorry, but this version only supports 100 named groups

  Running setup.py install for psycopg2cffi: started
    Running setup.py install for psycopg2cffi: finished with status 'error'
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-T6AT89/psycopg2cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-CazIs3-record/install-record.txt --single-version-externally-managed --compile:

    ================================================================================

    Found libpq at:
     -> /usr/pgsql-9.4/lib/libpq.so

    ================================================================================
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-T6AT89/psycopg2cffi/setup.py", line 126, in <module>
        setup(**setup_kwargs)
      File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 272, in __init__
        _Distribution.__init__(self,attrs)
      File "/usr/lib64/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 327, in finalize_options
        ep.load()(self, ep.name, value)
      File "/usr/lib64/python2.7/site-packages/cffi/setuptools_ext.py", line 181, in cffi_modules
        add_cffi_module(dist, cffi_module)
      File "/usr/lib64/python2.7/site-packages/cffi/setuptools_ext.py", line 48, in add_cffi_module
        execfile(build_file_name, mod_vars)
      File "/usr/lib64/python2.7/site-packages/cffi/setuptools_ext.py", line 24, in execfile
        exec(code, glob, glob)
      File "psycopg2cffi/_impl/_build_libpq.py", line 208, in <module>
        _config = PostgresConfig()
      File "psycopg2cffi/_impl/_build_libpq.py", line 20, in __init__
        from psycopg2cffi import _config
      File "/tmp/pip-build-T6AT89/psycopg2cffi/psycopg2cffi/__init__.py", line 4, in <module>
        from psycopg2cffi import extensions
      File "/tmp/pip-build-T6AT89/psycopg2cffi/psycopg2cffi/extensions.py", line 39, in <module>
        from psycopg2cffi._impl import connection as _connection
      File "/tmp/pip-build-T6AT89/psycopg2cffi/psycopg2cffi/_impl/connection.py", line 10, in <module>
        from psycopg2cffi._impl import exceptions
      File "/tmp/pip-build-T6AT89/psycopg2cffi/psycopg2cffi/_impl/exceptions.py", line 12, in <module>
        from psycopg2cffi._impl.libpq import libpq, ffi
      File "/tmp/pip-build-T6AT89/psycopg2cffi/psycopg2cffi/_impl/libpq.py", line 4, in <module>
        from psycopg2cffi._impl._build_libpq import ffi, C_SOURCE, C_SOURCE_KWARGS
      File "/tmp/pip-build-T6AT89/psycopg2cffi/psycopg2cffi/_impl/_build_libpq.py", line 369, in <module>
        ''')
      File "/usr/lib64/python2.7/site-packages/cffi/api.py", line 105, in cdef
        self._cdef(csource, override=override, packed=packed)
      File "/usr/lib64/python2.7/site-packages/cffi/api.py", line 119, in _cdef
        self._parser.parse(csource, override=override, **options)
      File "/usr/lib64/python2.7/site-packages/cffi/cparser.py", line 299, in parse
        self._internal_parse(csource)
      File "/usr/lib64/python2.7/site-packages/cffi/cparser.py", line 304, in _internal_parse
        ast, macros, csource = self._parse(csource)
      File "/usr/lib64/python2.7/site-packages/cffi/cparser.py", line 260, in _parse
        ast = _get_parser().parse(csource)
      File "/usr/lib64/python2.7/site-packages/cffi/cparser.py", line 40, in _get_parser
        _parser_cache = pycparser.CParser()
      File "/usr/lib/python2.7/site-packages/pycparser/c_parser.py", line 87, in __init__
        outputdir=taboutputdir)
      File "/usr/lib/python2.7/site-packages/pycparser/c_lexer.py", line 66, in build
        self.lexer = lex.lex(object=self, **kwargs)
      File "/usr/lib/python2.7/site-packages/pycparser/ply/lex.py", line 911, in lex
        lexobj.readtab(lextab, ldict)
      File "/usr/lib/python2.7/site-packages/pycparser/ply/lex.py", line 233, in readtab
        titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict)))
      File "/usr/lib64/python2.7/re.py", line 190, in compile
        return _compile(pattern, flags)
      File "/usr/lib64/python2.7/re.py", line 240, in _compile
        p = sre_compile.compile(pattern, flags)
      File "/usr/lib64/python2.7/sre_compile.py", line 512, in compile
        "sorry, but this version only supports 100 named groups"
    AssertionError: sorry, but this version only supports 100 named groups

    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-T6AT89/psycopg2cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-CazIs3-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-T6AT89/psycopg2cffi/

Improvement: Wheel-style bdists for Windows on PyPI

pip 1.5.4 now installs wheel by default so wheel-style bdists would allow Windows users to easier install psycopg2cffi. Most Windows machines do not have a compiler which makes source tarball installation problematic.
Notably original psycopg2 doesn't have this support either so it would provide psycopg2cffi a competitive edge against it.

Server-side (named) cursors leak memory

They run in constant space under psycopg2 proper, but leak memory under psycopg2cffi

Problem when importing extras?

Hi there - I'm using psycopg2cffi because I want to use psycopg2 with pypy.

Because psycopg2 doesn't work with pypy I can't install psycopg2 in my virtualenv.
However I can install psycopg2cffi, which is great, so I was hoping I could just change my imports and everything would be ok.

However when I try to import extras I have a problem because in the extras.py of psycopg2cffi there is an
import psycopg2
line, and unfortunately I don't have psycopg2 installed in my virtualenv.

I was just wondering if you could give me any advice on this? I'd love to be able to use psycopg2cffi in my virtualenv if possible because I'm keen to use pypy to speed up my code execution.

Many thanks,

Gareth Williams

Bulk inserts are noticably slower on PyPy

See the logs from a PyPy build and the logs from a CPython 3.4 build.
It's pretty consistent across builds and also reproduces itself in all CPython versions.

Database exceptions does not nave e.pgcoge ang e.pgerror properties.

Psycopg2 has those properties attached to each database exception. This is documented here: http://pythonhosted.org/psycopg2/module.html#exceptions

compatibility with psycopg2 2.6.1

is there a plan to make it compatible with psycopg2 2.6.1 ? there are a few bugs (psycopg/psycopg2#294) that we would love to have in psycopg2cffi

thanks for an awesome library !!

cannot use string() on <cdata 'char *' NULL> error when connecting to amazon redshift

I installed psycopg2cffi from git.

When I tried to connect to my redshift database, I got this error.

(venv)[mwhooker@Wolfe:dev/psycopg2cffi]$ python                                                                                                                                          [master] 17:37:04
Python 2.7.3 (7e4f0faa3d51, Nov 21 2012, 15:59:46)
[PyPy 2.0.0-beta1 with GCC 4.2.1] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``PyPy 1.6 released!''
>>>> from psycopg2cffi import compat
>>>> compat.register()
>>>> import psycopg2
>>>> conn = psycopg2.connect("dbname=*** host=*** port=5439 user=*** password=***")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "psycopg2cffi/__init__.py", line 107, in connect
    connection_factory=connection_factory, async=async)
  File "psycopg2cffi/_impl/connection.py", line 793, in _connect
    return connection_factory(dsn)
  File "psycopg2cffi/_impl/connection.py", line 118, in __init__
    self._connect_sync()
  File "psycopg2cffi/_impl/connection.py", line 134, in _connect_sync
    self._setup()
  File "psycopg2cffi/_impl/connection.py", line 590, in _setup
    libpq.PQparameterStatus(self._pgconn, 'DateStyle'))
  File "/Users/mwhooker/dev/natty/kafka2redshift/venv/site-packages/cffi/api.py", line 220, in string
    return self._backend.string(cdata, maxlen)
RuntimeError: cannot use string() on <cdata 'char *' NULL>

I am able to connect using psycopg2 just fine.

Please let me know if I can provide more information.

Can't install 2.7.2 or 2.7.1 on Ubuntu 15.04 running PyPy 2.6.1

I just updated to PyPy 2.6.1 running on Ubuntu 15.04 using the https://launchpad.net/~pypy/+archive/ubuntu/ppa packages.

Whenever I get a new PyPy from the launchpad repos my virtualenv for the old one stops working. As a result, I was in the process of setting up a new virtualenv for 2.6.1 when I ran into the following:

✘ adamj@Harvester  ~/Development/Tools/Python/VirtualEnv/x86_64  recomed-pypy-2.6.1-kubuntu/bin/pip install psycopg2cffi==2.7.2
Collecting psycopg2cffi==2.7.2
  Using cached psycopg2cffi-2.7.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in ./recomed-pypy-2.6.1-kubuntu/site-packages (from psycopg2cffi==2.7.2)
Requirement already satisfied (use --upgrade to upgrade): cffi>=1.0 in /usr/lib/pypy/lib_pypy (from psycopg2cffi==2.7.2)
Building wheels for collected packages: psycopg2cffi
  Running setup.py bdist_wheel for psycopg2cffi
  Complete output from command /home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/bin/pypy -c "import setuptools;__file__='/tmp/pip-build-rucUeB/psycopg2cffi/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpyuHgGypip-wheel-:

  ================================================================================

  Found libpq at:
   -> /usr/lib/x86_64-linux-gnu/libpq.so

  ================================================================================

  ================================================================================

  Found libpq at:
   -> /usr/lib/x86_64-linux-gnu/libpq.so

  ================================================================================
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/psycopg1.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/errorcodes.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/_json.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/pool.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/compat.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/extensions.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/_range.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/tz.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  copying psycopg2cffi/extras.py -> build/lib.linux-x86_64-2.7/psycopg2cffi
  creating build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/lobject.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/xid.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/typecasts.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/encodings.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/libpq.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/util.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/notify.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/adapters.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/exceptions.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/cursor.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/consts.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/_build_libpq.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  copying psycopg2cffi/_impl/connection.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/_impl
  creating build/lib.linux-x86_64-2.7/psycopg2cffi/tests
  copying psycopg2cffi/tests/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/tests
  copying psycopg2cffi/tests/test_notify.py -> build/lib.linux-x86_64-2.7/psycopg2cffi/tests
  running build_ext
  generating cffi module 'build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.c'
  creating build/temp.linux-x86_64-2.7
  building 'psycopg2cffi._impl._libpq' extension
  creating build/temp.linux-x86_64-2.7/build
  creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
  cc -O2 -fPIC -Wimplicit -I/usr/include/postgresql -I/home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/include -c build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.o
  build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.c:2:20: fatal error: Python.h: No such file or directory
   #include <Python.h>
                      ^
  compilation terminated.
  error: command 'cc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for psycopg2cffi
Failed to build psycopg2cffi
Installing collected packages: psycopg2cffi
  Found existing installation: psycopg2cffi 2.7.0
    Uninstalling psycopg2cffi-2.7.0:
      Successfully uninstalled psycopg2cffi-2.7.0
  Running setup.py install for psycopg2cffi
    Complete output from command /home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/bin/pypy -c "import setuptools, tokenize;__file__='/tmp/pip-build-rucUeB/psycopg2cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-EO7fkF-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/include/site/python2.7/psycopg2cffi:

    ================================================================================

    Found libpq at:
     -> /usr/lib/x86_64-linux-gnu/libpq.so

    ================================================================================

    ================================================================================

    Found libpq at:
     -> /usr/lib/x86_64-linux-gnu/libpq.so

    ================================================================================
    running install
    running build
    running build_py
    running build_ext
    generating cffi module 'build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.c'
    already up-to-date
    building 'psycopg2cffi._impl._libpq' extension
    cc -O2 -fPIC -Wimplicit -I/usr/include/postgresql -I/home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/include -c build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.o
    build/temp.linux-x86_64-2.7/psycopg2cffi._impl._libpq.c:2:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    error: command 'cc' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of psycopg2cffi
Command "/home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/bin/pypy -c "import setuptools, tokenize;__file__='/tmp/pip-build-rucUeB/psycopg2cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-EO7fkF-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/adamj/Development/Tools/Python/VirtualEnv/x86_64/recomed-pypy-2.6.1-kubuntu/include/site/python2.7/psycopg2cffi" failed with error code 1 in /tmp/pip-build-rucUeB/psycopg2cffi

I tested and get the same error with 2.7.1, but 2.7.0 installs fine. My cffi is 1.3.0.

I checked and I definitely have the pypy-dev package installed, so Python.h should be available.

Benchmarks fail to run under PyPy3 which might indicate a bigger problem

See tracebacks here for example.

Rename provided python module to psycopg2

This will make installation easier for django (no need to add a compat call in a separate file). But I am not sure - maybe it may cause inconvinence for others?

Can't install on OSX 10.9.2

I have PostgreSQL 9.3 installed via MacPorts (and pg_config is on my path). I am using PyPy 2.2.1 (also installed from MacPorts). I've tried using pip to install from PyPI, downloading the tarball and running develop / install and cloning the repo and trying it but they all hit this error.

It seems that the setup.py script is erroring out before it gets to the point of calling pg_config:

Thu May  1, 07:15 | /Users/dblewett/src/psycopg2cffi-2.5
kell% pypy setup.py build_py --pg-config=/opt/local/bin/pg_config
psycopg2cffi/_impl/__pycache__/_cffi__g3212cffex2df7bd7b.c:26:10: fatal error: 'postgres_ext.h' file not found
#include <postgres_ext.h>
         ^
1 error generated.
Traceback (most recent call last):
  File "app_main.py", line 72, in run_toplevel
  File "setup.py", line 250, in <module>
    import psycopg2cffi._impl.libpq
  File "/Users/dblewett/src/psycopg2cffi-2.5/psycopg2cffi/__init__.py", line 4, in <module>
    from psycopg2cffi import extensions
  File "/Users/dblewett/src/psycopg2cffi-2.5/psycopg2cffi/extensions.py", line 36, in <module>
    from psycopg2cffi._impl import connection as _connection
  File "/Users/dblewett/src/psycopg2cffi-2.5/psycopg2cffi/_impl/connection.py", line 7, in <module>
    from psycopg2cffi._impl import exceptions
  File "/Users/dblewett/src/psycopg2cffi-2.5/psycopg2cffi/_impl/exceptions.py", line 1, in <module>
    from psycopg2cffi._impl.libpq import libpq, ffi
  File "/Users/dblewett/src/psycopg2cffi-2.5/psycopg2cffi/_impl/libpq.py", line 221, in <module>
    ext_package='psycopg2cffi')
  File "/opt/local/lib/pypy/lib_pypy/cffi/api.py", line 339, in verify
    lib = self.verifier.load_library()
  File "/opt/local/lib/pypy/lib_pypy/cffi/verifier.py", line 74, in load_library
    self._compile_module()
  File "/opt/local/lib/pypy/lib_pypy/cffi/verifier.py", line 139, in _compile_module
    outputfilename = ffiplatform.compile(tmpdir, self.get_extension())
  File "/opt/local/lib/pypy/lib_pypy/cffi/ffiplatform.py", line 25, in compile
    outputfilename = _build(tmpdir, ext)
  File "/opt/local/lib/pypy/lib_pypy/cffi/ffiplatform.py", line 50, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
VerificationError: CompileError: command 'cc' failed with exit status 1

JSONB support?

Does psycopg2cffi have JSONB support? Psycopg2 added it in 2.5.4, but I'm not sure how the versions correspond.

Python 3 support

It would be awesome if psycopg2cffi had python 3 support.

(And if it does the .travis.yml should say so).

Error running tests

Hello,

This import below, generates an error during tests: https://github.com/chtd/psycopg2cffi/blob/master/psycopg2cffi/tests/psycopg2_tests/test_notify.py#L67

I added this 2 lines (below) and it worked. I'm not sure if this is a problem with my environment but it doesn't look like..

from psycopg2cffi import compat
compat.register()

Installing psycopg2cffi under pypy is failing with PyBaseExceptionObject

I'm running tox on a debian 7.9 64 bit env with pypy from pypy-2.6.1-linux_x86_64-portable.
Running tox -e pypy I'm getting a consistent failure while setting up psycopg2cffi with the following

 Running setup.py install for psycopg2
    building 'psycopg2._psycopg' extension
    cc -O2 -fPIC -Wimplicit -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.6.1 (dt dec pq3 ext lo64)" -DPG_VERSION_HEX=0x090309 -DHAVE_LO64=1 -I/home/jeff/lexmachina/deus_lex/.tox/pypy/include -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.3/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
    In file included from psycopg/psycopgmodule.c:38:0:
    ./psycopg/error.h:32:5: error: unknown type name ‘PyBaseExceptionObject’
    error: command 'cc' failed with exit status 1
    Complete output from command /home/jeff/lexmachina/deus_lex/.tox/pypy/bin/pypy -c "import setuptools, tokenize;__file__='/home/jeff/lexmachina/deus_lex/.tox/pypy/build/psycopg2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-FGf5ix-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/jeff/lexmachina/deus_lex/.tox/pypy/include/site/python2.7:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/psycopg2

copying lib/tz.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/extensions.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/errorcodes.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/_json.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/pool.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/psycopg1.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/extras.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/_range.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2

creating build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_with.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_cursor.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/dbapi20.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_bugX000.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_notify.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_module.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_dates.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/testutils.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_quote.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_lobject.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_transaction.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_types_basic.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_cancel.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_async.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_types_extras.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_psycopg2_dbapi20.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/dbapi20_tpc.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_connection.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_extras_dictcursor.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_bug_gc.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_copy.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/testconfig.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_green.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

running build_ext

building 'psycopg2._psycopg' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/psycopg

cc -O2 -fPIC -Wimplicit -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.6.1 (dt dec pq3 ext lo64)" -DPG_VERSION_HEX=0x090309 -DHAVE_LO64=1 -I/home/jeff/lexmachina/deus_lex/.tox/pypy/include -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.3/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement

In file included from psycopg/psycopgmodule.c:38:0:

./psycopg/error.h:32:5: error: unknown type name ‘PyBaseExceptionObject’

error: command 'cc' failed with exit status 1

Some tests are disabled since postgres is not configured to use two-phase transactions

Example: https://travis-ci.org/chtd/psycopg2cffi/jobs/66758351#L562
In order to get full coverage we need to configure Postgres to enable them.
The question is if we need to enable them on the default build or have different build jobs with two phase transactions and without.

Attribute error raised when psycopg2 raises an exception on PyPy 5.0

Here's the traceback:

.tox/pypy-unit/site-packages/pytest_django/fixtures.py:54: in _django_db_setup
    interactive=False)
.tox/pypy-unit/site-packages/django/test/runner.py:726: in setup_databases
    serialize=connection.settings_dict.get("TEST", {}).get("SERIALIZE", True),
.tox/pypy-unit/site-packages/django/db/backends/base/creation.py:56: in create_test_db
    self._create_test_db(verbosity, autoclobber, keepdb)
.tox/pypy-unit/site-packages/django/db/backends/base/creation.py:165: in _create_test_db
    with self._nodb_connection.cursor() as cursor:
.tox/pypy-unit/site-packages/django/db/backends/base/creation.py:29: in _nodb_connection
    return self.connection._nodb_connection
.tox/pypy-unit/site-packages/django/db/backends/postgresql/base.py:239: in _nodb_connection
    nodb_connection.ensure_connection()
.tox/pypy-unit/site-packages/django/db/backends/base/base.py:199: in ensure_connection
    self.connect()
.tox/pypy-unit/site-packages/django/db/backends/base/base.py:171: in connect
    self.connection = self.get_new_connection(conn_params)
.tox/pypy-unit/site-packages/django/db/backends/postgresql/base.py:175: in get_new_connection
    connection = Database.connect(**conn_params)

.tox/pypy-unit/site-packages/psycopg2cffi/__init__.py:104: in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
.tox/pypy-unit/site-packages/psycopg2cffi/_impl/connection.py:884: in _connect
    return connection_factory(dsn)
.tox/pypy-unit/site-packages/psycopg2cffi/_impl/connection.py:134: in __init__
    self._connect_sync()
.tox/pypy-unit/site-packages/psycopg2cffi/_impl/connection.py:143: in _connect_sync
    raise self._create_exception()
AttributeError: 'Connection' object has no attribute '_py_enc'

It seems like it fails to encode the exception message so I'm not sure what the real exception is.

Installation Error

Collecting psycopg2cffi
Using cached psycopg2cffi-2.7.5.tar.gz
Complete output from command python setup.py egg_info:
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing libffi.pc' to the PKG_CONFIG_PATH environment variable No package 'libffi' found Package libffi was not found in the pkg-config search path. Perhaps you should add the directory containing libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing libffi.pc' to the PKG_CONFIG_PATH environment variable No package 'libffi' found Package libffi was not found in the pkg-config search path. Perhaps you should add the directory containing libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.
Traceback (most recent call last):
File "/usr/lib/python3.5/distutils/unixccompiler.py", line 118, in _compile
extra_postargs)
File "/usr/lib/python3.5/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib/python3.5/distutils/spawn.py", line 159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 161, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 147, in call_command
    self.run_command(cmdname)
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
    self.run_command('build_ext')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/build_ext.py", line 75, in run
    _build_ext.run(self)
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
    self.build_extensions()
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 447, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 472, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 532, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.5/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.5/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 157, in save_modules
    yield saved
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 248, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 278, in run
    return func()
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 246, in runner
    _execfile(setup_script, ns)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 47, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-7tyc5k1y/cffi-1.9.1/setup.py", line 202, in <module>
  File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
    raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1101, in run_setup
    run_setup(setup_script, args)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 251, in run_setup
    raise
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 169, in save_modules
    saved_exc.resume()
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 144, in resume
    six.reraise(type, exc, self._tb)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/pkg_resources/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 157, in save_modules
    yield saved
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 248, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 278, in run
    return func()
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 246, in runner
    _execfile(setup_script, ns)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/sandbox.py", line 47, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-7tyc5k1y/cffi-1.9.1/setup.py", line 202, in <module>
  File "/usr/lib/python3.5/distutils/core.py", line 163, in setup
    raise SystemExit("error: " + str(msg))
SystemExit: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-j578fg5p/psycopg2cffi/setup.py", line 126, in <module>
    setup(**setup_kwargs)
  File "/usr/lib/python3.5/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/dist.py", line 315, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 850, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1122, in best_match
    return self.obtain(req, installer)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1134, in obtain
    return installer(requirement)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/dist.py", line 429, in fetch_build_egg
    return cmd.easy_install(req)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 665, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 695, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 876, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1115, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/vineet/env_ltweb/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1103, in run_setup
    raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Push releases to PyPI from Travis build, add wheels

Wheels will allow to avoid issues like #70

version mismatch, 1.3.0 != 1.2.1

I'm trying to setup pypy 2.6.1 with latest psycopg2cffi and Django (on fedora21/centos7) but I'm getting strange error.

Software versions:

Django 1.8.4
Postgres 9.4.4
PyPy from:
https://bitbucket.org/squeaky/portable-pypy/downloads/pypy-2.6.1-linux_x86_64-portable.tar.bz2

Python modules:

$ pip freeze
  backports.ssl-match-hostname==3.4.0.2
  cffi==1.2.1
  decorator==3.4.0
  Django==1.8.4
  django-filter==0.11.0
  djangorestframework==3.2.3
  iniparse==0.4
  Markdown==2.6.2
  psycopg2cffi==2.7.2
  pycparser==2.14
  pycurl==7.19.3.1
  pygobject==3.14.0
  pygpgme==0.3
  pyliblzma==0.5.3
  pyxattr==0.5.3
  rpm-python==4.12.0.1
  six==1.9.0
  slip==0.6.3
  slip.dbus==0.6.3
  urlgrabber==3.10.1
  yum-metadata-parser==1.1.4

Steps to reproduce:

$ export PYTHONPATH=/usr/lib64/python2.7/site-packages:/usr/lib/python2.7/site-packages
$ ./pypy ~/Development/workspace/python/easycollab/manage.py runserver 0.0.0.0:8001

================================================================================

Found libpq at:
 -> /usr/lib64/libpq.so

================================================================================
Traceback (most recent call last):
  File "/home/vukasin/Development/workspace/python/easycollab/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 303, in execute
    settings.INSTALLED_APPS
  File "/usr/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/usr/lib/python2.7/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/lib/python2.7/site-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/home/vukasin/Downloads/pypy-2.6.1-linux_x86_64-portable/lib-python/2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/vukasin/Development/workspace/python/easycollab/easycollab/settings.py", line 15, in <module>
    from psycopg2cffi import compat
  File "/usr/lib64/python2.7/site-packages/psycopg2cffi/__init__.py", line 4, in <module>
    from psycopg2cffi import extensions
  File "/usr/lib64/python2.7/site-packages/psycopg2cffi/extensions.py", line 39, in <module>
    from psycopg2cffi._impl import connection as _connection
  File "/usr/lib64/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 10, in <module>
    from psycopg2cffi._impl import exceptions
  File "/usr/lib64/python2.7/site-packages/psycopg2cffi/_impl/exceptions.py", line 12, in <module>
    from psycopg2cffi._impl.libpq import libpq, ffi
  File "/usr/lib64/python2.7/site-packages/psycopg2cffi/_impl/libpq.py", line 4, in <module>
    from psycopg2cffi._impl._build_libpq import ffi, C_SOURCE, C_SOURCE_KWARGS
  File "/usr/lib64/python2.7/site-packages/psycopg2cffi/_impl/_build_libpq.py", line 210, in <module>
    ffi = FFI()
  File "/usr/lib64/python2.7/site-packages/cffi/api.py", line 59, in __init__
    "version mismatch, %s != %s" % (backend.__version__, __version__)
AssertionError: version mismatch, 1.3.0 != 1.2.1

Wrong handling of postgresqls bigint

If I use bigint, psycopg2cffi only uses unsigned int32. Large numbers become 2147483647.

Exception handler fails when error contains utf8 bytes

psycopg2cffi/psycopg2cffi/_impl/connection.py

Line 828 in ec8a83a

pgmsg = bytes_to_ascii(ffi.string(pgmsg)) if pgmsg else None

  File "/usr/local/lib/pypy2.7/dist-packages/psycopg2cffi/_impl/connection.py", line 828, in _create_exception
   pgmsg = bytes_to_ascii(ffi.string(pgmsg)) if pgmsg else None
 File "/usr/local/lib/pypy2.7/dist-packages/psycopg2cffi/_impl/adapters.py", line 311, in bytes_to_ascii
   return b.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 164: ordinal not in range(128)

Postgres can return utf8 bytes in exceptions, e.g. when a query fails which contains utf8-bytes to be inserted into the database.

With regular psycopg2 I get the full error printed (which contains umlauts from the query).

Bad `copy_from` call doesn't raise `DataError`

Simple example:

import psycopg2
conn = psycopg2.connect("")
c = conn.cursor()
c.execute("CREATE TEMPORARY TABLE v (s int)")
import StringIO
c.copy_from(StringIO.StringIO('3\n4'), table='v')
try:
    c.copy_from(StringIO.StringIO('a'), table='v')
except psycopg2.DataError:
    print "passed!"
else:
    print "failed!"

Can't decode ascii exception messages

Hie,

working with psycopg2cffi backend, in exception database (SQL constrains) can't encode ASCII character.

Is it possible exceptions database to encoding with unicode?

More information:
https://bugs.tryton.org/issue6128

The traceback:

Traceback (most recent call last):
File "/home/xxx/tryton/trytond/trytond/protocols/dispatcher.py", line 160, in _dispatch
result = rpc.result(meth(*c_args, **c_kwargs))
File "/home/xxx/tryton/trytond/trytond/modules/account_parent_code/account.py", line 98, in create
accounts = super(Account, cls).create(vlist)
File "/home/xxx/tryton/trytond/trytond/model/modelsql.py", line 558, in create
exception, values, transaction=transaction)
File "/home/xxx/tryton/trytond/trytond/model/modelsql.py", line 307, in __raise_integrity_error
if name in str(exception):
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 31: ordinal not in range(128)

Blog post not reachable anymore

Your blog post http://chtd.ru/blog/bystraya-rabota-s-postgres-pod-pypy/ seems not reachable at this time, sadly.

typecaster raises malformed array

Looks like typecaster.py expects arrays in the form "{...}". When an array is in the form "[0:x]={...}", which is also valid in postgresql, it raises a dataerror.

Fix hstore tests

As found out by @ratazzi in #37, hstore tests are not run on travis, and enabling them (https://travis-ci.org/chtd/psycopg2cffi/builds/60182117) reveals some failures

cffi 0.7 is on pypi, update docs so people aren't installing from source anymore

Adopt version number compatible with psycopg

Hello,

psycopg2-cffi should have the same version number of psycopg-c to show the user what feature set it implements. So If you merge #10 I suggest you to bump version to 2.5

If bugs are found in -cffi and a new release is necessary before psycopg2-c you can use the ".post" part suggested by PEP 386: 2.5.post1, 2.5.post2 etc.

After the psycopg2 release 2.5.1 you would have psycopg2-cffi 2.5.1 too. If you find a bug you would release 2.5.1.post1, 2.5.1.post2 etc.

ascii decoding for database response

Hi! I'm having an issue, in method "bytes_to_ascii" in _impl/adapters.py it throws an UnicodeDecodeError since database response has special characters (it's not in english), for instance in my error is:

 'ERROR:  llave duplicada viola restricción de unicidad «companies_requiredfield_branch_id_7e08660e8fafa825_uniq»
DETAIL:  Ya existe la llave (branch_id, field_id)=(2, 60).'

This database error is actually an integrity error (because an unique constraint), in my app (written in django) I capture it and send a message to user, but since I get an UnicodeDecodeError instead of an IntegrityError, for now I'll just validate it before trying to save.

I tried changing this line in _impl/adapters.py

return b.decode('ascii')

return b.decode('utf-8')

and it worked

In async mode, connection is not marked as closed when server dies

I'm writing server failure detection code in momoko and hitting the following issue with psycopg2cffi. It works with the regular psycopg2.

The scenario:

Create asynchronous connection to postgres server
Kill postgress server (I mean kill, plain service postgres restart won't do).
Obtain cursor from connection and execute "SELECT 1".
After hitting exception - bug: conn.closed should be > 0, white it's equal 0

psycopg2cffi 2.7.2, Python 2.7, cffi 1.2.1, Postgres 9.3 (same behaviour with 9.2 and 9.4).

Reproduction:

from __future__ import print_function
from psycopg2cffi import compat
compat.register()
import psycopg2
import select
import traceback

def wait(conn):
    while 1:
        state = conn.poll()
        if state == psycopg2.extensions.POLL_OK:
            break
        elif state == psycopg2.extensions.POLL_WRITE:
            select.select([], [conn.fileno()], [])
        elif state == psycopg2.extensions.POLL_READ:
            select.select([conn.fileno()], [], [])
        else:
            raise psycopg2.OperationalError("poll() returned %s" % state)

try:
    aconn = psycopg2.connect(dsn="user=foo password=bar host=127.0.0.1 port=5432", async=1)
    wait(aconn)

    print("Kill postgres (sudo pkill -9 -f postgres) and hit Enter...", end="")
    raw_input()

    acur = aconn.cursor()
    acur.execute("SELECT 1")
    wait(aconn)
except Exception as err:
    print("Execution failed: %s" % err)
    traceback.print_exc()
    print("aconn.closed = %s" % aconn.closed)
else:
    print(acur.fetchone())

And let's run it:

$ python asynclose.py 
Kill postgres (sudo pkill -9 -f postgres) and hit Enter...^Z
[1]+  Stopped                 python asynclose.py
$ sudo pkill -9 -f postgres
Killed
$ fg
python asynclose.py

Execution failed: SSL SYSCALL error: EOF detected

Traceback (most recent call last):
  File "asynclose.py", line 29, in <module>
    wait(aconn)
  File "asynclose.py", line 10, in wait
    state = conn.poll()
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 480, in poll
    res = self._poll_query()
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 534, in _poll_query
    ret = self._poll_advance_read(self._is_busy())
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 774, in _is_busy
    ffi.string(libpq.PQerrorMessage(self._pgconn)))
OperationalError: SSL SYSCALL error: EOF detected

aconn.closed = 0
$

Note the aconn.closed = 0.

The similar code but in synchronous mode works fine:

from __future__ import print_function
from psycopg2cffi import compat
compat.register()
import psycopg2
import traceback

try:
    conn = psycopg2.connect(dsn="user=foo password=bar host=127.0.0.1 port=5432")

    print("Kill postgres (sudo pkill -9 -f postgres) and hit Enter...", end="")
    raw_input()

    cur = conn.cursor()
    cur.execute("SELECT 1")
except Exception as err:
    print("Execution failed: %s" % err)
    traceback.print_exc()
    print("conn.closed = %s" % conn.closed)
else:
    print(acur.fetchone())

And running it:

$PS1='$ '
$ 
$ python synclose.py 
Kill postgres (sudo pkill -9 -f postgres) and hit Enter...^Z
[1]+  Stopped                 python synclose.py
$ sudo pkill -9 -f postgres
Killed
$ fg
python synclose.py

Execution failed: SSL SYSCALL error: EOF detected

Traceback (most recent call last):
  File "synclose.py", line 15, in <module>
    cur.execute("SELECT 1")
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.py", line 30, in check_closed_
    return func(self, *args, **kwargs)
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/cursor.py", line 252, in execute
    conn._begin_transaction()
  File "/home/.../venvs/momoko/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 643, in _begin_transaction
    self._execute_command('BEGIN')
  File "/home/.../venvs/local/lib/python2.7/site-packages/psycopg2cffi/_impl/connection.py", line 660, in _execute_command
    raise exc
DatabaseError: SSL SYSCALL error: EOF detected

conn.closed = 2
$

conn.closed is greater than 0 as expected.

Add psycopg1 backwards compatibility module?

Psycopg2 has a psycopg2.psycopg1 backwards compatibility module for old-ish software which was moved to psycopg2 but not actually ported to it.

The file is fairly short, is there any chance it would be integrated if ported to psycopg2cffi?

Infinite recursion in errorcodes.lookup()

Unsure if globals() changed, or there is a bug in pypy.

pypy 5.3.1

$ import psycopg2cffi.errorcodes as errorcodes
$ errorcodes.lookup('40001')
..
File "/usr/local/site-packages/psycopg2cffi/errorcodes.py", line 47, in lookup
return lookup(code)
RuntimeError: maximum recursion depth exceeded

chtd / psycopg2cffi Goto Github PK

psycopg2cffi's People

Contributors

Stargazers

Watchers

Forkers

psycopg2cffi's Issues

Software versions:

Python modules:

Steps to reproduce:

Recommend Projects

Recommend Topics

Recommend Org

Jobs