igorcoding / asynctnt Goto Github PK

View Code? Open in Web Editor NEW

76.0 9.0 10.0 7.42 MB

A fast Tarantool Database connector for Python/asyncio.

Home Page: https://igorcoding.github.io/asynctnt

License: Apache License 2.0

Makefile 0.33% Python 54.58% Lua 2.23% Shell 0.03% C 6.83% Cython 36.01%

python python3 asyncio async-python database-connector high-performance tarantool

asynctnt's People

Contributors

Stargazers

Watchers

Forkers

theruziev vladimirburash droidroot1995 alekseyl1992 webclinic017 differentialorange avmartugin onurbulbul2 nio-o first-muchacho

asynctnt's Issues

Test case

import asyncio, asynctnt
import asynctnt.exceptions
import os


def sh(cmd):
    return asyncio.get_running_loop().run_in_executor(None, os.system, cmd)


async def run():
    c = asynctnt.Connection(reconnect_timeout=1, connect_timeout=1, request_timeout=1)
    await sh('docker run --rm --name tt -d -p 3301:3301 tarantool/tarantool:1.10.0')
    print('started Tarantool')

    await c.connect()
    print('connected')
    await c.call('box.info')
    print('first call ok')

    await sh('docker rm -fv tt')
    print('stopped Tarantool')

    await sh('docker run --rm --name tt -d -p 3301:3301 tarantool/tarantool:1.10.0')
    print('started Tarantool')

    for _ in range(10):
        try:
            if _ % 2 == 0:
                # The following two lines don't change anything
                await c.connect()
                print('re-connected')

            await c.call('box.info')
            print('subsequent call ok')  # <- This fails when Tarantool is back up
        except asynctnt.exceptions.TarantoolNotConnectedError as e:
            print('subsequent call error:', e)
        await asyncio.sleep(1)


try:
    print('asynctnt', asynctnt.__version__)
    asyncio.run(run())
finally:
    os.system('docker rm -fv tt')

Output:

asynctnt 0.1.13
a1d11906a6e915a19dbf4efed2a5c8f561f4905c587f5e8331579912a55c79e3
started Tarantool
connected
first call ok
Connect to Tarantool[127.0.0.1:3301] failed: CancelledError(). Retrying in 1.000000 seconds
tt
stopped Tarantool
4d568b320b62d10d9053407689dd27b20ff4abd41bb73e3c8c1b6e964878a57e
started Tarantool
re-connected
subsequent call error: Tarantool is not connected
subsequent call error: Tarantool is not connected
re-connected
subsequent call error: Tarantool is not connected
subsequent call error: Tarantool is not connected
re-connected
subsequent call error: Tarantool is not connected
subsequent call error: Tarantool is not connected
re-connected
subsequent call error: Tarantool is not connected
subsequent call error: Tarantool is not connected
re-connected
subsequent call error: Tarantool is not connected
subsequent call error: Tarantool is not connected
tt

Expected behavior

I expect Asynctnt to try to reconnect under the hood when the subsequent call() s are executed without requiring a user to connect() again.

no running event loop

I can't define connection and start functions without running the event loop.

if PY_37:
    __get_running_loop = asyncio.get_running_loop  # by the reason you can't use the module without a running event loop

I want the connection variable to be global and access functions from other modules while the event loop is not running yet
conn = asynctnt.Connection(host='127.0.0.1', port=3301)

At the moment I have to make a crutch out of the initial function:

async def init_conn():
    global conn
    conn = asynctnt.Connection(host='127.0.0.1', port=3301)
    await asyncio.sleep(0)

so that I can access functions from other modules.

the full example which doesn't work without a crutch:

import asyncio
import asynctnt

conn = asynctnt.Connection(host='127.0.0.1', port=3301)

async def insert(space: str, data, replace=False):
    if not conn.is_connected:
        await conn.connect()
    result = await conn.insert(space, list(data), replace=replace)
    return result[0] if len(result) == 1 else result


async def update(space: str, key, ops):
    if not conn.is_connected:
        await conn.connect()
    key = [key] if type(key) is not list else key
    result = await conn.update(space, key, ops)
    return result[0] if len(result) > 0 else None


async def select(space: str, key=None, index='primary', unpack=True):
    if not conn.is_connected:
        await conn.connect()
    key = [key] if type(key) is not list and key else key
    result = await conn.select(space, key=key, index=index)
    return result[0] if len(result) == 1 and unpack else result


async def sql(query, args=None, parse_metadata=False):
    if not conn.is_connected:
        await conn.connect()
    result = await conn.sql(query, args, parse_metadata=parse_metadata)
    return result


async def delete(space: str, key):
    if not conn.is_connected:
        await conn.connect()
    key = [key] if type(key) is not list else key
    await conn.delete(space, key)

Support for decimal and float types for arithmetic operations in update

TypeError when trying to update a numeric or decimal field using an addition or subtraction operation using the Decimal or float type.

Example:
init.lua

box.cfg{listen = 3301}

space_list = box.space

require "os"

if not box.schema.user.exists(os.getenv("TARANTOOL_USER_NAME")) then
    box.schema.user.create(os.getenv("TARANTOOL_USER_NAME"), {password = os.getenv("TARANTOOL_USER_PASSWORD"),
                                             if_not_exists = true})
    box.schema.user.grant(os.getenv("TARANTOOL_USER_NAME"),'read,write,execute,create,drop','universe')
end

if not space_list['customer'] then
    s = box.schema.space.create('customer')

    s:format({
        {name = 'customer_id', type = 'unsigned'},
        {name = 'username', type = 'string', is_nullable=true},
        {name = 'is_active', type = 'boolean'},
        {name = 'balance', type = 'number', is_nullable=true}
    })

    s:create_index('primary', {
        type = 'hash',
        parts = {'customer_id'}
    })
end

example.py

import os
import asynctnt
import asyncio


async def main():
    conn = asynctnt.Connection(
        host='127.0.0.1',
        port=3301,
        username=os.getenv('TARANTOOL_USER_NAME'),
        password=os.getenv('TARANTOOL_USER_PASSWORD')
    )
    await conn.connect()

    result = await conn.update('customer', [1], [('+', 'balance', 23.65)])

    print(f'RESULT: {result}')

    await conn.disconnect()


asyncio.run(main())

stderr

...
File "asynctnt\iproto\db.pyx", line 403, in asynctnt.iproto.protocol.Db.update
File "asynctnt\iproto\db.pyx", line 206, in asynctnt.iproto.protocol.Db._update
File "asynctnt\iproto\protocol.pyx", line 490, in asynctnt.iproto.protocol.BaseProtocol._execute_normal
File "asynctnt\iproto\requests/base.pyx", line 18, in asynctnt.iproto.protocol.BaseRequest.encode
File "asynctnt\iproto\requests/update.pyx", line 7, in asynctnt.iproto.protocol.UpdateRequest.encode_body
File "asynctnt\iproto\requests/update.pyx", line 220, in asynctnt.iproto.protocol.encode_request_update
File "asynctnt\iproto\requests/update.pyx", line 91, in asynctnt.iproto.protocol.encode_update_ops
TypeError: int argument required for Arithmetic and Delete operations

Space '281' does not exist

This happens when trying to connect to a Tarantool instance, which has not finished replication initialisation but has already started listening the socket.

# Python 3.7
# MacOS + Docker for Mac 18.06.0-ce-mac70 (26399)
# pip isntall asynctnt

import asynctnt
import asyncio
import threading
import os
import time


def t():
    time.sleep(0.2)
    os.system(
        'docker run --rm --name tt -d -p 3301:3301 -e "TARANTOOL_REPLICATION=localhost:3301,x:1" tarantool/tarantool:1.10.0'
    )


async def main():
    threading.Thread(target=t, daemon=True).start()
    c = asynctnt.Connection(reconnect_timeout=1, connect_timeout=10, request_timeout=1)
    await c.connect()


try:
    asyncio.run(main())
finally:
    os.system('docker logs tt')
    os.system('docker rm -fv tt')

$ python test.py
Connect to Tarantool[127.0.0.1:3301] failed: ConnectionRefusedError(61, "Connect call failed ('127.0.0.1', 3301)"). Retrying in 1.000000 seconds
455328f55d06caa8bcc85b4d07b0755c3af252d483f6295b0607ef9c6cdfe1d6
Tarantool[127.0.0.1:3301] Schema fetch failed: (36, "Space '281' does not exist")
Creating configuration file: /etc/tarantool/config.yml
Config:
---
pid_file: /var/run/tarantool/tarantool.pid
wal_dir: /var/lib/tarantool
listen: 3301
replication:
- localhost:3301
- x:1
vinyl_dir: /var/lib/tarantool
memtx_dir: /var/lib/tarantool
...

2018-08-07 17:06:39.149 [1] main/101/tarantool-entrypoint.lua C> Tarantool 1.10.0-0-g42612ec04
2018-08-07 17:06:39.149 [1] main/101/tarantool-entrypoint.lua C> log level 5
2018-08-07 17:06:39.150 [1] main/101/tarantool-entrypoint.lua I> mapping 268435456 bytes for memtx tuple arena...
2018-08-07 17:06:39.150 [1] main/101/tarantool-entrypoint.lua I> mapping 134217728 bytes for vinyl tuple arena...
2018-08-07 17:06:39.151 [1] iproto/101/main I> binary: bound to 0.0.0.0:3301
2018-08-07 17:06:39.152 [1] main/104/applier/localhost:3301 I> remote master is 1.10.0 at 127.0.0.1:3301

2018-08-07 17:06:39.158 [1] main/105/applier/x:1 I> can't connect to master
2018-08-07 17:06:39.159 [1] main/105/applier/x:1 coio_task.c:407 !> SystemError getaddrinfo: Name does not resolve: I/O error
2018-08-07 17:06:39.159 [1] main/105/applier/x:1 I> will retry every 1.00 second
tt
Traceback (most recent call last):
  File "test.py", line 26, in <module>
    asyncio.run(main())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 568, in run_until_complete
    return future.result()
  File "test.py", line 22, in main
    await c.connect()
  File "/.virtualenvs/venv-6AWFJD6r/lib/python3.7/site-packages/asynctnt/connection.py", line 289, in connect
    await self.__create_reconnect_coro(True)
  File "/.virtualenvs/venv-6AWFJD6r/lib/python3.7/site-packages/asynctnt/connection.py", line 259, in _connect
    raise e
  File "/.virtualenvs/venv-6AWFJD6r/lib/python3.7/site-packages/asynctnt/connection.py", line 235, in _connect
    await connected_fut
  File "asynctnt/iproto/db.pyx", line 43, in execute
  File "asynctnt/iproto/db.pyx", line 32, in asynctnt.iproto.protocol.Db.execute
asynctnt.exceptions.TarantoolDatabaseError: (36, "Space '281' does not exist")

Auto-reconnect does not work

Code:

import asyncio
import asynctnt
import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())


async def run():
    c = asynctnt.Connection(host='localhost', port=33301)
    await c.connect()
    while 1:
        await c.call('box.info')   # <-- Tarantool instance goes down here

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(run())
except KeyboardInterrupt:
    loop.stop()
    loop.close()

Error:

Traceback (most recent call last):
  File "app.py", line 15, in <module>
    loop.run_until_complete(run())
  File "uvloop/loop.pyx", line 1364, in uvloop.loop.Loop.run_until_complete
  File "app.py", line 11, in run
    await c.call('box.info')
  File "asynctnt/iproto/db.pyx", line 31, in execute (asynctnt/iproto/protocol.c:19633)
asynctnt.exceptions.TarantoolNotConnectedError: Lost connection to Tarantool

Environment:

Python 3.6.4
MacOS 10.13.3

asynctnt==0.1.11
uvloop==0.9.1

feature: Transactions support

Hello there!

Tarantool supports transactions, so it'd be really nice to have this feature in the library

Schema fetch error in case of tuned msgpack Tarantool serializer

Env:

pip3 show asynctnt
Name: asynctnt
Version: 1.2.1

Repro:

box.cfg{listen=3301}
require('msgpack').cfg{encode_use_tostring = true}

Error

Tarantool[127.0.0.1:3301] Schema fetch failed: Lost connection to Tarantool
Future exception was never retrieved
future: <Future finished exception=TarantoolNotConnectedError('Lost connection to Tarantool')>
asynctnt.exceptions.TarantoolNotConnectedError: Lost connection to Tarantool

In the same case, sync tarantool connector works fine

Running pip install asynctnt for python 3.12 results in the following error:
gcc -pthread -B /home/user/anaconda3/envs/broken/compiler_compat -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/user/anaconda3/envs/broken/include -fPIC -O2 -isystem /home/user/anaconda3/envs/broken/include -fPIC -Ithird_party -Iasynctnt/iproto -I/home/user/anaconda3/envs/broken/include/python3.12 -c asynctnt/iproto/protocol.c -o build/temp.linux-x86_64-cpython-312/asynctnt/iproto/protocol.o
asynctnt/iproto/protocol.c: In function ‘__pyx_f_8asynctnt_6iproto_8protocol_11SchemaSpace_get_index’:
asynctnt/iproto/protocol.c:7361:7: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
7361 | __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_logger); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 111, __pyx_L1_error)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/user/anaconda3/envs/broken/include/python3.12/dictobject.h:90,
from /home/user/anaconda3/envs/broken/include/python3.12/Python.h:61,
from asynctnt/iproto/protocol.c:30:
/home/user/anaconda3/envs/broken/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
asynctnt/iproto/protocol.c:7522:7: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
7522 | __Pyx_GetModuleGlobalName(__pyx_t_4, __pyx_n_s_TarantoolSchemaError); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 122, __pyx_L1_error)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/user/anaconda3/envs/broken/include/python3.12/dictobject.h:90,
from /home/user/anaconda3/envs/broken/include/python3.12/Python.h:61,
from asynctnt/iproto/protocol.c:30:
/home/user/anaconda3/envs/broken/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
asynctnt/iproto/protocol.c: In function ‘__pyx_f_8asynctnt_6iproto_8protocol_6Schema_get_or_create_space’:
asynctnt/iproto/protocol.c:8584:7: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
8584 | __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_TarantoolSchemaError); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 160, __pyx_L1_error)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/user/anaconda3/envs/broken/include/python3.12/dictobject.h:90,
from /home/user/anaconda3/envs/broken/include/python3.12/Python.h:61,
from asynctnt/iproto/protocol.c:30:
/home/user/anaconda3/envs/broken/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
asynctnt/iproto/protocol.c: In function ‘__pyx_f_8asynctnt_6iproto_8protocol_6Schema_create_dummy_space’:
asynctnt/iproto/protocol.c:8731:3: warning: ‘ma_version_tag’ is deprecated [-Wdeprecated-declarations]
8731 | __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_logger); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 169, __pyx_L1_error)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/user/anaconda3/envs/broken/include/python3.12/dictobject.h:90,
from /home/user/anaconda3/envs/broken/include/python3.12/Python.h:61,
from asynctnt/iproto/protocol.c:30:
/home/user/anaconda3/envs/broken/include/python3.12/cpython/dictobject.h:22:34: note: declared here
22 | Py_DEPRECATED(3.12) uint64_t ma_version_tag;
| ^~~~~~~~~~~~~~
asynctnt/iproto/protocol.c: In function ‘__pyx_f_8asynctnt_6iproto_8protocol_6Schema_parse_space’:
asynctnt/iproto/protocol.c:8969:3: warning: ‘Py_OptimizeFlag’ is deprecated [-Wdeprecated-declarations]
8969 | if (unlikely(!Py_OptimizeFlag)) {
| ^~
...
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for asynctnt
Running setup.py clean for asynctnt
Failed to build asynctnt

Datetime fails on modern 2.10

Tests fails in CI with Tarantool 2.10.1

ERROR: test__ext_datetime_write_before_1970 (test_mp_ext.MpExtTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/asynctnt/asynctnt/asynctnt/_testbase.py", line 67, in wrapper
    self.loop.run_until_complete(__meth__(self, *args, **kwargs))
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/runner/work/asynctnt/asynctnt/asynctnt/_testbase.py", line 293, in wrap
    return await res
  File "/home/runner/work/asynctnt/asynctnt/tests/test_mp_ext.py", line 223, in test__ext_datetime_write_before_1970
    resp = await self.conn.insert(sp, [1, dt])
asynctnt.exceptions.TarantoolDatabaseError: Invalid MsgPack - packet body

See https://github.com/DifferentialOrange/asynctnt/runs/7879026191?check_suite_focus=true

Add support for watchers

Related resources:

Failed to compile on Windows

I am on Windows 10 with Windows SDK 10.0.10240.0 installed.
After pip install asynctnt everything going fine until:
creating build\temp.win-amd64-3.6\Release\third_party\msgpuck
Than I got this error in console:

 C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I-Ithird_party "-Ic:\program files\python36\include" "-Ic:\program files\python36\include" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\winrt" /Tcasynctnt/iproto/protocol.c /Fobuild\temp.win-amd64-3.6\Release\asynctnt/iproto/protocol.obj
    protocol.c
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(174): error C2061: syntax error: identifier '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(174): error C2059: syntax error: ';'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(235): error C2079: 'packed' uses undefined struct '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(235): error C2146: syntax error: missing ';' before identifier 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(235): error C2065: 'cast': undeclared identifier
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(235): error C2143: syntax error: missing ';' before '{'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(235): error C2037: left of 'val' specifies undefined struct/union 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(239): error C2079: 'packed' uses undefined struct '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(239): error C2146: syntax error: missing ';' before identifier 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(239): error C2065: 'cast': undeclared identifier
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(239): error C2143: syntax error: missing ';' before '{'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(239): error C2037: left of 'val' specifies undefined struct/union 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(240): error C2079: 'packed' uses undefined struct '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(240): error C2146: syntax error: missing ';' before identifier 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(240): error C2065: 'cast': undeclared identifier
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(240): error C2143: syntax error: missing ';' before '{'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(240): error C2037: left of 'val' specifies undefined struct/union 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(241): error C2079: 'packed' uses undefined struct '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(241): error C2146: syntax error: missing ';' before identifier 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(241): error C2065: 'cast': undeclared identifier
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(241): error C2143: syntax error: missing ';' before '{'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(241): error C2037: left of 'val' specifies undefined struct/union 'cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(266): error C2061: syntax error: identifier 'mp_float_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(266): error C2059: syntax error: ';'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(266): error C2449: found '{' at file scope (missing function header?)
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(269): error C2059: syntax error: '}'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(288): error C2079: 'cast' uses undefined union 'mp_float_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(290): error C2224: left of '.u32' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(291): error C2224: left of '.f' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(291): warning C4033: 'mp_load_float' must return a value
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(288): error C2440: 'initializing': cannot convert from 'mp_float_cast' to 'int'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(297): error C2079: 'cast' uses undefined union 'mp_double_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(299): error C2224: left of '.u64' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(300): error C2224: left of '.d' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(300): warning C4033: 'mp_load_double' must return a value
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(297): error C2440: 'initializing': cannot convert from 'mp_double_cast' to 'int'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(306): error C2079: 'cast' uses undefined union 'mp_float_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(307): error C2224: left of '.f' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(308): error C2224: left of '.u32' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(309): error C2440: '=': cannot convert from 'int' to 'mp_float_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(316): error C2079: 'cast' uses undefined union 'mp_double_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(317): error C2224: left of '.d' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(318): error C2224: left of '.u64' must have struct/union type
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(319): error C2440: '=': cannot convert from 'int' to 'mp_double_cast'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(373): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(373): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(374): error C2143: syntax error: missing ')' before 'type'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(374): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(374): error C2085: 'mp_typeof': not in formal parameter list
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(383): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(383): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(384): error C2146: syntax error: missing ')' before identifier 'mp_sizeof_array'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(384): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(384): error C2082: redefinition of formal parameter '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(384): error C2146: syntax error: missing ',' before identifier 'mp_sizeof_array'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(384): error C2143: syntax error: missing ';' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(384): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(426): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(426): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(426): error C2146: syntax error: missing ')' before identifier 'ptrdiff_t'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(426): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(427): error C2085: 'mp_check_array': not in formal parameter list
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(439): error C2085: 'mp_decode_array': not in formal parameter list
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(448): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(448): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(449): error C2146: syntax error: missing ')' before identifier 'mp_sizeof_map'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(449): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(449): error C2082: redefinition of formal parameter '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(449): error C2146: syntax error: missing ',' before identifier 'mp_sizeof_map'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(449): error C2143: syntax error: missing ';' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(449): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(497): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(497): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(497): error C2146: syntax error: missing ')' before identifier 'ptrdiff_t'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(497): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(498): error C2085: 'mp_check_map': not in formal parameter list
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(510): error C2085: 'mp_decode_map': not in formal parameter list
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(527): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(527): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(528): error C2146: syntax error: missing ')' before identifier 'mp_sizeof_uint'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(528): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(528): error C2082: redefinition of formal parameter '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(528): error C2146: syntax error: missing ',' before identifier 'mp_sizeof_uint'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(528): error C2143: syntax error: missing ';' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(528): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(538): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(538): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(539): error C2146: syntax error: missing ')' before identifier 'mp_sizeof_int'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(539): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(539): error C2061: syntax error: identifier 'mp_sizeof_int'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(539): error C2059: syntax error: ';'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(539): error C2059: syntax error: '<parameter-list>'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(575): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(575): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(575): error C2146: syntax error: missing ')' before identifier 'ptrdiff_t'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(575): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(576): error C2085: 'mp_check_uint': not in formal parameter list
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): error C2143: syntax error: missing ')' before '('
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): error C2059: syntax error: ')'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): error C2146: syntax error: missing ')' before identifier 'ptrdiff_t'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): error C2091: function returns function
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): error C2082: redefinition of formal parameter '__attribute__'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): error C2146: syntax error: missing ',' before identifier 'ptrdiff_t'
    c:\users\valentinmk\appdata\local\temp\pip-build-b_fc1cix\asynctnt\asynctnt\iproto\../../third_party/msgpuck/msgpuck.h(587): fatal error C1003: error count exceeds 100; stopping compilation
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2

Broken links "Edit on Github" in the documentation

Documentation page contains a link "Edit on Github" that is broken (404) - https://github.com/igorcoding/asynctnt/blob/v2/docs/sql.md.

Session push hangs on reconnect

After asynctnt reconnected to Tarantool, PushIterator is blocked forever on waiting asyncio.Event.

The only solution I've come up with is constantly check id of connection._transport and once it's changed recreate PushIterator, eg:

init.lua

box.cfg{listen=3301}
push = require('push')

push.lua

local fiber = require('fiber')

local x = 0

local listen = function()
    while true do
        fiber.sleep(2)
        x = x + 1
        box.session.push(x)
    end
end

return {
    listen = listen
}

asynctnt client

import asyncio
import asynctnt


def get_push_iterator(connection):
    fut = connection.call("push.listen", push_subscribe=True)
    return asynctnt.PushIterator(fut)


async def main():
    async with asynctnt.Connection(port=3301) as conn:

        it = get_push_iterator(conn)
        transport_id = id(conn._transport)

        while True:
            if (current_transport_id := id(conn._transport)) != transport_id:
                transport_id = current_transport_id
                it = get_push_iterator(conn)

            try:
                result = await asyncio.wait_for(it.__anext__(), timeout=10)
            except asyncio.exceptions.TimeoutError:
                pass


if __name__ == "__main__":
    asyncio.run(main())

Any thoughts on this?

Add support for MP_INTERVAL type

Docs: https://github.com/tarantool/tarantool/wiki/Datetime-Internals#intervals-values-encoding

SQL disable upprecase conversion

Simple query:

SELECT COUNT(favorite) FROM "table1" WHERE  favorite = 1;

got exception:

asynctnt.exceptions.TarantoolDatabaseError: (176, "Can’t resolve field 'FAVORITE'")

I know that can do like that and it will be ok:

SELECT COUNT("favorite") FROM "table1" WHERE  "favorite" = 1;

but I would like good syntax, no conversion to uppercase, does it make sense?

A problem with reconnect

When a Tarantool instance I am connected to with asynctnt crashes the Connection.connect() method blocks forever despite connect_timeout is set.

# pip install asynctnt

import asynctnt
import asyncio
import logging
from asynctnt.exceptions import TarantoolNotConnectedError

logging.basicConfig()


async def main():
    c = asynctnt.Connection(
        host='localhost',
        port=3301,
        connect_timeout=5,
        request_timeout=5,
        reconnect_timeout=0,
    )
    try:
        while True:
            if not c.is_connected:
                await c.connect()  # <------------- Hangs here after the Tarantool instance crashes
                print('connected')
            try:
                input('press any key to segfalt...')
                await c.eval('''require('ffi').cast('char *', 0)[0] = 48''')
            except TarantoolNotConnectedError as e:
                print('EXCEPTION:', e.__class__.__name__, e)
    finally:
        await c.disconnect()
        print('disconnected')


asyncio.run(main())

Output

$ python t.py
connected
press any key to continue...
EXCEPTION: TarantoolNotConnectedError Lost connection to Tarantool


^Cdisconnected
Traceback (most recent call last):
  File "t.py", line 33, in <module>
    asyncio.run(main())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 555, in run_until_complete
    self.run_forever()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 523, in run_forever
    self._run_once()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 1722, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/selectors.py", line 558, in select
    kev_list = self._selector.control(None, max_ev, timeout)
KeyboardInterrupt

Python 3.7.0 on MacOS

Implement index-agnostic Response

asynctnt.Response's entries should be agnostic to indexing (either by an index or by a key).
Almost everything is there for implementing it (fields are already provided for each tuple in response), so it should be fairly easy to implement lazy build of dictionaries. body property should be deprecated.
Response should in some way contain tuples that can be accessed either by a key or a numeric index.

This produced some questions with non-space requests (such as call and eval), because there are no actual "tuples" (in sense of space) there.

By implementing this feature parameter tuple_as_dict to all requests would be also declared as deprecated and removed in the future.

Probably should wait until SQL protocol in Tarantool is documented to implement more general solution.

Add support for graceful shutdown

Related resources:

tarantool/doc#2633 (comment)

Select by ALL iterator if no key provided

Select by empty key fails for a HASH index when using EQ iterator.

Essentially, need to add checks to Db.select() function that check if key is empty and no iterator provided and then change iterator type to ALL.

The results of select() and call() function are different

Test space:

box.cfg{}
box.schema.create_space('test', { if_not_exists = true })
box.space.test:create_index('primary', { type = 'TREE', unique = true, if_not_exists = true })

-- function to search for multiple keys
function find(space, keys)
    local result = {}
    for i, key in pairs(keys) do
        result[i] = box.space[space]:select({key})[1]
    end
    return result
end

Tarantool prints the same results:

localhost:3301> box.space.test:select()
---
- - [20]
  - [42]
...

localhost:3301> find('test', {20,42})
---
- - [20]
  - [42]
...

Python test code snippet:

import asyncio
import asynctnt

loop = asyncio.get_event_loop()
tnt_conn = asynctnt.Connection()


async def tasks():
    if not tnt_conn.is_connected:
        await tnt_conn.connect()
    await tnt_conn.insert('test', [20], replace=True)
    await tnt_conn.insert('test', [42], replace=True)
    result_1 = await tnt_conn.select('test')
    print('result_1', result_1)
    result_2 = await tnt_conn.call('find', ['test', [20, 42]])
    print('result_2', result_2)


loop.run_until_complete(tasks())

asynctnt prints different results:

result_1 <Response sync=6 rowcount=2 data=[<TarantoolTuple 0=20>, <TarantoolTuple 0=42>]>
result_2 <Response sync=7 rowcount=1 data=[[[20], [42]]]>

Support encrypted binary protocol (Tarantool Enterprise)

The connector should be able to connect to Tarantool Enterprise using the encrypted protocol.

Documentation: https://www.tarantool.io/en/enterprise_doc/security/#enterprise-iproto-encryption

Similar issues in other Tarantool connectors:

(We plan to implement it in Q3'2022. The issue is filed for planning/tracking purposes.)

Installation fails on Python 3.7

root@5d91fb6a7515:/app# pip -V
pip 18.0 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
root@5d91fb6a7515:/app# python -V
Python 3.7.0
root@5d91fb6a7515:/app# pip install asynctnt
Looking in indexes: http://pypi.k.avito.ru/pypi/
Collecting asynctnt
  Downloading https://files.pythonhosted.org/packages/1d/7f/a5be00488bd40ee55d15eae3f0b71db796779b4e24f22a1241c2872f5641/asynctnt-0.1.11.tar.gz (268kB)
    100% |████████████████████████████████| 276kB 2.4MB/s
Collecting PyYAML>=3.12 (from asynctnt)
  Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
    100% |████████████████████████████████| 276kB 4.0MB/s
Building wheels for collected packages: asynctnt, PyYAML
  Running setup.py bdist_wheel for asynctnt ... error
  Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-vo45rneu/asynctnt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-7f0xaqx7 --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/asynctnt
  copying asynctnt/instance.py -> build/lib.linux-x86_64-3.7/asynctnt
  copying asynctnt/__init__.py -> build/lib.linux-x86_64-3.7/asynctnt
  copying asynctnt/connection.py -> build/lib.linux-x86_64-3.7/asynctnt
  copying asynctnt/log.py -> build/lib.linux-x86_64-3.7/asynctnt
  copying asynctnt/exceptions.py -> build/lib.linux-x86_64-3.7/asynctnt
  copying asynctnt/_testbase.py -> build/lib.linux-x86_64-3.7/asynctnt
  running egg_info
  writing asynctnt.egg-info/PKG-INFO
  writing dependency_links to asynctnt.egg-info/dependency_links.txt
  writing requirements to asynctnt.egg-info/requires.txt
  writing top-level names to asynctnt.egg-info/top_level.txt
  reading manifest file 'asynctnt.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching '*.h' under directory 'asynctnt'
  writing manifest file 'asynctnt.egg-info/SOURCES.txt'
  creating build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/__init__.py -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/buffer.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/buffer.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/cmsgpuck.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/const.pxi -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/coreproto.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/coreproto.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/db.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/db.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/protocol.c -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/protocol.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/protocol.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/python.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/rbuffer.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/rbuffer.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/request.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/request.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/response.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/response.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/schema.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/schema.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/tnt.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/unicode.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  copying asynctnt/iproto/unicode.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
  running build_ext
  building 'asynctnt.iproto.protocol' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/asynctnt
  creating build/temp.linux-x86_64-3.7/asynctnt/iproto
  creating build/temp.linux-x86_64-3.7/third_party
  creating build/temp.linux-x86_64-3.7/third_party/msgpuck
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I-Ithird_party -I/usr/local/include/python3.7m -c asynctnt/iproto/protocol.c -o build/temp.linux-x86_64-3.7/asynctnt/iproto/protocol.o
  asynctnt/iproto/protocol.c: In function ‘__Pyx_PyCFunction_FastCall’:
  asynctnt/iproto/protocol.c:35944:12: error: too many arguments to function ‘(struct PyObject * (*)(struct PyObject *, struct PyObject * const*, Py_ssize_t))meth’
       return (*((__Pyx_PyCFunctionFast)meth)) (self, args, nargs, NULL);
              ^
  asynctnt/iproto/protocol.c: In function ‘__Pyx__ExceptionSave’:
  asynctnt/iproto/protocol.c:36254:19: error: ‘PyThreadState’ has no member named ‘exc_type’
       *type = tstate->exc_type;
                     ^
  asynctnt/iproto/protocol.c:36255:20: error: ‘PyThreadState’ has no member named ‘exc_value’
       *value = tstate->exc_value;
                      ^
  asynctnt/iproto/protocol.c:36256:17: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       *tb = tstate->exc_traceback;
                   ^
  asynctnt/iproto/protocol.c: In function ‘__Pyx__ExceptionReset’:
  asynctnt/iproto/protocol.c:36263:22: error: ‘PyThreadState’ has no member named ‘exc_type’
       tmp_type = tstate->exc_type;
                        ^
  asynctnt/iproto/protocol.c:36264:23: error: ‘PyThreadState’ has no member named ‘exc_value’
       tmp_value = tstate->exc_value;
                         ^
  asynctnt/iproto/protocol.c:36265:20: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       tmp_tb = tstate->exc_traceback;
                      ^
  asynctnt/iproto/protocol.c:36266:11: error: ‘PyThreadState’ has no member named ‘exc_type’
       tstate->exc_type = type;
             ^
  asynctnt/iproto/protocol.c:36267:11: error: ‘PyThreadState’ has no member named ‘exc_value’
       tstate->exc_value = value;
             ^
  asynctnt/iproto/protocol.c:36268:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       tstate->exc_traceback = tb;
             ^
  asynctnt/iproto/protocol.c: In function ‘__Pyx__GetException’:
  asynctnt/iproto/protocol.c:36323:22: error: ‘PyThreadState’ has no member named ‘exc_type’
       tmp_type = tstate->exc_type;
                        ^
  asynctnt/iproto/protocol.c:36324:23: error: ‘PyThreadState’ has no member named ‘exc_value’
       tmp_value = tstate->exc_value;
                         ^
  asynctnt/iproto/protocol.c:36325:20: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       tmp_tb = tstate->exc_traceback;
                      ^
  asynctnt/iproto/protocol.c:36326:11: error: ‘PyThreadState’ has no member named ‘exc_type’
       tstate->exc_type = local_type;
             ^
  asynctnt/iproto/protocol.c:36327:11: error: ‘PyThreadState’ has no member named ‘exc_value’
       tstate->exc_value = local_value;
             ^
  asynctnt/iproto/protocol.c:36328:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       tstate->exc_traceback = local_tb;
             ^
  asynctnt/iproto/protocol.c: In function ‘__Pyx__ExceptionSwap’:
  asynctnt/iproto/protocol.c:36858:22: error: ‘PyThreadState’ has no member named ‘exc_type’
       tmp_type = tstate->exc_type;
                        ^
  asynctnt/iproto/protocol.c:36859:23: error: ‘PyThreadState’ has no member named ‘exc_value’
       tmp_value = tstate->exc_value;
                         ^
  asynctnt/iproto/protocol.c:36860:20: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       tmp_tb = tstate->exc_traceback;
                      ^
  asynctnt/iproto/protocol.c:36861:11: error: ‘PyThreadState’ has no member named ‘exc_type’
       tstate->exc_type = *type;
             ^
  asynctnt/iproto/protocol.c:36862:11: error: ‘PyThreadState’ has no member named ‘exc_value’
       tstate->exc_value = *value;
             ^
  asynctnt/iproto/protocol.c:36863:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
       tstate->exc_traceback = *tb;
             ^
  asynctnt/iproto/protocol.c: In function ‘__Pyx__ReturnWithStopIteration’:
  asynctnt/iproto/protocol.c:37958:22: error: ‘PyThreadState’ has no member named ‘exc_type’
       if (!__pyx_tstate->exc_type) {
                        ^
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for asynctnt
  Running setup.py clean for asynctnt
  Running setup.py bdist_wheel for PyYAML ... done
  Stored in directory: /root/.cache/pip/wheels/ad/da/0c/74eb680767247273e2cf2723482cb9c924fe70af57c334513f
Successfully built PyYAML
Failed to build asynctnt
Installing collected packages: PyYAML, asynctnt
  Running setup.py install for asynctnt ... error
    Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-vo45rneu/asynctnt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-8437v6y1/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/asynctnt
    copying asynctnt/instance.py -> build/lib.linux-x86_64-3.7/asynctnt
    copying asynctnt/__init__.py -> build/lib.linux-x86_64-3.7/asynctnt
    copying asynctnt/connection.py -> build/lib.linux-x86_64-3.7/asynctnt
    copying asynctnt/log.py -> build/lib.linux-x86_64-3.7/asynctnt
    copying asynctnt/exceptions.py -> build/lib.linux-x86_64-3.7/asynctnt
    copying asynctnt/_testbase.py -> build/lib.linux-x86_64-3.7/asynctnt
    running egg_info
    writing asynctnt.egg-info/PKG-INFO
    writing dependency_links to asynctnt.egg-info/dependency_links.txt
    writing requirements to asynctnt.egg-info/requires.txt
    writing top-level names to asynctnt.egg-info/top_level.txt
    reading manifest file 'asynctnt.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.h' under directory 'asynctnt'
    writing manifest file 'asynctnt.egg-info/SOURCES.txt'
    creating build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/__init__.py -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/buffer.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/buffer.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/cmsgpuck.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/const.pxi -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/coreproto.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/coreproto.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/db.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/db.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/protocol.c -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/protocol.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/protocol.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/python.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/rbuffer.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/rbuffer.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/request.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/request.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/response.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/response.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/schema.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/schema.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/tnt.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/unicode.pxd -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    copying asynctnt/iproto/unicode.pyx -> build/lib.linux-x86_64-3.7/asynctnt/iproto
    running build_ext
    building 'asynctnt.iproto.protocol' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/asynctnt
    creating build/temp.linux-x86_64-3.7/asynctnt/iproto
    creating build/temp.linux-x86_64-3.7/third_party
    creating build/temp.linux-x86_64-3.7/third_party/msgpuck
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I-Ithird_party -I/usr/local/include/python3.7m -c asynctnt/iproto/protocol.c -o build/temp.linux-x86_64-3.7/asynctnt/iproto/protocol.o
    asynctnt/iproto/protocol.c: In function ‘__Pyx_PyCFunction_FastCall’:
    asynctnt/iproto/protocol.c:35944:12: error: too many arguments to function ‘(struct PyObject * (*)(struct PyObject *, struct PyObject * const*, Py_ssize_t))meth’
         return (*((__Pyx_PyCFunctionFast)meth)) (self, args, nargs, NULL);
                ^
    asynctnt/iproto/protocol.c: In function ‘__Pyx__ExceptionSave’:
    asynctnt/iproto/protocol.c:36254:19: error: ‘PyThreadState’ has no member named ‘exc_type’
         *type = tstate->exc_type;
                       ^
    asynctnt/iproto/protocol.c:36255:20: error: ‘PyThreadState’ has no member named ‘exc_value’
         *value = tstate->exc_value;
                        ^
    asynctnt/iproto/protocol.c:36256:17: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         *tb = tstate->exc_traceback;
                     ^
    asynctnt/iproto/protocol.c: In function ‘__Pyx__ExceptionReset’:
    asynctnt/iproto/protocol.c:36263:22: error: ‘PyThreadState’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                          ^
    asynctnt/iproto/protocol.c:36264:23: error: ‘PyThreadState’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                           ^
    asynctnt/iproto/protocol.c:36265:20: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                        ^
    asynctnt/iproto/protocol.c:36266:11: error: ‘PyThreadState’ has no member named ‘exc_type’
         tstate->exc_type = type;
               ^
    asynctnt/iproto/protocol.c:36267:11: error: ‘PyThreadState’ has no member named ‘exc_value’
         tstate->exc_value = value;
               ^
    asynctnt/iproto/protocol.c:36268:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tstate->exc_traceback = tb;
               ^
    asynctnt/iproto/protocol.c: In function ‘__Pyx__GetException’:
    asynctnt/iproto/protocol.c:36323:22: error: ‘PyThreadState’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                          ^
    asynctnt/iproto/protocol.c:36324:23: error: ‘PyThreadState’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                           ^
    asynctnt/iproto/protocol.c:36325:20: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                        ^
    asynctnt/iproto/protocol.c:36326:11: error: ‘PyThreadState’ has no member named ‘exc_type’
         tstate->exc_type = local_type;
               ^
    asynctnt/iproto/protocol.c:36327:11: error: ‘PyThreadState’ has no member named ‘exc_value’
         tstate->exc_value = local_value;
               ^
    asynctnt/iproto/protocol.c:36328:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tstate->exc_traceback = local_tb;
               ^
    asynctnt/iproto/protocol.c: In function ‘__Pyx__ExceptionSwap’:
    asynctnt/iproto/protocol.c:36858:22: error: ‘PyThreadState’ has no member named ‘exc_type’
         tmp_type = tstate->exc_type;
                          ^
    asynctnt/iproto/protocol.c:36859:23: error: ‘PyThreadState’ has no member named ‘exc_value’
         tmp_value = tstate->exc_value;
                           ^
    asynctnt/iproto/protocol.c:36860:20: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tmp_tb = tstate->exc_traceback;
                        ^
    asynctnt/iproto/protocol.c:36861:11: error: ‘PyThreadState’ has no member named ‘exc_type’
         tstate->exc_type = *type;
               ^
    asynctnt/iproto/protocol.c:36862:11: error: ‘PyThreadState’ has no member named ‘exc_value’
         tstate->exc_value = *value;
               ^
    asynctnt/iproto/protocol.c:36863:11: error: ‘PyThreadState’ has no member named ‘exc_traceback’
         tstate->exc_traceback = *tb;
               ^
    asynctnt/iproto/protocol.c: In function ‘__Pyx__ReturnWithStopIteration’:
    asynctnt/iproto/protocol.c:37958:22: error: ‘PyThreadState’ has no member named ‘exc_type’
         if (!__pyx_tstate->exc_type) {
                          ^
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-vo45rneu/asynctnt/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-8437v6y1/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-vo45rneu/asynctnt/

Support new version policy

Since 2.10, Tarantool have a new release policy: https://www.tarantool.io/en/doc/latest/release/policy/

For example, versions like 2.10.0-beta1 is possible now. Versions with -alphaN, -betaN, -rcN and -entrypoint suffixes are prior to non-suffixed one. It is possible that 2.10.0-beta1 will not have all features that supported by 2.10.0. But now version restrictions in code and tests do not support new release policies.

Also it may be needed to rework approach to extracting a version for a connection. For example, when box.info.version and _TARANTOOL is 2.10.0-beta2-91-g08c9b4963-r472, greeting returns Tarantool 2.10.0 (Binary) string.

Don't resend initial request on schema change

If schema is changed asynctnt should run schema refetch in background and requests should not include schema_id at all

Decode error on binary data with MP_STR format

Got this error on .call() method, when try to resive binary data from tuple:

  File "asynctnt/iproto/response.pyx", line 315, in asynctnt.iproto.protocol.response_parse_body (asynctnt/iproto/protocol.c:14639)
  File "asynctnt/iproto/response.pyx", line 232, in asynctnt.iproto.protocol._response_parse_body_data (asynctnt/iproto/protocol.c:13857)
  File "asynctnt/iproto/response.pyx", line 144, in asynctnt.iproto.protocol._decode_obj (asynctnt/iproto/protocol.c:12876)
  File "asynctnt/iproto/response.pyx", line 128, in asynctnt.iproto.protocol._decode_obj (asynctnt/iproto/protocol.c:12656)
  File "asynctnt/iproto/unicode.pyx", line 17, in asynctnt.iproto.protocol.decode_string (asynctnt/iproto/protocol.c:3014)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 0: invalid continuation byte

Need to return binary data when it is not a unicode string

Suppress warnings when space format defined

Need a way to suppress warnings, without defining space format:

Field #0 of space auth_user is not in space format definition
Field #1 of space auth_user is not in space format definition
Field #2 of space auth_user is not in space format definition