oracle / python-oracledb

Python driver for Oracle Database conforming to the Python DB API 2.0 specification. This is the renamed, new major release of cx_Oracle

python-oracledb

python-oracledb is a Python programming language extension module allowing Python programs to connect to Oracle Database. It is the renamed, new major release of the popular cx_Oracle driver.

The module conforms to the Python Database API 2.0 specification with a considerable number of additions and a couple of minor exclusions, see the feature list.

Synchronous and concurrent coding styles are supported.

Installation

Run python -m pip install oracledb

See python-oracledb Installation.

Dependencies and Interoperability

• Python versions 3.7 through 3.12.

  Prebuilt packages are available for these Python versions on Windows, on macOS and on Linux.

  Source code is also available.

• Oracle Client libraries are optional.

  Thin mode: By default python-oracledb runs in a 'Thin' mode which connects directly to Oracle Database.

  Thick mode: Some advanced Oracle Database functionality is currently only available when optional Oracle Client libraries are loaded by python-oracledb. Libraries are available in the free Oracle Instant Client packages. Python-oracledb can use Oracle Client libraries 11.2 through 21c.

• Oracle Database

  Thin mode: Oracle Database 12.1 (or later) is required.

  Thick mode: Oracle Database 11.2 (or later) is required, depending on the Oracle Client library version. Oracle Database's standard client-server version interoperability allows connection to both older and newer databases. For example when python-oracledb uses Oracle Client 19c libraries, then it can connect to Oracle Database 11.2 or later.

Documentation

See the python-oracledb Documentation and Release Notes.

Samples

Examples can be found in the /samples directory and the Python and Oracle Database Tutorial.

Help

Questions can be asked in Github Discussions.

Problem reports can be raised in GitHub Issues.

Tests

See /tests

Contributing

This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide.

Security

Please consult the security guide for our responsible security vulnerability disclosure process.

License

See LICENSE, THIRD_PARTY_LICENSES, and NOTICE.

python-oracledb's Issues

thin mode not pure python

i would expect thin mode to be implemented purely in python but found a shared library thin_impl. this makes it impossible to run the code on different platforms with python support and is not the idea of "thin" according to my understanding.

Executemany uses first row as template for the following rows.

If the type for a column in the first row is different from another row it will throw an exception inside the executemany.

for instance the following parameters:

[ {'f1' : 1}, {'f1' : 1.0} ]
[ {'f1' : None}, {'f1' : 1.0} ]

will fail.
but these:

[ {'f1' : 1}, {'f1' : 1} ]
[ {'f1' : 1.0}, {'f1' : None} ]

will work.

1. What versions are you using?

platform.platform: Linux-5.14.0-1045-oem-x86_64-with-glibc2.29
sys.maxsize > 2**32: True
platform.python_version: 3.8.10

oracledb.__version__: 1.0.1

2. Is it an error or a hang or a crash?
   Error

3. What error(s) or behavior you are seeing?

  File "/home/tsuneki/tok/repos/tks-simulacao/src/core/interface/database/test_executemany.py", line 119, in <module>
    cursor.executemany(statement = query, parameters = dict)
  File "/home/tsuneki/.local/lib/python3.8/site-packages/oracledb/cursor.py", line 438, in executemany
    self._impl.bind_many(self, parameters)
  File "src/oracledb/impl/base/cursor.pyx", line 339, in oracledb.base_impl.BaseCursorImpl.bind_many
  File "src/oracledb/impl/base/cursor.pyx", line 54, in oracledb.base_impl.BaseCursorImpl._bind_values
  File "src/oracledb/impl/base/cursor.pyx", line 95, in oracledb.base_impl.BaseCursorImpl._bind_values_by_name
  File "src/oracledb/impl/base/bind_var.pyx", line 129, in oracledb.base_impl.BindVar._set_by_value
  File "src/oracledb/impl/base/var.pyx", line 176, in oracledb.base_impl.BaseVarImpl._check_and_set_value
  File "src/oracledb/impl/base/var.pyx", line 158, in oracledb.base_impl.BaseVarImpl._check_and_set_scalar_value
  File "/home/tsuneki/.local/lib/python3.8/site-packages/oracledb/errors.py", line 103, in _raise_err
    raise exc_type(_Error(message)) from cause
oracledb.exceptions.NotSupportedError: DPY-3013: unsupported Python type float for variable DB_TYPE_VARCHAR

4. Does your application call init_oracle_client()?

yes
oracledb.init_oracle_client(lib_dir='/usr/lib/oracle/19.8/client64/lib')

5. Include a runnable Python script that shows the problem.

CREATE TABLE "S1"."T1" 
   ("float_number" NUMBER(18,9));

import oracledb

oracledb.init_oracle_client(lib_dir='/usr/lib/oracle/19.8/client64/lib')

dictList = [{
    "float_number"   : None
},{
    "float_number"   : 2.0
}]

conn = oracledb.connect(dsn='db:1521/db', user='user', password='password')

query = '''INSERT INTO S1.T1 (float_number)
VALUES (:float_number)'''

with conn.cursor() as cursor:
    cursor.executemany(statement=query, parameters=dictList)
    cursor.commit()

DPY-3015: password verifier type 09X939 is not supported by python-oracledb in thin mode

Any clue about how to solve this issue without installing any client as there is no internet connection in the pc?

Enhance type hinting in the subscr module

The subscr module defines classes related to CQN functionality. It would be really nice to see these classes fully decked out with type hints. Here are some ideas:

• Message.queries could return List[MessageQuery]
• Message.tables could return List[MessageTable]
• MessageTable.rows could return List[MessageRow]

These enhancements would make working with subscription message objects much easier. Since there are several rows of nesting involved with interacting with the actual rows, it's sometimes hard to keep track of where we are 💫

Thanks for your consideration!

Thin mode should support DB_NCHARSET 'UTF8'

Connecting with encoding='UTF8'—as DB_NCHARSET is set so, and same with 'utf-8'—I get 'DPY-3012: national character set id 871 is not supported by python-oracledb in thin mode'.
I do not want a national character set, but good standard 'utf-8'.
What is the "Thin" mode for, when it does not support the basics, or do I misinterpret things? Is there a work-around or missing extra parameter?

So I propose the enhancement: Thin mode should support DB_NCHARSET 'UTF8'.

UnicodeDecodeError: 'utf-8' ocdec can't decode byte 0x96 in position 1283: invalid start byte

1. What versions are you using?

2. Is it an error or a hang or a crash?

error

4. What error(s) or behavior you are seeing?

UnicodeDecodeError: 'utf-8' ocdec can't decode byte 0x96 in position  1283: invalid start byte

5. Does your application call init_oracle_client()?

no. im in thin mode.

7. Include a runnable Python script that shows the problem.

i do not have company permission to do so.

Thin client error: DPY-6000: cannot connect to database. Listener refused connection. (Similar to ORA-12660)

1. What versions are you using?

python-oracledb 1.0.3
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0
platform.platform: Linux-3.10.0-1160.71.1.el7.x86_64-x86_64-with-glibc2.17
platform.python_version: 3.9.6
sys.maxsize > 2**32: True

2. Is it an error or a hang or a crash?

Connection fails to Exadata RAC with the thin client.

connection = oracledb.connect(user=admin, password=args.adminpwd,
dsn=args.service_name, config_dir = tnsdir)

3. What error(s) or behavior you are seeing?

With thin client connection to a single instance non RAC db seems to work fine.
But I cannot connect to Exadata RAC instance with thin client, it causes the exception:

oracledb.exceptions.OperationalError: DPY-6000: cannot connect to database. Listener refused connection. (Similar to ORA-12660)

Problem tns entry is like:
x01 =
(DESCRIPTION =
(ADDRESS_LIST =
(FAILOVER = ON)
(LOAD_BALANCE = ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = xt01dev-scan.home.net)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = xt02dev-scan.home.net)(PORT = 1521))
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = dbox_svc.tst.tns)
)
)

if I call init_oracle_client() it starts working.

5. Does your application call init_oracle_client()?

No.

But if I call init_oracle_client(), it starts working.

6. Include a runnable Python script that shows the problem.

# Uncomment below line and the problem goes away
# oracledb.init_oracle_client()  
connection = oracledb.connect(user=admin, password=args.adminpwd, 
                                dsn=args.service_name, config_dir = tnsdir)

InvalidSwitchError - Invalid switch into Greenlet OR BlockingSwitchOutError

1. What versions are you using?
   platform.platform: Linux-4.14.186-146.268.amzn2.x86_64-x86_64-with-glibc2.31
   sys.maxsize > 2**32: True
   platform.python_version: 3.9.10
   oracledb.version: 1.0.2
   sqlalchemy.version: 1.4.35
   gevent.version: 21.12.0
   gunicorn.version: 20.1.0
   oracle database version: Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0

2. Is it an error or a hang or a crash?
   Crash

3. What error(s) or behavior you are seeing?
   Gunicorn (gevent workers) requests failed with error InvalidSwitchError

2022-07-31 11:58:56,377 ERROR sqlalchemy.pool.impl.QueuePool Exception during reset or similar
Traceback (most recent call last):
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 739, in _finalize_fairy
    fairy._reset(pool)
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 988, in _reset
    pool._dialect.do_rollback(self)
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 682, in do_rollback
    dbapi_connection.rollback()
  File "/usr/local/project_venv/lib/python3.9/site-packages/oracledb/connection.py", line 634, in rollback
    self._impl.rollback()
  File "src/oracledb/impl/thin/connection.pyx", line 399, in oracledb.thin_impl.ThinConnImpl.rollback
  File "src/oracledb/impl/thin/protocol.pyx", line 301, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 302, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 262, in oracledb.thin_impl.Protocol._process_message
  File "src/oracledb/impl/thin/protocol.pyx", line 240, in oracledb.thin_impl.Protocol._process_message
  File "src/oracledb/impl/thin/protocol.pyx", line 308, in oracledb.thin_impl.Protocol._receive_packet
  File "src/oracledb/impl/thin/buffer.pyx", line 1092, in oracledb.thin_impl.ReadBuffer.receive_packet
  File "src/oracledb/impl/thin/buffer.pyx", line 427, in oracledb.thin_impl.ReadBuffer._receive_packet_helper
  File "src/oracledb/impl/thin/buffer.pyx", line 291, in oracledb.thin_impl.ReadBuffer._get_data_from_socket
  File "/usr/local/project_venv/lib/python3.9/site-packages/gevent/_socketcommon.py", line 693, in recv_into
    self._wait(self._read_event)
  File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
  File "src/gevent/_hub_primitives.py", line 322, in gevent._gevent_c_hub_primitives.wait_on_socket
  File "src/gevent/_hub_primitives.py", line 304, in gevent._gevent_c_hub_primitives._primitive_wait
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 57, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
gevent.exceptions.InvalidSwitchError: Invalid switch into <Greenlet at 0x7fa41355d480: _handle_and_close_when_done(functools.partial(<bound method GeventWorker.handl, <bound method StreamServer.do_close of <StreamServ, (<gevent._socket3.socket at 0x7fa41855c820 object,)>: got None (expected <gevent._gevent_c_waiter.Waiter object at 0x7fa40b4f1cc0>; waiting on <Hub '' at 0x7fa4364b4340 epoll default pending=0 ref=9 fileno=6 resolver=<gevent.resolver.thread.Resolver at 0x7fa42f8e9760 pool=<ThreadPool at 0x7fa42f8ebb20 tasks=0 size=2 maxsize=10 hub=<Hub at 0x7fa4364b4340 thread_ident=0x7fa43a301740>>> threadpool=<ThreadPool at 0x7fa42f8ebb20 tasks=0 size=2 maxsize=10 hub=<Hub at 0x7fa4364b4340 thread_ident=0x7fa43a301740>> thread_ident=0x7fa43a301740> with <io at 0x7fa40bb6d8c0 native=0x7fa40bb6d900 fd=27 events=READ active callback=<bound method Waiter.switch of <gevent._gevent_c_waiter.Waiter object at 0x7fa40b4f1cc0>> args=(<gevent._gevent_c_waiter.Waiter object at 0x7fa40b4f1cc0>,)>)

I got a few of this error as well (not sure if this is related but seems related)

2022-07-31 17:35:35,961 ERROR sqlalchemy.pool.impl.QueuePool Exception during reset or similar
Traceback (most recent call last):
File "src/oracledb/impl/thin/protocol.pyx", line 240, in oracledb.thin_impl.Protocol._process_message
File "src/oracledb/impl/thin/protocol.pyx", line 308, in oracledb.thin_impl.Protocol._receive_packet
File "src/oracledb/impl/thin/buffer.pyx", line 1092, in oracledb.thin_impl.ReadBuffer.receive_packet
File "src/oracledb/impl/thin/buffer.pyx", line 427, in oracledb.thin_impl.ReadBuffer._receive_packet_helper
File "src/oracledb/impl/thin/buffer.pyx", line 291, in oracledb.thin_impl.ReadBuffer._get_data_from_socket
File "/usr/local/project_venv/lib/python3.9/site-packages/gevent/_socketcommon.py", line 693, in recv_into
self._wait(self._read_event)
File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
File "src/gevent/_hub_primitives.py", line 322, in gevent._gevent_c_hub_primitives.wait_on_socket
File "src/gevent/_hub_primitives.py", line 304, in gevent._gevent_c_hub_primitives._primitive_wait
File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_hub_primitives.py", line 55, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_waiter.py", line 154, in gevent._gevent_c_waiter.Waiter.get
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 64, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 67, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch_out
File "src/gevent/_greenlet_primitives.py", line 68, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch_out
gevent.exceptions.BlockingSwitchOutError: Impossible to call blocking function in the event loop callback

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 739, in _finalize_fairy
fairy._reset(pool)
File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 988, in _reset
pool._dialect.do_rollback(self)
File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 682, in do_rollback
dbapi_connection.rollback()
File "/usr/local/project_venv/lib/python3.9/site-packages/oracledb/connection.py", line 634, in rollback
self._impl.rollback()
File "src/oracledb/impl/thin/connection.pyx", line 399, in oracledb.thin_impl.ThinConnImpl.rollback
File "src/oracledb/impl/thin/protocol.pyx", line 301, in oracledb.thin_impl.Protocol._process_single_message
File "src/oracledb/impl/thin/protocol.pyx", line 302, in oracledb.thin_impl.Protocol._process_single_message
File "src/oracledb/impl/thin/protocol.pyx", line 261, in oracledb.thin_impl.Protocol._process_message
File "src/oracledb/impl/thin/protocol.pyx", line 344, in oracledb.thin_impl.Protocol._reset
File "src/oracledb/impl/thin/buffer.pyx", line 1092, in oracledb.thin_impl.ReadBuffer.receive_packet
File "src/oracledb/impl/thin/buffer.pyx", line 427, in oracledb.thin_impl.ReadBuffer._receive_packet_helper
File "src/oracledb/impl/thin/buffer.pyx", line 291, in oracledb.thin_impl.ReadBuffer._get_data_from_socket
File "/usr/local/project_venv/lib/python3.9/site-packages/gevent/_socketcommon.py", line 693, in recv_into
self._wait(self._read_event)
File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
File "src/gevent/_hub_primitives.py", line 322, in gevent._gevent_c_hub_primitives.wait_on_socket
File "src/gevent/_hub_primitives.py", line 304, in gevent._gevent_c_hub_primitives._primitive_wait
File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_hub_primitives.py", line 55, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_waiter.py", line 154, in gevent._gevent_c_waiter.Waiter.get
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 64, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 67, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch_out
File "src/gevent/_greenlet_primitives.py", line 68, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch_out
gevent.exceptions.BlockingSwitchOutError: Impossible to call blocking function in the event loop callback

5. Does your application call init_oracle_client()?
   No

6. Include a runnable Python script that shows the problem.
   As far as I can see, this is very rare - 1 request failed among thousands who succeeded (for a few hours test) - hence I can't reproduce it for now

Using create_pool with Oracle Wallet

We currently use the cx_Oracle-package-SessionPool() to connect to our Oracle-database.
Username and password are taken from an Oracle Wallet configured in sqlnet.ora:

WALLET_LOCATION =
    (SOURCE =
        (METHOD = FILE)
        (METHOD_DATA =
            (DIRECTORY = /opt/shared/shsdb/operation/etc/oracle_wallet)
        )
    )
SQLNET.WALLET_OVERRIDE = TRUE

Content of tnsnames.ora:

SHSDQD.WORLD =
  (DESCRIPTION =
    (CONNECT_TIMEOUT=10)(RETRY_COUNT=3)
    (ADDRESS_LIST =
      (ADDRESS=(PROTOCOL=TCP)(HOST= HOST_1)(PORT=1721))
      (ADDRESS=(PROTOCOL=TCP)(HOST= HOST_2)(PORT=1721))
    )
    (CONNECT_DATA=(SERVICE_NAME=SHSDQD_PRIM.WORLD)(SERVER=DEDICATED))
  )

We currently use code from this cx_Oracle issue.

import cx_Oracle
con=cx_Oracle.SessionPool(dsn="SHSDQD.WORLD", encoding="UTF-8", externalauth=True, homogeneous=False)
con.acquire()
<cx_Oracle.Connection to externally identified user>

With a single connect-call this works also with python-oracledb in thick-mode:

import oracledb
oracledb.init_oracle_client()
oracledb.connect("SHSDQD.WORLD")
<oracledb.Connection to externally identified user>

Unfortunately, when I try the equivalent code to cx_Oracle.SessionPool in python-oracledb I get an error:

import oracledb
oracledb.init_oracle_client()
con = oracledb.create_pool(dsn="SHSDQD.WORLD", externalauth=True, homogeneous=False)
con.acquire()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../VENV/lib64/python3.8/site-packages/oracledb/pool.py", line 148, in acquire
    return self._connection_type(user=user, password=password,
  File ".../VENV/lib64/python3.8/site-packages/oracledb/connection.py", line 131, in __init__
    impl.connect(params_impl, pool_impl)
  File "src/oracledb/impl/thick/connection.pyx", line 360, in oracledb.thick_impl.ThickConnImpl.connect
  File "src/oracledb/impl/thick/utils.pyx", line 410, in oracledb.thick_impl._raise_from_odpi
  File "src/oracledb/impl/thick/utils.pyx", line 400, in oracledb.thick_impl._raise_from_info
oracledb.exceptions.DatabaseError: ORA-24415: Missing or null username.

Are there other options I need to use in python-oracledb to make session pools work in python-oracledb with Oracle Wallet?

We currently use the following Oracle Client:

>>> oracledb.clientversion()
(12, 2, 0, 1, 0)

Change return types to avoid IDE errors/warnings

Various IDEs complain about the Cursor type not being iterable even though it is.

For example:

def example():
    with connect( user="***", password="***", dsn="***") as connection:
        with connection.cursor() as cursor:
            execute_stmt = cursor.execute("select * from table")
            if execute_stmt:
                return [r for r in execute_stmt]

When using the pyright lsp with neovim, I get this error:

 "Type[Cursor]" is not iterable

When using pycharm I get a warning of:

Expected type 'collections.Iterable', got 'Type[Cursor]' instead 

As temporary work around I have modified the signature of the execute method of Cursor class in oracledb/cursor.py to return Union['Cursor', None] instead of Union[Type["Cursor"], None].

original:

    def execute(self, statement: Union[str, None],
                parameters: Union[list, tuple, dict]=None,
                **keyword_parameters: dict) -> Union[Type["Cursor"], None]:

new:

    def execute(self, statement: Union[str, None],
                parameters: Union[list, tuple, dict]=None,
                **keyword_parameters: dict) -> Union['Cursor', None]:

OS: macOS BigSur v11.6.1
pycharm 2022.1.4 (Professional Edition)
NVIM v0.7.2
pyright 1.1.270

Need Support with connection string for CMAN connection to DB (Source_route=yes) in connection string

We use a OCM which is running in my jump server, it is like the gateway for our DB which is in another VCN in OCI.
We want to secure the DB and make sure an extra layer of security with the Jump server authentication.

Configuration is like below

Client machine ---> Jump Server ---> Database
CMAN is in Jump Server -----> Database

We basically open a Terminal connection in SSH (Port 22) and create a tunnel to port 1480 of OCM which will forward the connection to Database

ssh -i user@jumpserver -L 127.0.0.1:1481:jumpserver:1480

connection string will be then like below (Note: we make the address list with 1 host as local machine with the same port as local tunnel port opened in above step)

(description=(address_list=(address=(protocol=tcp)(port=1481)(host=127.0.0.1))(address=(protocol=tcp)(port=1521)(host=Test_DB_Scan_Server_Name)))(connect_data=(service_name=ORA_TEST))(source_route=yes))

I am able to connect from SQL Developer which is using Thin driver and it works good.

I discussed this issue in discussion already (#77)

`import oracledb
import os

oracledb.defaults.config_dir = "/Users/eprince/PycharmProjects/flaskProject2/"
un = "SCOTT"
pw = "pwd"

c_dsn = "(DESCRIPTION=(RETRY_DELAY=2)(TRANSPORT_CONNECT_TIMEOUT=100 ms)(FAILOVER=on)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=1481))(ADDRESS=(PROTOCOL=tcp)(HOST=db_scan_server_url)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=ORA_TEST_DB))(source_route=yes))"

with oracledb.connect(user=un,password=pw,dsn=c_dsn) as connection:
with connection.cursor() as cursor:
sql = "select * from employee"
for r in cursor.execute(sql):
print(r)`

Thin mode ORA-01017: invalid username/password with 12.1.0.2

1. What versions are you using?

platform.platform: Darwin-20.6.0-x86_64-i386-64bit

sys.maxsize > 2**32: True

platform.python_version: 3.7.13

oracledb.version: 1.0.1

Database Version: 12.1.0.2.0

2. Is it an error or a hang or a crash?
   Crash (error)
3. What error(s) or behavior you are seeing?

#test-2.py

import oracledb
import os

un = os.environ.get('USER_NAME')
pw = os.environ.get('PASSWORD')
cs = os.environ.get('CONNECT_STRING')
oracledb.init_oracle_client()
with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
   with connection.cursor() as cursor:
      sql = """select version
               from   v$instance"""
      for r in cursor.execute(sql):
         print(r)

python test-2.py
('12.1.0.2.0',)

#test-2.py

import oracledb
import os

un = os.environ.get('USER_NAME')
pw = os.environ.get('PASSWORD')
cs = os.environ.get('CONNECT_STRING')
#oracledb.init_oracle_client()
with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
   with connection.cursor() as cursor:
      sql = """select version
               from   v$instance"""
      for r in cursor.execute(sql):
         print(r)

python test-2.py
Traceback (most recent call last):
File "test-2.py", line 10, in
with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
File "/Users/asrajag/miniconda3/envs/oracle/lib/python3.7/site-packages/oracledb/connection.py", line 1000, in connect
return conn_class(dsn=dsn, pool=pool, params=params, **kwargs)
File "/Users/asrajag/miniconda3/envs/oracle/lib/python3.7/site-packages/oracledb/connection.py", line 128, in init
impl.connect(params_impl)
File "src/oracledb/impl/thin/connection.pyx", line 345, in oracledb.thin_impl.ThinConnImpl.connect
File "src/oracledb/impl/thin/connection.pyx", line 163, in oracledb.thin_impl.ThinConnImpl._connect_with_params
File "src/oracledb/impl/thin/connection.pyx", line 129, in oracledb.thin_impl.ThinConnImpl._connect_with_description
File "src/oracledb/impl/thin/connection.pyx", line 250, in oracledb.thin_impl.ThinConnImpl._connect_with_address
File "src/oracledb/impl/thin/protocol.pyx", line 205, in oracledb.thin_impl.Protocol._connect_phase_two
File "src/oracledb/impl/thin/protocol.pyx", line 296, in oracledb.thin_impl.Protocol._process_message
oracledb.exceptions.DatabaseError: ORA-01017: invalid username/password; logon denied

4. Does your application call init_oracle_client()?

Thick mode works, thin mode does not.
5. Include a runnable Python script that shows the problem.

#test-2.py

import oracledb
import os

un = os.environ.get('USER_NAME')
pw = os.environ.get('PASSWORD')
cs = os.environ.get('CONNECT_STRING')
#oracledb.init_oracle_client()
with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
   with connection.cursor() as cursor:
      sql = """select version
               from   v$instance"""
      for r in cursor.execute(sql):
         print(r)

Can not create a connection pool with a proxy user while this is possible in cx_Oracle.

1. What versions are you using?

database: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

platform.platform: macOS-10.16-x86_64-i386-64bit
sys.maxsize > 2**32: True
platform.python_version: 3.9.5
oracledb.version: 1.2.0

3. Is it an error or a hang or a crash?

error

4. What error(s) or behavior you are seeing?

I am displaying the number 1, session user and proxy user for three different connection types (self describing below):

This the output:

test_oracledb_connect
(1, 'BC_BO', 'BC_PROXY')
test_oracledb_create_pool
(1, 'BC_PROXY', None)
test_cx_oracle_sessionpool
(1, 'BC_BO', 'BC_PROXY')

5. Does your application call init_oracle_client()?

Yes

6. Include a runnable Python script that shows the problem.

import os
import oracledb
import cx_Oracle


PROXY_USER = 'BC_PROXY'
SESSION_USER = 'BC_BO'
USER = f"{PROXY_USER}[{SESSION_USER}]"
PASSWORD = os.getenv('PASSWORD')
DSN = 'bc_acc'


def process(connection):
        with connection.cursor() as cursor:
            rs = cursor.execute("""\
select  :idx
,       sys_context('USERENV', 'SESSION_USER') as session_user
,       sys_context('USERENV', 'PROXY_USER') as proxy_user
from    dual""", idx=1)
            if rs:  # it is a query
                for row in rs:
                    print(row)


def test_oracledb_connect():
        print('test_oracledb_connect')
        oracledb.init_oracle_client()
        connection = oracledb.connect(user=USER,
                                      password=PASSWORD,
                                      dsn=DSN)
        process(connection)


def test_oracledb_create_pool():
        print('test_oracledb_create_pool')
        oracledb.init_oracle_client()
        pool = oracledb.create_pool(user=USER,
                                    password=PASSWORD,
                                    dsn=DSN,
                                    homogeneous=False,
                                    encoding="UTF-8")
        connection = pool.acquire()
        process(connection)


def test_cx_oracle_sessionpool():
        print('test_cx_oracle_sessionpool')
        # Basic Authentication with proxy
        pool = cx_Oracle.SessionPool(user=USER,
                                     password=PASSWORD,
                                     dsn=DSN,
                                     homogeneous=False,
                                     encoding="UTF-8")
        connection = pool.acquire()
        process(connection)
        

def main():
        test_oracledb_connect()
        test_oracledb_create_pool()
        test_cx_oracle_sessionpool()
        

if __name__ == '__main__':
        main()

`Windows fatal exception: access violation` in thick mode

1. What versions are you using?

platform.platform: Windows-10-10.0.19044-SP0
sys.maxsize > 2**32: True
platform.python_version: 3.10.4
oracledb.__version__: 1.0.0

I'm using oracle 21.3.0.0, with client version (21, 3, 0, 0, 0)

2. Is it an error or a hang or a crash?
   A crash of the python interpreter

3. What error(s) or behavior you are seeing?

Windows fatal exception: access violation

Current thread 0x00001558 (most recent call first):
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\oracledb\connection.py", line 302 in close
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\engine\default.py", line 599 in do_close
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\pool\base.py", line 316 in _close_connection
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\pool\base.py", line 812 in __close
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\pool\base.py", line 745 in invalidate
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\pool\base.py", line 1343 in invalidate
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\engine\base.py", line 633 in invalidate
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\test\engine\test_reconnect.py", line 1112 in test_rollback_on_invalid_twophase
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\testing\exclusions.py", line 121 in _do
  File "C:\Users\my_user\Dev\GitHub\sqlalchemy\lib\sqlalchemy\testing\exclusions.py", line 93 in decorate
  File "<string>", line 2 in test_rollback_on_invalid_twophase
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\python.py", line 192 in pytest_pyfunc_call
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_callers.py", line 39 in _multicall
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_manager.py", line 80 in _hookexec
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_hooks.py", line 265 in __call__
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\python.py", line 1761 in runtest
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 166 in pytest_runtest_call
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_callers.py", line 39 in _multicall
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_manager.py", line 80 in _hookexec
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_hooks.py", line 265 in __call__
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 259 in <lambda>
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 338 in from_call
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 258 in call_runtest_hook
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 219 in call_and_report
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 130 in runtestprotocol
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\runner.py", line 111 in pytest_runtest_protocol
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_callers.py", line 39 in _multicall
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_manager.py", line 80 in _hookexec
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_hooks.py", line 265 in __call__
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\main.py", line 347 in pytest_runtestloop
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_callers.py", line 39 in _multicall
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_manager.py", line 80 in _hookexec
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_hooks.py", line 265 in __call__
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\main.py", line 322 in _main
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\main.py", line 268 in wrap_session
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\main.py", line 315 in pytest_cmdline_main
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_callers.py", line 39 in _multicall
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_manager.py", line 80 in _hookexec
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\pluggy\_hooks.py", line 265 in __call__
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\config\__init__.py", line 164 in main
  File "C:\Users\my_user\miniconda3\envs\sa\lib\site-packages\_pytest\config\__init__.py", line 187 in console_main
  File "C:\Users\my_user\miniconda3\envs\sa\Scripts\pytest.exe\__main__.py", line 7 in <module>
  File "C:\Users\my_user\miniconda3\envs\sa\lib\runpy.py", line 86 in _run_code
  File "C:\Users\my_user\miniconda3\envs\sa\lib\runpy.py", line 196 in _run_module_as_main
Windows fatal exception: code 0xebadebad

4. Does your application call init_oracle_client()?

yes

5. Include a runnable Python script that shows the problem.

Sadly I cannot reproduce it. I've tried multiple times re-running the same command but it does not reproduce.
The crash happened while implementing support for the oracledb driver in sqlalchemy, running pytest --dburi oracle+oracledb://scott:[email protected]:1521/?service_name=XEPDB1 .\test\engine\ and momentarily enabling the thick mode.

The interpreter crashed while executing this test:
test/engine/test_reconnect.py::RealReconnectTest_oracle+oracledb_21_3_0_0_0::test_rollback_on_invalid_twophase
https://github.com/sqlalchemy/sqlalchemy/blob/ad86d32f7fbd1c6deda8ff3bebe0595c0f2986cc/test/engine/test_reconnect.py#L1108-L1112

I know it's not much to work on, but it's all I have.
I've reported since from the stack trace it seems to be a double free or similar since it happened in the close method, so maybe something useful can be gathered from it event with this few information.

Let me know if I can be of further help

`oracledb.defaults.fetch_lobs` changes how integers are fetched

I have the following table

create table oracledbtest(testint integer);

and I've inserted one record via

insert into oracledbtest values (1);

With this, the following Python script

import oracledb
for fetch_lobs in (False, True):
	oracledb.defaults.fetch_lobs = fetch_lobs
	db = oracledb.connect("user/pwd@db", config_dir='/Users/walter/oracle/instantclient_12_2/network/admin')
	c = db.cursor()
	data = c.execute("select * from oracledbtest").fetchall()
	print(f"{fetch_lobs=}: {data=}")

outputs

fetch_lobs=False: data=[(1.0,)]
fetch_lobs=True: data=[(1,)]

i.e. with oracledb.defaults.fetch_lobs = False integers are fetched as floats, but with oracledb.defaults.fetch_lobs = True they are fetched as ints.

IMHO they should always be fetched as ints.

Versions from

import sys, platform, oracledb

print(f"{platform.platform()=}")
print(f"{sys.maxsize > 2**32=}")
print(f"{platform.python_version()=}")
print(f"{oracledb.__version__=}")

are

platform.platform()='macOS-12.4-x86_64-i386-64bit'
sys.maxsize > 2**32=True
platform.python_version()='3.10.4'
oracledb.__version__='1.0.0'

The database is an Oracle XE 21.0.0.0.0

Improve TLS connection implementation to support use of one way TLS certificates without needing OS recognition

Talking about version 1.1.0.

The current version does not allow to use unencrypted .pem file for mTLS connection, for which the following code, found at line 134 of the 'src/oracledb/impl/thin/crypto.pyx' module, fails in that case:

ssl_context.load_cert_chain(pem_file_name,
                            password=params._get_wallet_password())

Things work if we make this code line conditional, for instance on whether a password to decrypt the certificate is actually provided:

password = params._get_wallet_password()
if password is not None:
   ssl_context.load_cert_chain(pem_file_name,
                               password=password)

Cf this discussion on the forum: something like that is necessary when one is using the Oracle Cloud functionality of the Amazon Cloud service, and notably when one is not admin of the server actually hosting the Oracle database.

DB_TYPE_NVARCHAR recipe from cx_Oracle does not seem to work with oracledb

hey Anthony -

we here are still working with the changes you suggested in oracle/python-cx_Oracle#596. These are working for cx_Oracle but failing for oracledb.

Test script:

#import cx_Oracle
import oracledb as cx_Oracle
import random


conn = cx_Oracle.connect(
    user="scott",
    password="tiger",
    dsn=cx_Oracle.makedsn("oracle18c", 1521, service_name="xe"),
)


cursor = conn.cursor()

try:
    cursor.execute("drop table long_text")
except:
    pass

cursor.execute(
    """
CREATE TABLE long_text (
    x INTEGER,
    y NCLOB,
    z INTEGER
)
"""
)


# the third character is the failure character
word_seed = "ab🐍’«cdefg"

data = " ".join(
    "".join(random.choice(word_seed) for j in range(150))
    for i in range(100)
)

# succeeds
# cursor.setinputsizes(**{"y": cx_Oracle.NCLOB})

# fails with oracledb only:
# ORA-01461: can bind a LONG value only for insert into a LONG column
cursor.setinputsizes(**{"y": cx_Oracle.DB_TYPE_NVARCHAR})

# no setinputsizes: fails on oracledb only with:
# ORA-01483: invalid length for DATE or NUMBER bind variable

cursor.execute(
    "INSERT INTO long_text (x, y, z) VALUES (:x, :y, :z)",
    {"x": 5, "y": data, "z": 10},
)

With cx_Oracle. all three setinputsizes patterns: using NCLOB, using DB_TYPE_NVARCHAR, not calling setinputsizes, all succeed.

using oracledb: NCLOB succeeds, DB_TYPE_NVARCHAR produces "ORA-01461: can bind a LONG value only for insert into a LONG column", and not using setinputsizes produces "ORA-01483: invalid length for DATE or NUMBER bind variable".

I am trying to integrate every improvement suggested in the above mentioned issue as can be seen in this patch: https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/3903 tests all succeed for cx_Oracle but we have those failures for oracledb.

FWIW, before I made those changes, we do have oracledb passing all tests that cx_Oracle passes, very good!

ONE problem needs to solve

oracledb.exceptions.NotSupportedError: DPY-3010: connections to this database server version are not supported by python-oracledb in thin mode

should i add
code: oracledb.init_oracle_client(……)
to try it again?

ConnectionPool.acquire instance method returns type checking errors

1. What versions are you using?

Oracle 19c

platform.platform: macOS-11.6.8-x86_64-i386-64bit
sys.maxsize > 2**32: True
platform.python_version: 3.10.6

oracledb.__version__: 1.0.3

2. Is it an error or a hang or a crash?

None; it is a type checking error

3. What error(s) or behavior you are seeing?

When calling the instance method ConnectionPool.acquire using the with statement, type checking tools report errors:

• "Type[Connection]" has no attribute "__enter__"
• "Type[Connection]" has no attribute "__exit__"

4. Does your application call init_oracle_client()?

N/A

5. Include a runnable Python script that shows the problem.

from oracledb import pool

dbpool: pool.ConnectionPool = pool.create_pool(
    user="username",
    password="password",
    dsn="dsn"
)

with dbpool.acquire() as conn:  # type checking error shows here
    conn.ping()  # works as expected, but conn is Unknown in type checkers

Not able to set a new wait_timeout on the create_pool()

oracledb v1.0.0

Even set a new value for wait_timout, the fixed value of 5000 is kept.

    oracledb.init_oracle_client()
    oracleclient = oracledb.create_pool(
        user=settings.ORACLE_USER,
        password=settings.ORACLE_PASS,
        dsn=settings.ORACLE_DSN,
        min=1, max=1, increment=0,
        encoding='UTF-8',
        expire_time=60,              # minutes
        max_lifetime_session=14400,  # seconds
        ping_interval=60,            # seconds
        wait_timeout=10000,          # milliseconds
        getmode=oracledb.POOL_GETMODE_TIMEDWAIT
    )

After creating the pool, if I print the print(oracleclient.wait_timeout) I'm getting the value 5000.

Thin vs Thick Performance Implications

Hi @anthony-tuininga

Is there any performance implications by using the thin style instead of the thick style?

Assuming there is no performance implications, and I don't use any of the features that require the thick style, are there any other draw backs to converting my application to the thin style?

Most importantly, why did you change the name from cx_Oracle? I've been using it a long time, it will be an adjustment to remember to import oracledb going forward :).

ImportError: dynamic module does not define module export function (PyInit_base_impl)

I am trying to use the connect from oraclebd package, like this:
with oracledb.connect(user=user, password=password, dsn=dsn) as connection:
but I am receiving the following error:

from . import base_impl, thick_impl, thin_impl
ImportError: dynamic module does not define module export function (PyInit_base_impl)
python-BaseException

Segmentation fault using soda API on Oracle Linux 9 (aarch64)

1. What versions are you using?

Database version: 19c, running on Oracle cloud

platform.platform: Linux-5.15.0-1.43.4.1.el9uek.aarch64-aarch64-with-glibc2.34

sys.maxsize > 2**32: True

platform.python_version: 3.9.10

oracledb._version_: 1.1.1

Oracle instant client installed: oracle-instantclient19.10-basic-19.10.0.0.0-1.aarch64.rpm

2. Is it an error or a hang or a crash?

A crash: Segmentation fault (core dumped)

3. What error(s) or behavior you are seeing?

The example code samples/soda_basic.py crashes when reaching line 125: documents = collection.find().filter({'name': {'$like': 'Ma%'}}).getDocuments()

(venv-oracledb) [opc@instance-20220908-1936 ~]$ python soda_basic.py
The key of the new SODA document is: 126EDFEDD0A04F6EBFAA85642294C0B3
Retrieved SODA document dictionary is:
{'name': 'Matilda', 'address': {'city': 'Melbourne'}}
Retrieved SODA document string is:
{"name":"Matilda","address":{"city":"Melbourne"}}
Names matching 'Ma%'
Segmentation fault (core dumped)

The virtual environment is clean:

(venv-oracledb) [opc@instance-20220908-1936 ~]$ pip freeze
cffi==1.15.1
cryptography==38.0.1
oracledb==1.1.1
pycparser==2.21

The code also crashes in the same manner at line 136: c = collection.find().filter({'address.city': {'$regex': '.*o.*'}}).remove(), if I skip the lines 125~128.

4. Does your application call init_oracle_client()?

Yes. SODA API requires thick mode, and I'm quite sure that my credentials are correct.

The connection to the DB is also good, since the most part of samples/soda_basic.py works fine.

6. Include a runnable Python script that shows the problem.

Just the samples/soda_basic.py. I only changed the config_dir for oracledb.init_oracle_client(), and the DB credentials user, password, and dsn for oracledb.connect().

The same code works fine on AMD64 Linux and on Windows, with the corresponding versions of instant clients installed.

I suspect it is something related to the ARM64 version of the instant client.

As shown in the console snippet above, I ran the script on an OCI instance, on which Oracle products are supposed to be well-supported...

Conda support

The original cx_Oracle had conda packages built for it. Will there eventually be conda packages built for this package? Thank you!

TQ MSGID not returned as it was working before

1. What versions are you using?

2. Is it an error or a hang or a crash?

3. What error(s) or behavior you are seeing?

4. Does your application call init_oracle_client()?

5. Include a runnable Python script that shows the problem.

a=queue.enqone(con.msgproperties(payload=book))
print(a)

oracledb.exceptions.NotSupportedError: DPY-3012: national character set id 871 is not supported by python-oracledb in thin mode

Getting below error oracledb.exceptions.NotSupportedError: DPY-3012: national character set id 871 is not supported by python-oracledb in thin mode after installing python-oracledb

Database is running in utf-8 and it can't be changed. How can this be resolved?

Support for asyncio

This is a continuation of the original request made on cx_Oracle: oracle/python-cx_Oracle#178.

The current status is that with the addition of the thin driver, adding suport for asyncio will be considerably simpler (and will only work in thin mode). If anyone has suggestions or recommendations on API, please share them!

Connection String with Failover

1. What versions are you using?

Give your database version.

19.11

Also run Python and show the output of:

import sys
import platform

print("platform.platform:", platform.platform())
print("sys.maxsize > 2**32:", sys.maxsize > 2**32)
print("platform.python_version:", platform.python_version())

import sys
import platform
print("platform.platform:", platform.platform())
platform.platform: macOS-10.16-x86_64-i386-64bit
print("sys.maxsize > 232:", sys.maxsize > 232)
sys.maxsize > 2**32: True
print("platform.python_version:", platform.python_version())
platform.python_version: 3.10.4

And:

import python-oracledb
print("python-oracledb.version:", python-oracledb.version)

import python-oracledb
File "", line 1
import python-oracledb
^
SyntaxError: invalid syntax
import oracledb
print("python-oracledb.version:", python-oracledb.version)
Traceback (most recent call last):
File "", line 1, in
NameError: name 'python' is not defined
print("python-oracledb.version:", oracledb.version)
python-oracledb.version: 1.0.0

2. Describe the problem

I am unable to connect with either
i) a connect string with FAILOVER=ON
or
ii) using a service name that is in a tnsnames.ora file with FAILOVER=ON

i)
Traceback (most recent call last):
File "/Users/asrajag/WorkSpace/python/oracle/opcig/source_code_no_cx.py", line 7, in
with cxo.connect(user=db['user_id'], password=db['password'], dsn=db['dsn']) as connection:
File "/Users/asrajag/miniconda3/lib/python3.10/site-packages/oracledb/connection.py", line 995, in wrapped
return conn_class(dsn=dsn, pool=pool, params=params, **kwargs)
File "/Users/asrajag/miniconda3/lib/python3.10/site-packages/oracledb/connection.py", line 104, in init
dsn = params_impl.parse_dsn(dsn, thin)
File "src/oracledb/impl/base/connect_params.pyx", line 457, in oracledb.base_impl.ConnectParamsImpl.parse_dsn
File "src/oracledb/impl/base/connect_params.pyx", line 432, in oracledb.base_impl.ConnectParamsImpl.parse_connect_string
File "src/oracledb/impl/base/connect_params.pyx", line 430, in oracledb.base_impl.ConnectParamsImpl.parse_connect_string
File "src/oracledb/impl/base/connect_params.pyx", line 258, in oracledb.base_impl.ConnectParamsImpl._parse_connect_string
File "src/oracledb/impl/base/connect_params.pyx", line 98, in oracledb.base_impl._parse_connect_descriptor
File "/Users/asrajag/miniconda3/lib/python3.10/site-packages/oracledb/errors.py", line 103, in _raise_err
raise exc_type(_Error(message)) from cause
oracledb.exceptions.DatabaseError: DPY-4017: invalid connect descriptor "(CONNECT_DATA = (SERVICE_NAME = OPCIG_OPCIGD1)"

ii)
Traceback (most recent call last):
File "/Users/asrajag/WorkSpace/python/oracle/test/test.py", line 11, in
with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
File "/Users/asrajag/miniconda3/lib/python3.10/site-packages/oracledb/connection.py", line 995, in wrapped
return conn_class(dsn=dsn, pool=pool, params=params, **kwargs)
File "/Users/asrajag/miniconda3/lib/python3.10/site-packages/oracledb/connection.py", line 104, in init
dsn = params_impl.parse_dsn(dsn, thin)
File "src/oracledb/impl/base/connect_params.pyx", line 457, in oracledb.base_impl.ConnectParamsImpl.parse_dsn
File "src/oracledb/impl/base/connect_params.pyx", line 432, in oracledb.base_impl.ConnectParamsImpl.parse_connect_string
File "src/oracledb/impl/base/connect_params.pyx", line 430, in oracledb.base_impl.ConnectParamsImpl.parse_connect_string
File "src/oracledb/impl/base/connect_params.pyx", line 298, in oracledb.base_impl.ConnectParamsImpl._parse_connect_string
File "/Users/asrajag/miniconda3/lib/python3.10/site-packages/oracledb/errors.py", line 103, in _raise_err
raise exc_type(_Error(message)) from cause
oracledb.exceptions.DatabaseError: DPY-4000: cannot connect to database. Unable to find "OPCIG_OPCIGD1" in /Users/asrajag/Library/Oracle/Network/Admin/tnsnames.ora

3. Include a runnable Python script that shows the problem.

i)

config.json
{
"database": {
"db_name": "OPCIG_OPCIGD1",
"user_id": "myuser",
"password": "mypassword",
"dsn": "(DESCRIPTION = (FAILOVER=ON) (LOAD_BALANCE=OFF) (ADDRESS_LIST= (LOAD_BALANCE=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = lpdstora-scan.mclaneco.com)(PORT = 1521))) (ADDRESS_LIST= (LOAD_BALANCE=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = lpdstexa-scan.mclaneco.com)(PORT = 1521))) (ADDRESS_LIST= (LOAD_BALANCE=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = lddscora-scan.mclaneco.com)(PORT = 1521))) (ADDRESS_LIST= (LOAD_BALANCE=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = lddscexa-scan.mclaneco.com)(PORT = 1521))) (CONNECT_DATA = (SERVICE_NAME = OPCIG_OPCIGD1))"
}
}

#test.py
import oracledb as cxo
import json

data = json.loads(open('./properties/config.json').read())
db = data['database']

with cxo.connect(user=db['user_id'], password=db['password'], dsn=db['dsn']) as connection:
with connection.cursor() as cursor:
sql = """SELECT DBMS_METADATA.get_ddl('PACKAGE', 'STANDARD', 'SYS')
FROM dual
"""
for r in cursor.execute(sql):
print(r)

ii)

tnsnames.ora
OPCIG_OPCIGD1,OPCIGD1 =
(DESCRIPTION = (FAILOVER=ON) (LOAD_BALANCE=OFF)
(ADDRESS_LIST= (LOAD_BALANCE=ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = lpdstora-scan.mclaneco.com)(PORT = 1521))
)
(ADDRESS_LIST= (LOAD_BALANCE=ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = lpdstexa-scan.mclaneco.com)(PORT = 1521))
)
(ADDRESS_LIST= (LOAD_BALANCE=ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = lddscora-scan.mclaneco.com)(PORT = 1521))
)
(ADDRESS_LIST= (LOAD_BALANCE=ON)
(ADDRESS = (PROTOCOL = TCP)(HOST = lddscexa-scan.mclaneco.com)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = OPCIG_OPCIGD1)
)
)

export USER_NAME=myuser
export PASSWORD=mypassword
export CONNECT_STRING=OPCIG_OPCIGD1

#test.py

import oracledb
import os

un = os.environ.get('USER_NAME')
pw = os.environ.get('PASSWORD')
cs = os.environ.get('CONNECT_STRING')

with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
with connection.cursor() as cursor:
sql = """select sysdate from dual"""
for r in cursor.execute(sql):
print(r)

Consider making `check_and_return_mode` public

1. Describe your new request in detail

I think it would be useful to have a way of finding the mode of the oracledb library without having to connect to the db to execute the suggested https://python-oracledb.readthedocs.io/en/latest/user_guide/tracing.html#vsessconinfo

Looking at the library it seems that there is an internal function check_and_return_mode that returns this information.
Given its returning value also a simpler wrapper called something like def is_thin_mode()->bool: ... would be enough

2. Give supporting information about tools and operating systems. Give relevant product version numbers

n/a

DPY-6005: cannot connect to database. Connection failed with "[Errno -3] Temporary failure in name resolution"

When I am trying to connect Oracledb from pythonanywhere web app facing this error, when I try to connect in local pycharm it is working fine. Could you please let me know how to resolve this error

Using pyinstaller to compile a simple code with oracledb throws an error in the executable file

I am trying to compile a test code that simply connects to an oracle datable using pyinstaller.
I'm doing it with the --onefile option as shown below:

pyinstaller test_oracle.py --onefile

When I later run the executable, at first I got a few errors, just some modules that were missing ('secrets', 'ssl', 'getpass', 'cryptography'). I fixed it with the option --hidden-import from pyinstaller.

After fixing that, a new error showed up, which I have not been able to fix:

Traceback (most recent call last):
File "test_oracle.py", line 1, in
File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
File "oracledb_init_.py", line 38, in
File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
File "oracledb\connection.py", line 39, in
File "src\oracledb\impl/thin/crypto.pyx", line 35, in init oracledb.thin_impl
ImportError: cannot import name pbkdf2
[18124] Failed to execute script 'test_oracle_sac' due to unhandled exception!

I have tried importing the module in different ways and it still does not work.

Any help would be appreciated!

1. What versions are you using?

oracledb==1.0.1
python 3.10.0
pyinstaller==5.1

ORA-01008: not all variables bound - but they are bound

1. What versions are you using?
   platform.platform: Linux-5.10.104-linuxkit-x86_64-with-glibc2.31
   sys.maxsize > 2**32: True
   platform.python_version: 3.9.13
   oracledb.version: 1.0.2
   sqlalchemy.version: 1.4.35
   pyramid.version: 2.0

2. Is it an error or a hang or a crash?
   crash

3. What error(s) or behavior you are seeing?

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/router.py", line 270, in __call__
    response = self.execution_policy(environ, self)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/router.py", line 276, in default_execution_policy
    return router.invoke_request(request)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/router.py", line 245, in invoke_request
    response = handle_request(request)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/tweens.py", line 43, in excview_tween
    response = _error_handler(request, exc)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/tweens.py", line 17, in _error_handler
    reraise(*exc_info)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/util.py", line 733, in reraise
    raise value
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/tweens.py", line 41, in excview_tween
    response = handler(request)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/router.py", line 143, in handle_request
    response = _call_view(
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/view.py", line 674, in _call_view
    response = view_callable(context, request)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/viewderivers.py", line 392, in viewresult_to_response
    result = view(context, request)
  File "/usr/local/project_venv/lib/python3.9/site-packages/pyramid/viewderivers.py", line 141, in _requestonly_view
    response = view(request)
  File "/opt/app/AppCardAPI/tests/test_shurgi_pyramid.py", line 27, in get_api
    promotion_data = request.oracle_dbsession.execute(
  File "<string>", line 2, in execute
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1692, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
    util.raise_(
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/project_venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/project_venv/lib/python3.9/site-packages/oracledb/cursor.py", line 378, in execute
    impl.execute(self)
  File "src/oracledb/impl/thin/cursor.pyx", line 122, in oracledb.thin_impl.ThinCursorImpl.execute
  File "src/oracledb/impl/thin/protocol.pyx", line 301, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 302, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 295, in oracledb.thin_impl.Protocol._process_message
sqlalchemy.exc.DatabaseError: (oracledb.exceptions.DatabaseError) ORA-01008: not all variables bound

[SQL:
                SELECT description
                FROM pmt.promotions
                WHERE promotion_id = :promotion_id
            ]
[parameters: {'promotion_id': 8597}]

(Background on this error at: https://sqlalche.me/e/14/4xp6)

4. Does your application call init_oracle_client()?
   No (the issue doesn't reproduce when using Thick mode)

5. Include a runnable Python script that shows the problem.

Run the server:

import sys
import oracledb
oracledb.version = "8.3.0"
sys.modules["cx_Oracle"] = oracledb
import cx_Oracle

from wsgiref.simple_server import make_server
import os
from pyramid.config import Configurator
from pyramid.response import Response
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker


def post_api(request):
    promotion_data = request.oracle_dbsession.execute(
        """      
                SELECT description
                FROM pmt.promotions
                WHERE promotion_id = :promotion_id
            """, {"promotion_id": request.matchdict["club_id"]}
    ).fetchone()
    request.oracle_dbsession.commit()
    return Response(status=204)

def get_api(request):
    promotion_data = request.oracle_dbsession.execute(
        """      
                SELECT description
                FROM pmt.promotions
                WHERE promotion_id = :promotion_id
            """, {"promotion_id": 8597}
    ).fetchone()
    request.oracle_dbsession.commit()
    return Response(status=200)


if __name__ == '__main__':
    # oracledb.init_oracle_client()
    with Configurator() as config:
        config.add_route('post_api', '/post_api/{club_id}')
        config.add_route('get_api', '/get_api')
        config.add_view(post_api, route_name='post_api')
        config.add_view(get_api, route_name='get_api')

        # Oracle connection
        engine = create_engine(
            f'oracle://{os.getenv("oracle_user")}:{os.getenv("oracle_pass")}@{os.getenv("oracle_host")}:{os.getenv("oracle_port")}/{os.getenv("oracle_db")}',
            convert_unicode=False, pool_recycle=10, pool_size=50, echo=True, echo_pool="debug"
        )
        oracle_session_factory = scoped_session(sessionmaker(bind=engine))
        config.add_request_method(
            lambda r: oracle_session_factory, "oracle_dbsession", reify=True
        )

        app = config.make_wsgi_app()
    server = make_server('0.0.0.0', 6333, app)
    print("--- start server")
    server.serve_forever()

Run the test and get the error on the server:

def test_load():
    import requests
    for i in range(2):
        res_1 = requests.post("http://0.0.0.0:6333/post_api/123",)
        res_2 = requests.get("http://0.0.0.0:6333/get_api",)
        print(res_1.status_code, res_2.status_code, i)

`LOB.getchunksize()` not supported in thin mode

The following Python script

import oracledb

db = oracledb.connect("user/pwd@db", config_dir=os.environ["ORACLE_HOME"] + "/network/admin")
lob = db.cursor().execute("select to_clob('foo') from dual").fetchone()[0]
lob.getchunksize()

gives the following stacktrace:

Traceback (most recent call last):
  File "/Users/walter/oracledb_bug.py", line 6, in <module>
    lob.getchunksize()
  File "/Users/walter/pyvenvs/default/lib/python3.10/site-packages/oracledb/lob.py", line 75, in getchunksize
    return self._impl.get_chunk_size()
  File "/Users/walter/pyvenvs/default/lib/python3.10/site-packages/oracledb/utils.py", line 51, in wrapped_f
    errors._raise_err(errors.ERR_FEATURE_NOT_SUPPORTED,
  File "/Users/walter/pyvenvs/default/lib/python3.10/site-packages/oracledb/errors.py", line 103, in _raise_err
    raise exc_type(_Error(message)) from cause
oracledb.exceptions.NotSupportedError: DPY-3001: getting the chunk size of a LOB is only supported in python-oracledb thick mode

The documentation for getchunksize() states:

Returns the chunk size for the internal LOB. Reading and writing to the LOB in chunks of multiples of this size will improve performance.

Is there even something like an "optmal chunk size" in thin mode? If this isn't relevant for thin mode the documentation (and/or the exception message?) should suggest which chunk size should be used for reading a LOB.

Timestamp values differ between oracledb and cx_Oracle

Continued from #2

Timestamps fetched by oracledb have different values that those fetched by cx_Oracle. Using alter session set time_zone doesn't make a difference.

Python 3.10.4 (main, Apr 26 2022, 19:42:59) [Clang 13.1.6 (clang-1316.0.21.2)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.3.0 -- An enhanced Interactive Python. Type '?' for help.
🐍 — 1 ❯ odb = oracledb.connect("user/pwd@db")
🐍 — 2 ❯ cdb = cx_Oracle.connect("user/pwd@db")
🐍 — 3 ❯ oc = odb.cursor()
🐍 — 4 ❯ cc = cdb.cursor()
🐍 — 5 ❯ oc.execute("alter session set time_zone='Europe/Berlin'")
🐍 — 6 ❯ cc.execute("alter session set time_zone='Europe/Berlin'")
🐍 — 7 ❯ cc.execute("select start_date from dba_scheduler_jobs where job_name = 'LOG_CLEANUP_JOB'").fetchone()
🐍 — 7 ❮ (datetime.datetime(2019, 3, 26, 4, 33),)
🐍 — 8 ❯ oc.execute("select start_date from dba_scheduler_jobs where job_name = 'LOG_CLEANUP_JOB'").fetchone()
🐍 — 8 ❮ (datetime.datetime(2019, 3, 26, 2, 33),)
🐍 — 9 ❯

The script

import sys, platform, oracledb

print(f"{platform.platform()=}")
print(f"{sys.maxsize > 2**32=}")
print(f"{platform.python_version()=}")
print(f"{oracledb.__version__=}")

gives the following output on my machine:

platform.platform()='macOS-12.4-x86_64-i386-64bit'
sys.maxsize > 2**32=True
platform.python_version()='3.10.4'
oracledb.__version__='1.0.0'

I'm using oracledb in thin mode.

The following Oracle related environment variables are set:

export ORACLE_HOME=~/oracle/instantclient_12_2
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$ORACLE_HOME
export PATH=$PATH:$ORACLE_HOME
export NLS_LANG=GERMAN_GERMANY.AL32UTF8

Code error in docs

1. What is the link to the documentation section that needs improving?

https://python-oracledb.readthedocs.io/en/latest/user_guide/connection_handling.html

2. Describe the confusion

dsn = f'{username}/{userpwd}@{port}:{host}/{service_name}'

3. Suggest changes that would help

dsn = f'{username}/{userpwd}@{host}:{port}/{service_name}'

Add support for DB_TYPE_OBJECT in thin mode

Hi, I'm working on a pull request django/django#15841 which replaces the use of cx_Oracle module with oracledb in django.

However, there is a failed test that related to DB_TYPE_OBJECT.
The reason seems to be that DB_TYPE_OBJECT is not supported in oracledb.thin_impl.MessageWithData._process_column_data.

This test could be passed with cx_Oracle. Is it possible to add the same support to oracledb.thin_impl.MessageWithData._process_column_data?

Related codes:

• django/contrib/gis/db/backends/oracle/introspection.py
  Here django maps its GeometryField to oracledb.DB_TYPE_OBJECT (previously cx_Oracle.OBJECT).
• django/tests/gis_tests/inspectapp/tests.py
  Here is the code to the test.

---

1. What versions are you using?
   It's an internal database used in the CI of django. I think is version should be 19.

2. Is it an error or a hang or a crash?
   crash
3. What error(s) or behavior you are seeing?

Traceback (most recent call last):
  File "/home/jenkins/workspace/pull-requests-oracle/database/oragis19/label/oracle/python/python3.10/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/home/jenkins/workspace/pull-requests-oracle/database/oragis19/label/oracle/python/python3.10/django/db/backends/oracle/base.py", line 566, in execute
    return self.cursor.execute(query, self._param_generator(params))
  File "/home/jenkins/workspace/pull-requests-oracle/database/oragis19/label/oracle/python/python3.10/tests/.env/lib/python3.10/site-packages/oracledb/cursor.py", line 378, in execute
    impl.execute(self)
  File "src/oracledb/impl/thin/cursor.pyx", line 121, in oracledb.thin_impl.ThinCursorImpl.execute
  File "src/oracledb/impl/thin/protocol.pyx", line 375, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 376, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 336, in oracledb.thin_impl.Protocol._process_message
  File "src/oracledb/impl/thin/protocol.pyx", line 315, in oracledb.thin_impl.Protocol._process_message
  File "src/oracledb/impl/thin/messages.pyx", line 281, in oracledb.thin_impl.Message.process
  File "src/oracledb/impl/thin/messages.pyx", line 789, in oracledb.thin_impl.MessageWithData._process_message
  File "src/oracledb/impl/thin/messages.pyx", line 869, in oracledb.thin_impl.MessageWithData._process_row_data
  File "src/oracledb/impl/thin/messages.pyx", line 595, in oracledb.thin_impl.MessageWithData._process_column_data
  File "/home/jenkins/workspace/pull-requests-oracle/database/oragis19/label/oracle/python/python3.10/tests/.env/lib/python3.10/site-packages/oracledb/errors.py", line 103, in _raise_err
    raise exc_type(_Error(message)) from cause
oracledb.exceptions.NotSupportedError: DPY-3007: database type "DB_TYPE_OBJECT" is not supported

4. Does your application call init_oracle_client()?
   No.

5. Include a runnable Python script that shows the problem.

TypeError: cannot convert 'NoneType' object to bytearray when connecting to oracledb server 19.2 in thin mode

server:

Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

client:

>>> print("platform.platform:", platform.platform())
platform.platform: Linux-5.10.0-17-amd64-x86_64-with-glibc2.31
>>> print("sys.maxsize > 2**32:", sys.maxsize > 2**32)
sys.maxsize > 2**32: True
>>> print("platform.python_version:", platform.python_version())
platform.python_version: 3.9.2
>>> print("oracledb.__version__:", oracledb.__version__)
oracledb.__version__: 1.0.3

thats the script:

import oracledb
c = oracledb.connect(user="user", password="mypassword", dsn="servername:1521/orcl")

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/oracledb/connection.py", line 1000, in connect
    return conn_class(dsn=dsn, pool=pool, params=params, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/oracledb/connection.py", line 128, in __init__
    impl.connect(params_impl)
  File "src/oracledb/impl/thin/connection.pyx", line 294, in oracledb.thin_impl.ThinConnImpl.connect
  File "src/oracledb/impl/thin/connection.pyx", line 184, in oracledb.thin_impl.ThinConnImpl._connect_with_params
  File "src/oracledb/impl/thin/connection.pyx", line 157, in oracledb.thin_impl.ThinConnImpl._connect_with_description
  File "src/oracledb/impl/thin/connection.pyx", line 114, in oracledb.thin_impl.ThinConnImpl._connect_with_address
  File "src/oracledb/impl/thin/protocol.pyx", line 218, in oracledb.thin_impl.Protocol._connect_phase_two
  File "src/oracledb/impl/thin/protocol.pyx", line 336, in oracledb.thin_impl.Protocol._process_message
  File "src/oracledb/impl/thin/protocol.pyx", line 315, in oracledb.thin_impl.Protocol._process_message
  File "src/oracledb/impl/thin/messages.pyx", line 281, in oracledb.thin_impl.Message.process
  File "src/oracledb/impl/thin/messages.pyx", line 2107, in oracledb.thin_impl.ProtocolMessage._process_message
TypeError: cannot convert 'NoneType' object to bytearray

Add support for named time zones in thin mode

Fetching timestamp with time zone raises error ORA-01805: possible error in date/time operation instead of returning datetime with time zone information.

1. What versions are you using?

   • database version: Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
   • platform.python_version: 3.9.9
   • oracledb.__version__: 1.0.1

2. Is it an error or a hang or a crash? Error

3. What error(s) or behavior you are seeing?

Error oracledb.exceptions.DatabaseError: ORA-01805: possible error in date/time operation is raised when trying to use at time zone in SQL . Below is a sample code to reproduce the problem.

5. Does your application call init_oracle_client()? Yes, uses Thick mode.

6. Include a runnable Python script that shows the problem.

import oracledb
from oracledb.thick_impl import init_oracle_client


def f(sql):
    with oracledb.connect(user='hr', password='password', dsn='127.0.0.1:1511/xe') as connection:
        with connection.cursor() as cursor:
            query = cursor.execute(sql)
            row = query.fetchone()
            print(row[0].time())


init_oracle_client()

# this works because returns datetime without time zone
f("select systimestamp from dual")

# this raises error ORA-01805
f("select systimestamp at time zone 'America/Montreal' as d from dual")


# this is manual walkaround but datetime is still missing time zone information
f("select cast(systimestamp at time zone 'America/Montreal' as timestamp) as d from dual")

The above is an example using systimestamp but similar error can be achieved by crating table with column type timestamp with time zone

create table tz_table
(
	tz_timestamp TIMESTAMP(6) WITH TIME ZONE not null
);

insert into tz_table values ( TIMESTAMP '2022-06-16 08:00:00 US/Central');

Fallback by thick mode

1. What versions are you using?

oracledb: 1.0.3
python: 3.10.5
database: 19.10.0.0.0

2. Is it an error or a hang or a crash?

crash

4. What error(s) or behavior you are seeing?

DPY-2019: python-oracledb thick mode cannot be used because a thin mode connection has already been created

5. Does your application call init_oracle_client()?

Yes. Im trying to use thin mode first and switch to thick when the database is incompatible

6. Include a runnable Python script that shows the problem.

try:
  connection = oracledb.connect(params=params)
except oracledb.NotSupportedError as err:
  print( err.args[0].message ) # DPY-3015: password verifier type 0x939 is not supported by python-oracledb in thin mode
  oracledb.init_oracle_client()  # Try fallback in thick mode
  connection = oracledb.connect(params=params)

AttributeError: 'NoneType' object has no attribute 'SSLError'

1. What versions are you using?

platform.platform: Linux-5.10.60.1-microsoft-standard-WSL2-x86_64-with-glibc2.29
sys.maxsize > 2**32: True
platform.python_version: 3.8.10
oracledb.version: 1.0.3

2. Is it an error or a hang or a crash?

There is a message in console reading "Exception ignored".

3. What error(s) or behavior you are seeing?

...
...
Exception ignored in: <function Connection.__del__ at 0x7f5a0eb16280>
Traceback (most recent call last):
  File "/home/toby/projects/mai/.venv/lib/python3.8/site-packages/oracledb/connection.py", line 144, in __del__
  File "src/oracledb/impl/thin/connection.pyx", line 283, in oracledb.thin_impl.ThinConnImpl.close
AttributeError: 'NoneType' object has no attribute 'SSLError'

4. Does your application call init_oracle_client()?

No, running Thin mode.

5. Include a runnable Python script that shows the problem.

I'm running other's code and there is a little wrapper class as follows. I run a Flask app that connects to Oracle DB, and the message always shows when I do Ctrl+C to stop the process. I see this message occasionally otherwise, but not sure about the root cause.

import oracledb as cx_Oracle

class OracleDB(object):
    def __init__(self, connection_str):
        self.con = cx_Oracle.connect(
            self.connection_str, encoding="UTF-8", nencoding="UTF-8", threaded=True
        )
        self.cursor = None
        self.cursor2 = None

    # some other methods here...

    def close(self):
        logger.info("closing db connection...")
        try:
            self.cursor.close()
            del self.cursor
        except:
            pass

        try:
            self.cursor2.close()
            del self.cursor2
        except:
            pass

        try:
            self.con.close()
            del self.con
        except:
            pass

return object attributes as int if metadata says it can be expressed as int

Currently when fetching objects, scale and precision of numbers is not considered and NUMBER attributes are always returned as python floats, even if scale is 0 and the NUMBER could be better represented as int.

Please add the same logic for objects depending on metadata as used when fetching numbers.

See also discussion #88

Worth mentioning in docs that DPY-6000 with ORA-12660 can be caused by server having NNE enabled?

1. What is the link to the documentation section that needs improving?

https://python-oracledb.readthedocs.io/en/latest/user_guide/appendix_b.html#native-network-encryption-and-checksumming

2. Describe the confusion

I was wondering if it might be worth citing the specific error someone would get raised if they try make a thin connection to a server with NNE enabled - I believe it's DPY-6000 with ORA-12660 mentioned.

I'm currently testing an OCI Oracle Base Database server for replacing an on-premises one and was baffled initially that thin client connections weren't working, when I'd had them behave fine in my on-premises environment, but if I enabled thick mode the connection would work fine.

The error message I was seeing was:

oracledb.exceptions.OperationalError: DPY-6000: cannot connect to database. Listener refused connection. (Similar to ORA-12660)

Which initially made me wonder if I was getting the connection string wrong, but checking the tnslistener logs I could see the connection was being made. Googling for DPY-6000 didn't find me anything useful (there is another issue which mentions it, but has a different ORA mentioned), but on googling ORA-12660 I saw that it relates to crypto/encryption.

I then found that the sqlnet.ora set up automatically for the OCI Oracle Base Database service seems to have NNE enabled. A bit more digging in the oracledb docs and I found that documentation section mentioning that thin doesn't support NNE.

3. Suggest changes that would help

Assuming I'm correct that a raised DPY-6000 citing ORA-12660 is related to the server rejecting the thin client connection where NNE is enabled when the thin client can't handle it, then as per above I wondered if it worth a quick mention in the oracledb docs just so if someone does get that error then googling it/searching the docs will help them more quickly figure out the issue :). It feels like it might be one of the more common things people might bump into and not realise about (if they haven't configured the encryption settings on the server themselves).

But of course I don't mind at all if you'd rather not add anything to docs (at least with this issue existing now I've created it, even if marked don't fix, there's some sort of pointer mention ORA-12660 :) ).

Support for `fetch_lobs` and `fetch_decimal` per connection instead of globally

It would be great, if fetch_lobs and fetch_decimal were supported as parameters for oracledb.connect() directly instead of only as global configuration parameters in oracledb.defaults that only take effect on the next oracledb.connect() call.

So I would like to be able to do:

db = oracledb.connect("user/pwd@db", fetch_lobs=False)

instead of:

oracledb.defaults.fetch_lobs = False
db = oracledb.connect("user/pwd@db")

`LOB.read(0)` gives unhelpful error message in thin mode

The following Python script:

import oracledb
db = oracledb.connect("user/pwd@db")
c = db.cursor()
c.execute("select to_clob('foo') from dual").fetchone()[0].read(0)

gives the following output:

Traceback (most recent call last):
  File "/Users/walter/oracledb_bug.py", line 4, in <module>
    c.execute("select to_clob('foo') from dual").fetchone()[0].read(0)
  File "/Users/walter/pyvenvs/default/lib/python3.10/site-packages/oracledb/lob.py", line 116, in read
    return self._impl.read(offset, amount)
  File "src/oracledb/impl/thin/lob.pyx", line 140, in oracledb.thin_impl.ThinLobImpl.read
  File "src/oracledb/impl/thin/protocol.pyx", line 294, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 295, in oracledb.thin_impl.Protocol._process_single_message
  File "src/oracledb/impl/thin/protocol.pyx", line 288, in oracledb.thin_impl.Protocol._process_message
oracledb.exceptions.DatabaseError: ORA-03137: malformed TTC packet from client rejected: [kpolob:offset 0] [0] [0] [2] [] [] [] []

In thick mode I get:

Traceback (most recent call last):
  File "/Users/walter/oracledb_bug.py", line 5, in <module>
    c.execute("select to_clob('foo') from dual").fetchone()[0].read(0)
  File "/Users/walter/pyvenvs/default/lib/python3.10/site-packages/oracledb/lob.py", line 116, in read
    return self._impl.read(offset, amount)
  File "src/oracledb/impl/thick/lob.pyx", line 168, in oracledb.thick_impl.ThickLobImpl.read
  File "src/oracledb/impl/thick/utils.pyx", line 409, in oracledb.thick_impl._raise_from_odpi
  File "src/oracledb/impl/thick/utils.pyx", line 399, in oracledb.thick_impl._raise_from_info
oracledb.exceptions.DatabaseError: ORA-24801: Unzulässiger Parameterwert in OCI-LOB-Funktion

which at least gives a hint at what the problem is.

import sys, platform, oracledb

print(f"{platform.platform()=}")
print(f"{sys.maxsize > 2**32=}")
print(f"{platform.python_version()=}")
print(f"{oracledb.__version__=}")

prints

platform.platform()='macOS-12.4-x86_64-i386-64bit'
sys.maxsize > 2**32=True
platform.python_version()='3.10.4'
oracledb.__version__='1.0.0'

The database is an Oracle XE.

select value from nls_database_parameters where parameter='NLS_RDBMS_VERSION';

returns 21.0.0.0.0.

Kerberos Authentication Support in Thin Mode

1. Describe your new request in detail

So the thick mode of connecting to Oracle Database supports Kerberos External Authentication.
While it's not supported in Thin mode.
This makes it a pain as we have to bundle the client libraries as well when deploying the applications.
This means that our size of application increases a lot! Something not great.

If Kerberos was supported in Thin Mode, then we no longer need to load the client libraries!

3. Give supporting information about tools and operating systems. Give relevant product version numbers

Latest Oracledb Driver and Instant Client 19.X

Thank you for considering. I am wondering if other DBs such as MySQL or Postgress SQL support Kerberos in Thin mode?

Thin via SCAN listener fails with DPY-6000 -- SERVICE_NAME not passed to local listener

1. What versions are you using?

listeners: 18, database: 18
platform.platform: Windows-10-10... (We see the same error from Lunix clients)
sys.maxsize > 2**32: True
platform.python_version: 3.10.5
oracledb.version: 1.0.2

2. Is it an error or a hang or a crash?

error

3. What error(s) or behavior you are seeing?

if we replace the SCAN address with the local listener address, the connection functions without error.

Traceback (most recent call last):
  File "C:\apps\tmp\test.py", line 38, in <module>
    with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
  File "C:\apps\Python\Python310\lib\site-packages\oracledb\connection.py", line 1000, in connect
    return conn_class(dsn=dsn, pool=pool, params=params, **kwargs)
  File "C:\apps\Python\Python310\lib\site-packages\oracledb\connection.py", line 128, in __init__
    impl.connect(params_impl)
  File "src\oracledb\impl/thin/connection.pyx", line 347, in oracledb.thin_impl.ThinConnImpl.connect
  File "src\oracledb\impl/thin/connection.pyx", line 163, in oracledb.thin_impl.ThinConnImpl._connect_with_params
  File "src\oracledb\impl/thin/connection.pyx", line 129, in oracledb.thin_impl.ThinConnImpl._connect_with_description
  File "src\oracledb\impl/thin/protocol.pyx", line 241, in oracledb.thin_impl.Protocol._process_message
  File "src\oracledb\impl/thin/connection.pyx", line 234, in oracledb.thin_impl.ThinConnImpl._connect_with_address
  File "src\oracledb\impl/thin/protocol.pyx", line 150, in oracledb.thin_impl.Protocol._connect_phase_one
  File "src\oracledb\impl/thin/protocol.pyx", line 262, in oracledb.thin_impl.Protocol._process_message
  File "src\oracledb\impl/thin/protocol.pyx", line 241, in oracledb.thin_impl.Protocol._process_message
  File "src\oracledb\impl/thin/messages.pyx", line 1578, in oracledb.thin_impl.ConnectMessage.process
  File "C:\apps\Python\Python310\lib\site-packages\oracledb\errors.py", line 103, in _raise_err
    raise exc_type(_Error(message)) from cause
oracledb.exceptions.OperationalError: DPY-6000: cannot connect to database. Listener refused connection. (Similar to ORA-12504)

the local listener logs the following:

22-JUL-2022`` 10:53:48 * (CONNECT_DATA=(SERVICE_NAME=)(SERVER=dedicated)(CID=(PROGRAM=C:\apps\Python\Python310\python.exe)(HOST=___)(USER=___))) * establish * 12504
TNS-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA
 TNS-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA

4. Does your application call init_oracle_client()?

no, we wish to use thin.

5. Include a runnable Python script that shows the problem.

# test.py
import oracledb
import os

oracledb.defaults.config_dir = "C:/apps"
un = os.environ.get('USER_NAME')
pw = os.environ.get('PASSWORD')
cs = os.environ.get('CONNECT_STRING')

with oracledb.connect(user=un, password=pw, dsn=cs) as connection:
 with connection.cursor() as cursor:
  sql = """select sysdate from dual"""
  for r in cursor.execute(sql):
    print(r)

ConnectParams(user='', proxy_user=None, host='host..org', port=1521, protocol='tcp', https_proxy=None, https_proxy_port=0, service_name='mydb.___.org', sid=None, server_type=None, cclass=None, purity=0, expire_time=0, retry_count=0, retry_delay=0, tcp_connect_timeout=60.0, ssl_server_dn_match=True, ssl_server_cert_dn=None, wallet_location=None, events=False, mode=0, disable_oob=False, stmtcachesize=20, edition=None, tag=None, matchanytag=False, config_dir='C:\apps\', appcontext=None, shardingkey=None, supershardingkey=None, debug_jdwp=None)

have tested many connection options -- all end with the DPY-6000 unless a local listener is used.
our SCAN listener configuration may be at fault here, but we have other thin connections connecting via our SCAN listeners without problems.

`outputtypehandler` not respecting `NLS_TERRITORY` in thin mode

1. What versions are you using?

platform.platform: Windows-10-10.0.19044-SP0
sys.maxsize > 2**32: True
platform.python_version: 3.10.4
oracledb.__version__: 1.0.0

I'm using oracle 21.3.0.0, with client version (21, 3, 0, 0, 0)

2. Is it an error or a hang or a crash?

An unexpected result was returned

3. What error(s) or behavior you are seeing?

It seems that setting NLS_TERRITORY on the session and using outputtypehandler to return floating values to string does not take into consideration the set NLS_TERRITORY parameter

4. Does your application call init_oracle_client()?

both

5. Include a runnable Python script that shows the problem.

import oracledb

thin_mode = True
if not thin_mode:
    oracledb.init_oracle_client()

def go():
    with oracledb.connect(
        user="scott",
        password="tiger",
        service_name="XEPDB1",
        host="localhost",
        port=11521,
    ) as conn:
        cursor = conn.cursor()
        # NLS_TERRITORY can be set to any nation that uses comma. also GERMANY works
        cursor.execute("ALTER SESSION SET NLS_TERRITORY='ITALY'")
        res = cursor.execute(
            "select cast(1.1 as varchar(128)) from dual"
        ).fetchall()[0]
        print("with cast", res[0], res[0] == "1,1")

        def output_type_handler(
            cursor, name, defaultType, size, precision, scale
        ):
            return cursor.var(oracledb.STRING, 255, arraysize=cursor.arraysize)

        cursor.outputtypehandler = output_type_handler
        res = cursor.execute("select 1.1 from dual").fetchall()[0]
        print("with outputtypehandler", res[0], res[0] == "1,1")

go()

with thin_mode = True the above script prints the following.

with cast 1,1 True
with outputtypehandler 1.1 False

the second print ignores the NLS_TERRITORY setting.
Setting thin_mode = False prints the expected

with cast 1,1 True
with outputtypehandler 1,1 True

Support Native Network Encryption in thin mode

1. Describe your new request in detail
   All of our Oracle databases require connections to use Oracle Database Native Network Encryption, which is currently only supported in thick mode. It would be wonderful if python-oracledb added support for NNE in thin mode so we could remove the need for the Oracle client.

2. Give supporting information about tools and operating systems. Give relevant product version numbers
   I am primarily looking to utilize this in python3.9 AWS Lambdas (they run Amazon Linux 2) that connect to Oracle 19c (both on-prem and AWS RDS). Currently we have a lambda layer that provides the oracle instant client that we add to each lambda but that is what I would like to eliminate.

Thank you!

DPY-3016: python-oracledb thin mode cannot be used because the cryptography package is not installed

Greetings. I'm trying to understand what's going wrong with oracledb when it's run from a PyInstaller-packaged executable. While I have no problems running my code through standard Python interpreter, the users expected for the program won't necessarily be, hence the effort to package the code via PyInstaller.

I've isolated the problem to a simple, single-file program that just imports oracledb (running current version 1.2.0) and attempts to make a connection to an Oracle database, verifying that it works when run via Python interpreter. Confirming that PyInstaller is including the cryptography package, I still get the error message:

oracledb.exceptions.NotSupportedError: DPY-3016: python-oracledb thin mode cannot be used because the cryptography package is not installed

The stack trace points me to the following:

python-oracledb/src/oracledb/impl/thin/connection.pyx

Lines 70 to 74 in af750c6

	def __init__(self, str dsn, ConnectParamsImpl params):
	if not HAS_CRYPTOGRAPHY:
	errors._raise_err(errors.ERR_NO_CRYPTOGRAPHY_PACKAGE)
	BaseConnImpl.__init__(self, dsn, params)
	self._protocol = Protocol()

But I'm not really sure what's involved in the HAS_CRYPTOGRAPHY check. Any ideas where I should continue investigation? I'd appreciate some guidance here. Thanks in advance.

Single tuple causing cursor.executemany ORA-03147: missing mandatory TTC field

Discussed in #29

^{Originally posted by Loncor June 27, 2022}
I am getting this error when executing the following. Uncommenting the lines below makes it work. So I have concluded if a single tuple in a list is used executemany has a problem. I've got a workaround so not a show stopper. It is not peculiar to this insert as I have tried other tables and differing SQL:-

sql_ins_doc_spell = """
            begin
               insert into doc.spell_correct (unknown_word, default_word ) values (:1 , :2);
            exception
                when dup_val_on_index then
                   null;
            end;                              
       """

spell_correct = []

spell_correct.append(('jick', 'jack')) 
#spell_correct.append(('ramana', 'ramona')) 

with connection.cursor() as cursor:
    cursor.executemany(sql_ins_doc_spell, spell_correct)
    connection.commit()
```</div>