padila50 / pyodbc Goto Github PK

I am successfully connected to a db2-database server (on a mainframe as
well as on a aix workstation).

Doing a select on a table with columns with decimals (with positions after
the decimal point) only yields values without positions after the decimal
point (the value is 10 to the power of the positions after the decimal point).

Example: The value in the table is 3.14 and the type is decimal(n, 2). Then
the result in Python is Decimal("314").

The '-3' switch was added to Python 2.6 in order to provide warnings for
deprecated usages in Py3k. Currently the only warning I have seen is the
following, upon import:
pyodbc.pyd:6: DeprecationWarning: the sha module is deprecated; use the
hashlib module instead

This will break compatibility with Python 3.0 (although I'm not sure this
is even in the immediate roadmap)

However, given that hashlib was introduced in 2.5, you will need to use
some sort of try/except import hack to maintain the current level of
backwards compatibility. Something along the lines of:
try:
    from hashlib import sha1 as sha
except ImportError:
    from sha import sha

As reported by Joe Salmeri (via email), the 2.1.x is slower than the 2.0.x
branch.

After examining some ODBC trace logs, we determined that it was extra
SQLDescribeParam calls that were added to work around binding None/NULL.

When passing parameters, pyodbc has to tell the driver the data type of the
parameter.  When None is passed, pyodbc obviously doesn't know -- it is
NULL and there is no way to tell if it is going into a character column, an
integer, etc.

In the 2.0 branch, pyodbc always set the data type to char, but SQL Server
will not let you insert a character value into a binary column even if you
are passing NULL.  To fix this, I use SQLDescribeParam, if supported by the
driver, to determine the correct data type.  Unfortunately, this adds 1
driver call per parameter for every query, which is way too much overhead.

On my 64-bit Kubuntu Linux box I enyounter the following problem:

Decimal-values with NULL-value produce an exception when reading. This
happens because in src/getdata.cpp, function GetDataDecimal, the variable
cbFetched is of type SQLLEN which is 64 bits long. The return value -1 for
a NULL value is a 32 bit number, so it is not -1 in 64 bits and the
function tries to convert the empty string '' to a decimal which raises an
exception.

Possible solution to the problem: cbFetched must be of a 32-bit type,
regardless of the systems bit size. I am sorry, but I don't know, which
type could be the right one for this.

Regards, Axel

Is this a known problem?

I'm using the Oracle 10g client on WinXP and data is written to the
database even though I haven't set autocommit (or set it explicitely to
false) and never call the commit() method.

The one issue I've run into is with connection strings. According to PEP 249:

    [1] As a guideline the connection constructor parameters should be
        implemented as keyword parameters for more intuitive use and
        follow this order of parameters:

          dsn         Data source name as string
          user        User name as string (optional)
          password    Password as string (optional)
          host        Hostname (optional)
          database    Database name (optional)

        E.g. a connect could look like this:

          connect(dsn='myhost:MYDB',user='guido',password='234$')

Of course, I realize that implementing this specific interface for ODBC may
be troublesome... plus the fact that this is only a "should", rather than a
"must". Still, it's just one of those things that would be nice. This did
actually cause me some trouble with django-pyodbc, with the way it
constructs its connection strings. You can see a ticket related to that
here: http://code.google.com/p/django-pyodbc/issues/detail?id=18

Preferably I would have refactored it to work as proposed to Django's
psycopg2 adaptor here: http://code.djangoproject.com/ticket/6710  however
that isn't really possible with the current connect interface in pyodbc.

I'm thinking it may also be possible to implement something like this
without breaking API compatibility with the 2.x series. Since currently,
kwargs are not the recommended usage for passing connection strings, then
perhaps if they are used, it could be taken as referring to the interface
mentioned above and otherwise it uses the current pyodbc connect interface.

Would like to write code like this:

with pyodbc.connection(conn_string) as conn:
    #do cool things here

Not necessary and is error-prone.

What steps will reproduce the problem?
1. i have MSSQL Database on a remote machine
2. i have created a System DSN in my local machine using python script 
which uses ctypes module
3. using pyodbc module i'm trying to connect to remote database.

What is the expected output? What do you see instead?
establish connection without crashing the python

What version of the product are you using? On what operating system?
OS-: win 2003 server, 32bit
Installed python products: python 2.5,pyodbc-2.0.52,pywin32-211

Please provide any additional information below.
1) The script just exists with return code 128.
2) Sometime after(approx 20-30 mins) if i run the scripts,i was able to 
connect and no crash found.
3)After establishing the connection,if we terminate (Keyboard Interrupt)
the script without closing the connection and once again try to 
connect,python.exe crashes

The scripts terminates abnormally and doesnt give any traceback.
i can see the dumps only when i reboot the machine.

Following is the code snippet of my script that are used to do the above 
tasks
Script to Create a System DSN
----------------------------------------------
def create_sys_dsn(driver, **kw):
    "Create a  system DSN"
    ODBC_ADD_SYS_DSN = 4
    nul = chr(0)
    attributes = []
    for attr in kw.keys():
        atrbStr = "%s=%s" % (attr, kw[attr])
        #print atrbStr
        attributes.append(atrbStr)
    #print driver
    return ctypes.windll.ODBCCP32.SQLConfigDataSource(0, ODBC_ADD_SYS_DSN, 
driver, nul.join(attributes))

retCode = create_sys_dsn("SQL Server", SERVER=server, DESCRIPTION="TestDB 
DSN", DSN="TestDB", Database="testDB")
if retCode != 1:
    mesg = "Cannot Create TestDB DSN"
    print mesg
    sys.exit(1)
else:
    mesg = "TestDB DSN Created"
    print mesg
    sys.exit(0)
---------------------------------------------------------------------------

Script to connect to database
---------------------------------------------------------------------------
import pyodbc
try:
            DBCon = pyodbc.connect("DSN=TestDB;UID=tester;PWD=tester")
except pyodbc.Error, erno:
            mesg = "Unable to connect testDB Database check the UserID and 
Password"
            print mesg,erno
            return 1
return 0
:
:
:
:
DBCon.close()

On Linux, there is currently no way to know if pyodbc was built for iODBC
or unixODBC.

(It would be better if we could dynamically choose, but I don't think
that's an option.)

Add a variable like __build__ with this info.  (I'll need to see what other
Python modules do.  Perhaps copy a variable name from sys?)

The source distribution of pyodbc-2.1.1 is missing a MANIFEST.in file. This
means that when you try to run "python setup.py bdist_rpm" (to build an rpm
of pyodbc), it fails because the .cpp and .h files haven't been packaged
into the build environment.

If a simple MANIFEST.in file is supplied, then running "python setup.py
bdist_rpm" works flawlessly.

The MANIFEST.in should look something like:
 recursive-include . *.py
 recursive-include src *.h
 recursive-include src *.cpp

What version of the product are you using? On what operating system?
 pyodbc 2.1.1 (from zip)
 Fedora 8

The current version 2.1.3 doesn't compile under (my) Linux. I had to apply
the attached patches which, for me, seem to be harmless and just syntactic
sugar.

Regards

Axel

The old SourceForge homepage at http://pyodbc.sourceforge.net and its
download link do not redirect here and does not contain any hint about the
move. I only accidentally found that I was using an outdated version and
there is a newer version available here and on PyPI.

In Python based Web applications it is important to do pagination. SQL
syntax for this varies from a server to server (limit by,
ROW_NUMBER() OVER () etc). However ODBC function SQLFetchScroll provides a
mechanism that could be used to retrieve just a window of resulting data
set in DBMS independent way.

It could work as follows: a new method skip(count) on Cursor objects would
use SQLFetchScroll to skip a given number of rows.

What steps will reproduce the problem?
1. Create an Access Database
2. Create a Python Script to write to the database such as: 

#!
import pyodbc

DBfile = 'C:\Documents and Settings\jakech\Desktop\ovenscheduler_be.mdb'
conn = pyodbc.connect('DRIVER={Microsoft Access Driver
(*.mdb)};DBQ='+DBfile, autocommit=True)
csr = conn.cursor()

# Insert Record into Access Database

csr.execute("INSERT INTO tmpKanban ( KanbanNo ) SELECT 991235;")
conn.commit()

csr.close()
conn.close()

What is the expected output? What do you see instead?
I expected it to write to the database, but no record was added.  I had the
same problem with a Delete * statement.  I am betting there is some write
access limitation, but haven't found it.  

What version of the product are you using? On what operating system?
2.4, Windows XP

Please provide any additional information below.

What steps will reproduce the problem?
1. Call pyodbc.connect("DSN=MyDSN", True)
2.
3.

What is the expected output? What do you see instead?
This call worked with version 2.0.36. In 2.1.3, this error is the result:
TypeError: function takes at most 1 non-keyword argument

What version of the product are you using? On what operating system?
2.1.3, on Windows XP SP3

Please provide any additional information below.
According to the documentation, the method should support up to 3 non-
keyword args:
connect(str, autocommit=False, ansi=False, **kwargs)

> What version of the product are you using? On what operating system?
Mac OSX 10.5, Python 2.5, latest stable pyodbc, MS SQL DB using the
Actual SQL ODBC driver.

I am getting the following error message when trying to access certain
char fields:

Assertion failed: (size >= 0), function PyUnicodeUCS2_EncodeUTF8, file
Objects/unicodeobject.c, line 1416. 

It would be a nice option to make the SQL and parameters available to
exceptions and even in the default exception messages.

Since there might be passwords and other sensitive items, this should be an
option, not a default.

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?
[The output must be (Using console SQL from MSSQL Server)]
(1 filas afectadas)

@@error1 is 0.
Servidor: mensaje 547, nivel 16, estado 1, línea 1
Instrucción INSERT en conflicto con la restricción COLUMN CHECK
'CK_tm_asunto'. El conflicto ha aparecido en la base de datos 'ctrl_emi',
tabla 'taller_movimiento', column 'asunto'.


[What happend instead inside of try block, not enter in except block]
>>> u.execute(s)
1

What version of the product are you using? On what operating system?
Windows XP SP3
SQL Server 2000
pyodbc 2.0.58 (I use Zope so I need using with python 2.4. I not know how
compile with this version)

Please provide any additional information below.
The query is
SET XACT_ABORT ON
BEGIN TRANSACTION
DECLARE @err int
UPDATE taller SET
  comercial =  ''
where taller = 1
select @err = @@error
PRINT '@@error1 is ' + ltrim(str(@@error)) + '.'
IF (@ERR <> 0) BEGIN
   raiserror('error al actualizar',16,1)
   ROLLBACK TRANSACTION 
END
insert into taller_movimiento (estado, taller, fecha, asunto, observacion
) values(
  '14',
  1,
  '',
  '',
  ''
)
select @err = @@error
PRINT '@@error2 is ' + ltrim(str(@@error)) + '.'
IF (@ERR <> 0) BEGIN
   raiserror('error al insertar',16,1)
   ROLLBACK TRANSACTION 
END
COMMIT TRANSACTION

I've noticed in my multithreaded app using pyodbc with MS SQL Server 
occasional application freezes typical of the GIL not being released. It 
always seemed to happen when two threads were both calling stored 
procedures. In every case I noticed that there was a blocking SQL Server 
process with last command "exec sp_sproc_columns", I assume this is called 
by the driver when you are trying to build up the cursor.description. The 
fact that all my app threads freeze means the GIL is not released. As 
pyodbc is the only C python extension in use, it must be the one not 
releasing the GIL. 

Changing my app to use adodbapi fixed the problem 

Sorry I can't provide a test case that duplicates this problem. My app is 
huge and it would be a lot of work for me to whittle it down to a simple 
test case. If you make any changes that you think might fix this problem 
let me know and I'll rerun my app using pyodbc to see if the problem goes 
away. 

The readme was renamed to .rst for github support, but the manifest was not
updated to include this new name.

What is the expected output? What do you see instead?

Expected output is reading an image/binary objects from database.
Unfortunatelly for some binary objects I recieve this error message
followed by a crash of python:

assertion failed: getdata.cpp(222)
cbRead == 0 || pch[cbRead-element_size] != 0


What version of the product are you using? On what operating system?

Python 2.5.2
pyodbc-2.0.58.win32
Windows XP Professional SP 3
Microsoft SQL Server 2008 (but I am pretty sure it also occurs on 2005)

Do you have any idea how to solve the problem?

using MySQL with pyodbc, result values from table selects appear to come
back as Python unicode objects (though oddly, not from literal string
selects).   But then it also will fail to compare a unicode value to a
non-unicode value on the server, leading to this contradictory behavior:

    import pyodbc
    c= pyodbc.connect("dsn=mysql_test;TrustedConnection=Yes")

    cursor = c.cursor()

    # passes
    cursor.execute("drop table foo")
    cursor.execute("create table foo(data varchar(30))")
    cursor.execute("insert into foo values (?)", u'test')
    cursor.execute("select * from foo")
    x = cursor.fetchone()[0]
    cursor.execute("select count(1) from foo where data=?", x)
    assert cursor.fetchone()[0] == 1

    # fails
    cursor.execute("drop table foo")
    cursor.execute("create table foo(data varchar(30))")
    cursor.execute("insert into foo values (?)", 'test')
    cursor.execute("select * from foo")
    x = cursor.fetchone()[0]
    cursor.execute("select count(1) from foo where data=?", x)
    assert cursor.fetchone()[0] == 1


Where in the second test, the value of 'test' in the database does not
compare to that of u'test'.  not sure if theres an ODBC setting I should be
using here.  

When trying to iterate through the results of a query, I'm getting a
MemoryError on Mac OS 10.4. The same exact code runs on Linux using a
similar ODBC stack. They're both using iODBC with the latest pyodbc
connecting to the same server.

connection = pyodbc.connect(settings.ETOOLS_ODBC_DSN)
cursor = connection.cursor()
cursor.execute("SELECT TOP 5 * FROM tb_FSAR_Data_SampArtReq WHERE Job_ID =
49297")
    for ejob in results: 
        job = Job()

What steps will reproduce the problem?
1. Create NVARCHAR (or NTEXT) column in MSSQL 2005 Express database table.
   create table test_nvarchar (id int identity, nvarchar_col nvarchar(10))

   -- Insert capital U with double acute.
   insert into test_nvarchar values (N'Ű')
2. Run this python script on RedHat 9:

import pyodbc
cnxn = pyodbc.connect("DSN=SQLSERVER_SAMPLE")
cursor = cnxn.cursor()
cursor.execute("select * from test_nvarchar where id = 1")
for row in cursor:
    print row.nvarchar_col
    print repr(row.nvarchar_col

What is the expected output?
Ű
u'\u0170    '


What do you see instead?

u''

What version of the product are you using? On what operating system?
redhat 9
pyodbc 2.1.2
python 2.5.1 (UCS-2 build)
Easysoft SQL Server ODBC driver 1.1.4
SQL Server 2005 Express

Please provide any additional information below.
The script works as expected with the same setup if pyodbc 2.0.58 is used
or a "hybrid" 2.1.2 built with the 2.0.58 getdata.cpp.

What steps will reproduce the problem?
1. SQL Server 2008 has new data types: DATE, DATETIME2, TIME and
DATETIMEOFFSET.
2. when selecting date for columns of these datatypes, returned values are
all unicode strings:
 * u'2008-01-01'
 * u'2008-01-01 00:00:00.0000000'
 * u'00:00:00.0000000'
 * u'2008-01-01 00:00:00.0000000 +00:00'

What is the expected output? What do you see instead?
 * I would suggest to use python's:
 - datetime.date for DATE
 - datetime.time for TIME
 - datetime.datetime for DATETIME2 and DATETIMEOFFSET


What version of the product are you using? On what operating system?
 * SQL Server 2008 Express, python 2.6 and pyodbc-2.1.1

Please provide any additional information below.

What steps will reproduce the problem?
1. The information given on configuring for using Native Client is wrong 
for 2005.

What is the expected output? What do you see instead?

It suggests that the version number is required for using the Native Client 
driver. While that may be true for 2008, it's not true for 2005.

What version of the product are you using? On what operating system?

The installable version for 2.5.


Please provide any additional information below.

An example working connection string:
cn = pyodbc.connect("DRIVER={SQL Native 
Client};SERVER=localhost;UID=user;PWD=pass;DATABASE=database;")

(copied from SourceForge issue 2306749)

I cannot find any online information to show that pyodbc supports file
dsn's. I would like to request this feature be added, as for now I am
forced to use other scripting solutions that does support file dsn's such
as perl's DBI:ODBC module.