GithubHelp home page GithubHelp logo

rogerbinns / apsw Goto Github PK

View Code? Open in Web Editor NEW
694.0 29.0 95.0 38.27 MB

Another Python SQLite wrapper

Home Page: https://rogerbinns.github.io/apsw/

License: Other

Python 51.59% C 47.59% Makefile 0.62% Shell 0.19%
python3 sqlite3 c

apsw's Introduction

APSW logo - links to documentation

APSW stands for Another Python SQLite Wrapper. APSW supports CPython 3.8 onwards.

About

APSW lets you get the most out of the SQLite embedded relational database engine from Python, and the most out of Python from SQLite. APSW glues together the complete SQLite C API and Python's C API, staying up to date with both SQLite and Python.

It is recommended to use the builtin sqlite3 module if you want SQLite to appear interchangeable with the other database drivers.

Use APSW when you want to use SQLite fully, and have an improved developer experience. The documentation has a section on the differences between APSW and sqlite3.

Help/Documentation

There is a tour and example code using APSW at https://rogerbinns.github.io/apsw/example.html

The latest documentation is at https://rogerbinns.github.io/apsw/

Mailing lists/contacts

Releases and Changes

Releases are made to PyPI (install using pip) and Github

New release announcements are sent to the Python SQLite discussion group and there is an RSS feed from PyPI.

Full detailed list of changes

Bugs

You can find existing and fixed bugs by clicking on Issues and using "New Issue" to report previously unknown issues.

License

See LICENSE - in essence any OSI approved open source license.

apsw's People

Contributors

dependabot[bot] avatar grantjenks avatar rogerbinns avatar rstuart85 avatar sunpoet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

apsw's Issues

Allow derived cursors

From rogerbinns on March 03, 2009 22:54:57

It should be possible to make derived cursors by passing the class to the
Connection.cursor() method. Check interaction with weakrefs and close
calls. fetchall may also need to use virtual step.

Example code:

class foo: pass

con=apsw.Connection("...")
cur=con.cursor(foo)

Original issue: http://code.google.com/p/apsw/issues/detail?id=43

UPDATE during SELECT

From [email protected] on June 08, 2009 11:20:52

In http://apsw.googlecode.com/svn/publish/cursor.html it says that the
preferred way to do changes to a database is this one:

| You can also get all the rows immediately by filling in a list:
|
| rows=list( cursor.execute("select * from example") )
| for row in rows:
| cursor.execute("insert ...")
|
| This last approach is recommended since you don’t have to worry
| about the database changing while doing the select.

From this I understand that changing the database (even if using a
different cursor) may mess with the result set of the running query.

However, when I tried to test this behaviour (try attached testcase) it
seems that the result set is not affected.

Unfortunately I also could not find any information about the expected
behavior in this situation on the SQLite website.

Do you have any information on which behaviour I can rely on? I would
rather avoid having te fetch the entire result set into a list, both for
performance and style reasons.

Attachment: 05_apsw.py

Original issue: http://code.google.com/p/apsw/issues/detail?id=48

Allow customization of cursor objects

From pavlos.casa on November 26, 2008 05:30:27

Would be a good idea to allow customization of the standard cursor object.

Perhaps apsw.Connection can be extended to receive a custom cursor class as
a parameter and if provided, the cursor return is an instance of the custom
class. for that to work well the cursor object should allow subclassing in
python.

here is a typical use case:

cursor with fetchall

class MyCursor(apsw.Cursor):

def fetchall(self):
return [i for in in self]

con = apsw.Connection(custom_cursor = MyCursor)

Original issue: http://code.google.com/p/apsw/issues/detail?id=37

Update to SQLite 3.6.3

From rogerbinns on August 09, 2008 03:43:57

Diff of header files suggests these

sqlite3_db_config
SQLITE_CONFIG_CHUNKALLOC
SQLITE_CONFIG_LOOKASIDE
SQLITE_DBCONFIG_LOOKASIDE
SQLITE_STATUS_MALLOC_SIZE (read comment changes in detail)
SQLITE_STATUS_PARSER+STACK
SQLITE_STATUS_PAGECACHE_SIZE
SQLITE_STATUS_SCRATCH_SIZE
sqlite3_db_status
SQLITE_DBSTATUS_LOOKASIDE_USED

Original issue: http://code.google.com/p/apsw/issues/detail?id=22

Break apsw.c into pieces, use sphinx for doc

From rogerbinns on September 01, 2008 16:28:09

Google chokes on individual files larget than 200kb. It is also harder to
find code or potentially disable it. So split the C code into several
files (eg connection.c, cursor.c, vtable.c, vfs.c) and then have apsw.c

include them all.

Also investigate using sphinx for doc and folding the various API doc into
comments above the corresponding C implementation.

Original issue: http://code.google.com/p/apsw/issues/detail?id=23

setbusytimeout doesn't wait

From [email protected] on June 09, 2009 16:13:32

I have a testcase (which is still too long to attach it here) which fails
with the following output:

Traceback (most recent call last):
File "./tests/02_fs_api.py", line 43, in tearDown
self.server.close()
File "/home/nikratio/lib/EclipseWorkspace/s3ql/s3ql/fs.py", line 597, in
close
cur.cursor.execute("VACUUM")
File "apsw.c", line 4238, in resetcursor
BusyError: BusyError: database is locked

However, line 597 in fs.py actually looks like this:

    cur.cursor.getconnection().setbusytimeout(50000)
    cur.cursor.execute("VACUUM")

so it seems to me that the tests should take at least 50 seconds to run.
But it aborts in a fraction of a second.

I know that this behaviour is probably not the cause of my real bug
(because manually adding a sleep(50) doesn't fix it), but I wonder if it
may somehow be possible that apsw thinks the database is locked (and
therefore returns the exception) while it actually isn't (which is why it
doesn't wait).

Original issue: http://code.google.com/p/apsw/issues/detail?id=49

Error on attempting to use the same connection in multiple threads.

From amicitas on October 24, 2008 13:05:39

What steps will reproduce the problem? 1. Attempt to simultaneously use the same connection in two threads. What is the expected output? What do you see instead? It would be very nice if an error was raised if this is attempted. At this
point the application just hangs with no error output. What version of the product are you using? On what operating system? apsw 3.5.9- r2 , python 2.5 & 2.6, windows xp sp3 Please provide any additional information below. The documentation does clearly state that a connection cannot be used in
more than one thread. If it is attempted the application just hangs. Both
threads stop as well as the main program. I would be nice if an error was
thrown as it is if the cursor is used in two threads.

-- amicitas

Original issue: http://code.google.com/p/apsw/issues/detail?id=31

Sub-optimal error message, different behaviour for Cursor.execute with/without bindings

From basti1302 on April 02, 2008 03:10:38

The method Cursor.executes seems to behaves differently when used with the
optional bindings argument compared to using it without the bindings
argument. More specifically, the error message when trying to insert a
NULL value into an PRIMARY KEY NOT NULL column (without autoincrement in
place) is different. The error message in the first case (with bindings)
is less helpful than it could be. I'm not sure if this is intended
behaviour, IMHO it's a bit confusing.

The following program demonstrates the difference:

import apsw

print "apsw.apswversion(): " + apsw.apswversion()
print "apsw.sqlitelibversion(): " + apsw.sqlitelibversion()

connection = apsw.Connection("dbfile")
cursor = connection.cursor()

try:
print "test 1"
try:
cursor.execute("DROP TABLE A_TABLE")
cursor.execute("CREATE TABLE A_TABLE (ID ABC PRIMARY KEY NOT NULL,
SOME_VALUE TEXT)")
cursor.execute("INSERT INTO A_TABLE VALUES (NULL, 'Foobar')")
except Exception, e:
print e

print "test 2"
try:
    cursor.execute("DROP TABLE A_TABLE")
    cursor.execute("CREATE TABLE A_TABLE (ID ABC PRIMARY KEY NOT NULL, 

SOME_VALUE TEXT)")
cursor.execute("INSERT INTO A_TABLE VALUES (NULL, ?)", ("Foobar",))
except Exception, e:
print e

finally:
connection.close()

The output of this program is:

apsw.apswversion(): 3.3.13- r1 apsw.sqlitelibversion(): 3.3.13
test 1
ConstraintError: A_TABLE.ID may not be NULL
test 2
ConstraintError: not an error

In both tests, a table is created with an ID column and a value column.
The column affinity of ID is set to ABC (which is a nonsense value), so it
defaults to NUMERIC, afaik. The point is that it the column is not INTEGER
PRIMARY KEY NOT NULL, so we have no autoincrement, as specified in the
sqlite docs. Because of that, inserting NULL should be illegal.

In test 1, the exception raised is "ConstraintError: A_TABLE.ID may not be
NULL", as expected. However, in test 2, the exception is "ConstraintError:
not an error" which sounds a bit paradox to me.

Note 1: This is on Windows 2000, Python 2.5

Note 2: This may or may not be related to this bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=456500 but I don't really know how this Debian package bug reports relate to apsw
bug reports.

Regards
Bastian Krol

Original issue: http://code.google.com/p/apsw/issues/detail?id=4

BLOB columns are converted to/from UTF8

From [email protected] on July 17, 2008 08:34:37

I want to store short binary strings in a table. Therefore I do not want to
use the stream interface, but simply say, e.g.

cursor.execute("INSERT INTO foo VALUES(?)", ("\xCF\xDE",))

but this produces a conversion error, even if the column affinity is BLOB:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xcf in position 4:
ordinal not in range(128)

Original issue: http://code.google.com/p/apsw/issues/detail?id=18

MSVC 9 barfs

From rogerbinns on April 11, 2009 11:18:10

C:\builds\apsw-3.6.11- r1 >python setup.py build
SQLite: Using amalgamation C:\builds\apsw-3.6.11- r1 \sqlite3.c
running build
running build_ext
building 'apsw' extension
C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W
3 /GS- /DNDEBUG -DSQLITE_THREADSAFE=1 -DEXPERIMENTAL=1 -DNDEBUG=1
-DAPSW_USE_SQL
ITE_AMALGAMATION="C:\builds\apsw-3.6.11- r1 \sqlite3.c" -Isrc
-IC:\builds\apsw-3
.6.11- r1 -IC:\Python26\include -IC:\Python26\PC /Tcsrc/apsw.c
/Fobuild\temp.win3
2-2.6\Release\src/apsw.obj
apsw.c
c:\builds\apsw-3.6.11- r1 \src\exceptions.c(173) : error C2143: syntax error
: mis
sing ';' before ''
c:\builds\apsw-3.6.11- r1 \src\exceptions.c(173) : error C2143: syntax error
: mis
sing ';' before 'const'
c:\builds\apsw-3.6.11- r1 \src\exceptions.c(174) : warning C4047:
'initializing' :
'int' differs in levels of indirection from 'PyObject **'
c:\builds\apsw-3.6.11- r1 \src\exceptions.c(174) : error C2078: too many
initializ
ers

etc etc

Original issue: http://code.google.com/p/apsw/issues/detail?id=47

Deal with forking

From rogerbinns on April 05, 2008 03:37:25

See last paragraph of http://www.sqlite.org/faq.html#q6 Currently I can't find any way of catching interpretter forks. The
PyOS_AfterFork method is called but doesn't provide any way to hook into it.

A possible solution is to check the pid in calls and if different from when
the object was allocated, mark it and all related cursors/blobs as closed
(which will leak memory but can't be avoided).

Update the doc for Connection.close() to mention it would be a good idea to
call it before forking.

Original issue: http://code.google.com/p/apsw/issues/detail?id=10

The example in "APSW - Another Python SQLite Wrapper" fails under Windows

From rogerbinns on March 10, 2008 01:58:18

(pysqlite ticket 192)

apsw Example on http://initd.org/pub/software/pysqlite/apsw/3.3.13-r1/apsw.html Has a bug when run under Windows. This bug is caused by the conflict
between Windows use of backslash () as the path separator and Python's use
of forwardslash (/) as the path separator.

with line: sysdirs=",".join(["'%s'" % (x,) for x in sys.path[1:] if len(x)
and os.path.isdir(x)]) The execution error terminates with the tail of its
output as follows:

no commits out of hours commit was not allowed

Traceback (most recent call last):

File "D:\Python\00 Projects\IRR\apswTest.py", line 342, in -toplevel-

    cursor.execute("create virtual table sysfiles using

filesource("+sysdirs+")")

File "apsw.c", line 3518, in resetcursor File "apsw.c", line 2460, in

VirtualTable?.xCreate File "D:\Python\00 Projects\IRR\apswTest.py", line
291, in Create

    columns,data=getfiledata([eval(a) for a in args]) # eval strips off

layer of quotes

File "D:\Python\00 Projects\IRR\apswTest.py", line 278, in getfiledata

    for f in os.listdir(directory):

WindowsError?: [Errno 123] The filename, directory name, or volume label
syntax is incorrect: 'C:\x08in\Python24\Lib\idlelib/.'

with line: sysdirs=",".join(["'%s'" % (x.replace('','/'),) for x in
sys.path[1:] if len(x) and os.path.isdir(x)]) The execution terminates
normally without error.

Original issue: http://code.google.com/p/apsw/issues/detail?id=1

Changing the table invalidates result set

From [email protected] on July 18, 2008 09:27:29

(from the sqlite mailinglist):

From: Nikolaus Rath [email protected]
Subject: Re: Does changing the db change result sets?
Newsgroups: gmane.comp.db.sqlite.general
To: [email protected]
Date: the 18th of July 2008 at 12:09
Reply-To: General Discussion of SQLite Database
[email protected]

"Igor Tandetnik" [email protected] writes:

"Nikolaus Rath" [email protected] wrote in
message news:[email protected]

What happens if

  1. I prepare and execute a select statement
  2. I retrieve a couple of rows
  3. I execute a new query that would change the result of the
    query in 1

On the same connection, I assume.

Yes.

  1. I continue to retrieve the results of 1)

Will I get the results as if step 3 hasn't happened, is the result
undefined, or will my result set somehow be updated?

You may or may not see the changes introduced by step 3. E.g. if
step 3 updates a record that you've already visited, you naturally
won't visit it again and thus won't see the changes. But if it
updates a record you haven't visited yet, when you get to it you
will see the new data.

This doesn't seem right when I test it. "numbers" is a table with one
column containing the numbers 1 to 10. When using the python
interface:

import apsw
conn=apsw.Connection("data")
cursor = conn.cursor()
res = cursor.execute("select * from numbers")
res.next()
(1,)
res.next()
(2,)
res.next()
(3,)
cursor.execute("delete from numbers where no=5")
res.next()
Traceback (most recent call last):
File "", line 1, in
StopIteration

So as soon as I make a change in the database, I do not get any more
results. If I understood you correctly, I should have gotten (4,)
instead.

Is this a bug?

rom: Dennis Cote [email protected]
Subject: Re: Does changing the db change result sets?
Newsgroups: gmane.comp.db.sqlite.general
To: General Discussion of SQLite Database
[email protected]
Date: the 18th of July 2008 at 16:00
Reply-To: General Discussion of SQLite Database
[email protected]

Nikolaus Rath wrote:

So as soon as I make a change in the database, I do not get any more
results. If I understood you correctly, I should have gotten (4,)
instead.

Is this a bug?

It is probably a bug (or a design decision) in the apsw wrapper. The
following program produces the expected output using the C API functions.

include <stdio.h>

include <stdlib.h>

include <sqlite3.h>

using namespace std;

int main(int argc, char argv[])
{
sqlite3
db;

 sqlite3_open("test.db3", &db);
 sqlite3_exec(db, "create table numbers (n integer)", 0, 0, 0);

 char sql[100];
 for (int i = 1; i <=10; ++i) {
     sprintf(sql, "insert into numbers values(&#37;d)", i);
     sqlite3_exec(db, sql, 0, 0, 0);
 }

 sqlite3_stmt* s;
 sqlite3_prepare(db, "select * from numbers", -1, &s, 0);

 sqlite3_step(s);
 printf("&#37;d\n", sqlite3_column_int(s, 0));
 sqlite3_step(s);
 printf("&#37;d\n", sqlite3_column_int(s, 0));
 sqlite3_step(s);
 printf("&#37;d\n", sqlite3_column_int(s, 0));

 sqlite3_exec(db, "delete from numbers where n = 5", 0, 0, 0);

 sqlite3_step(s);
 printf("&#37;d\n", sqlite3_column_int(s, 0));
 sqlite3_step(s);
 printf("&#37;d\n", sqlite3_column_int(s, 0));
 sqlite3_step(s);
 printf("&#37;d\n", sqlite3_column_int(s, 0));

 sqlite3_finalize(s);
 sqlite3_close(db);

 system("PAUSE");
 return EXIT_SUCCESS;

}

This produces:

1
2
3
4
6
7
Press any key to continue . . .

Original issue: http://code.google.com/p/apsw/issues/detail?id=19

CHM help files is EMPTY - no help files

From dawmster on February 24, 2009 05:41:11

What steps will reproduce the problem? 1. Just open the CHM file and click on a topic in contents tab What is the expected output? What do you see instead? 1. There should documentation apear about clicked topic
2. But in contents windows there is only : Incorrect Address What version of the product are you using? On what operating system? APSW v3.6.11- r1 documentation Please provide any additional information below. Windows XP Pro SP3, IE 7 installed.

Cheers

Original issue: http://code.google.com/p/apsw/issues/detail?id=42

APSW would crash when used with Stackless Python

From nirvana117 on March 22, 2008 03:38:18

What steps will reproduce the problem? 1. import apsw
2. apsw.Connection(":memory:") What version of the product are you using? On what operating system? apsw-3.3.13- r1 , Stackless_Python_2.5.2, on windows_xp_sp2 or 2003_sp2. Please provide any additional information below. I have tested pysqlite and apsw, but only apsw would crash stackless
python, not pysqlite. So I thought there may be some problem in apsw.

Original issue: http://code.google.com/p/apsw/issues/detail?id=3

Allow apsw to be used as an extension

From rogerbinns on September 25, 2008 15:52:02

An example use case is for the filesource from the example to work in the
SQLite command line interpreter.

There are a number of parts needed to get this to work:

  • Compile apsw using sqlite3ext.h although it is missing several functions
  • Work out which python file to load as there are no ways to pass
    arguments to the extension loading routine
  • Probably hook into all database connections using sqlite3_auto_extension

Original issue: http://code.google.com/p/apsw/issues/detail?id=28

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.