using SQLite;
db=SQLiteDB("E:\\sqliteData\\sqliteDB.db")
strSQL =" SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name='TEST'  "
output =query(strSQL);

in juno, julia produces the error:

"query " has no method matching query(::ASCIIString)  .

but in SQLITE EXPERT SQL , it works. why?

I'm not sure if it is by design or not but table inserts by dictionary is different to array data. e.g.

query(db, "insert into T values (:a,:b)", Dict(:a=>true, :b=>1) )
query(db, "insert into T values (?,?)", [true, 2] )

query(db, "select * from T")
2x2 SQLite.ResultSet
| Row | "col1" | "col2" |
|-----|--------|--------|
| 1   | true   | 1      |
| 2   | 1      | 2      |

In the the first instance the boolean values was serialized, while in the second case it was converted to an integer.
Is there a way to obtain the second behaviour using data in the form of a dictionary?

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2014-10-05 the testing status was Tests fail, but package loads.
• On 2014-10-08 the testing status changed to Package doesn't load.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

Special message from @IainNZ: This change may be due to breaking changes to Dict in JuliaLang/julia#8521, or the removal of deprecated syntax in JuliaLang/julia#8607.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("SQLite")' log

WARNING: deprecated syntax "(String=>String)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/BinDeps.jl:146.
Use "Dict{String,String}()" instead.

WARNING: deprecated syntax "(String=>String)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/BinDeps.jl:147.
Use "Dict{String,String}()" instead.

WARNING: deprecated syntax "(String=>String)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/BinDeps.jl:148.
Use "Dict{String,String}()" instead.

WARNING: deprecated syntax "(String=>String)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/BinDeps.jl:149.
Use "Dict{String,String}()" instead.

WARNING: deprecated syntax "(Symbol=>Any)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/dependencies.jl:224.
Use "Dict{Symbol,Any}()" instead.

WARNING: deprecated syntax "(Symbol=>Any)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/dependencies.jl:383.
Use "Dict{Symbol,Any}()" instead.

WARNING: deprecated syntax "{a=>b, ...}" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/dependencies.jl:387.
Use "Dict{Any,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "(Any=>Any)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/dependencies.jl:494.
Use "Dict{Any,Any}()" instead.

WARNING: deprecated syntax "(Any=>Any)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/dependencies.jl:555.
Use "Dict{Any,Any}()" instead.

WARNING: deprecated syntax "(Any=>Any)[]" at /home/idunning/pkgtest/.julia/v0.4/BinDeps/src/dependencies.jl:660.
Use "Dict{Any,Any}()" instead.

WARNING: deprecated syntax "{a=>b, ...}" at /home/idunning/pkgtest/.julia/v0.4/SQLite/deps/build.jl:9.
Use "Dict{Any,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "{a=>b, ...}" at /home/idunning/pkgtest/.julia/v0.4/SQLite/deps/build.jl:12.
Use "Dict{Any,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "{a=>b, ...}" at /home/idunning/pkgtest/.julia/v0.4/SQLite/deps/build.jl:13.
Use "Dict{Any,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "[a=>b, ...]" at /home/idunning/pkgtest/.julia/v0.4/SQLite/deps/build.jl:21.
Use "Dict(a=>b, ...)" instead.
INFO: Installing ArrayViews v0.4.6
INFO: Installing BinDeps v0.3.5
INFO: Installing DataArrays v0.2.2
INFO: Installing DataFrames v0.5.9
INFO: Installing GZip v0.2.13
INFO: Installing Reexport v0.0.1
INFO: Installing SHA v0.0.3
INFO: Installing SQLite v0.1.6
INFO: Installing SortingAlgorithms v0.0.2
INFO: Installing StatsBase v0.6.6
INFO: Installing URIParser v0.0.3
INFO: Building SQLite
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of SQLite
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using SQLite' log

WARNING: deprecated syntax "(T=>Int)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/scalarstats.jl:98.
Use "Dict{T,Int}()" instead.

WARNING: deprecated syntax "(T=>Int)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/scalarstats.jl:122.
Use "Dict{T,Int}()" instead.

WARNING: deprecated syntax "(T=>Float64)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/counts.jl:162.
Use "Dict{T,Float64}()" instead.

WARNING: deprecated syntax "(T=>Int)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/counts.jl:192.
Use "Dict{T,Int}()" instead.

WARNING: deprecated syntax "(T=>W)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/counts.jl:193.
Use "Dict{T,W}()" instead.

WARNING: deprecated syntax "(T=>Int)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/misc.jl:66.
Use "Dict{T,Int}()" instead.

WARNING: deprecated syntax "(T=>Int)[]" at /home/idunning/pkgtest/.julia/v0.4/StatsBase/src/misc.jl:77.
Use "Dict{T,Int}()" instead.

WARNING: deprecated syntax "[a=>b, ...]" at /home/idunning/pkgtest/.julia/v0.4/DataFrames/src/RDA.jl:11.
Use "Dict(a=>b, ...)" instead.

WARNING: deprecated syntax "[a=>b, ...]" at /home/idunning/pkgtest/.julia/v0.4/SQLite/src/SQLite_consts.jl:109.
Use "Dict(a=>b, ...)" instead.

WARNING: deprecated syntax "[a=>b, ...]" at /home/idunning/pkgtest/.julia/v0.4/SQLite/src/SQLite_api.jl:87.
Use "Dict(a=>b, ...)" instead.
Julia Version 0.4.0-dev+998
Commit e24fac0 (2014-10-07 22:02 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

ERROR: `Dict{Symbol,Union(AbstractArray{Real,1},Real)}` has no method matching Dict{Symbol,Union(AbstractArray{Real,1},Real)}(::(), ::())
 in DataFrame at /home/idunning/pkgtest/.julia/v0.4/DataFrames/src/dataframe/dataframe.jl:85
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at ./loading.jl:152
 in _require at ./loading.jl:67
 in require at ./loading.jl:52
 in require_3B_3964 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:293
 in _start at ./client.jl:362
 in _start_3B_3789 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.4/SQLite/src/SQLite.jl, in expression starting on line 36
while loading /home/idunning/pkgtest/.julia/v0.4/SQLite/testusing.jl, in expression starting on line 2

>>> test log
no tests to run
>>> end of log

Does this error fall under the known issue or something else? It seems as though it can find my local machine .dylib, but not sure where to go next.

Note, I can connect to SQLite and create a test database, so I am getting partial functionality.

julia> sqldf("Select * from test where origin = 'SAN'")
ERROR: ccall: could not find function sqlite3_close_v2 in library /usr/lib/libsqlite3.dylib
 in sqldf at /Users/randyzwitch/.julia/SQLite/src/SQLite.jl:374

julia> versioninfo()
Julia Version 0.3.0-prerelease+422
Commit 4b74801* (2013-12-12 23:13 UTC)
Platform Info:
System: Linux (x86_64-linux-gnu)
WORD_SIZE: 64
BLAS: libopenblas (NO_LAPACK NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY)
LAPACK: liblapack.so.3
LIBM: libopenlibm

idunning@IAIN-DESKTOP:~/.../SQLite/test$ julia test.jl
Warning: using FS.rename in module DataFrames conflicts with an existing identifier.
ERROR: [sqlite]: no such table: Employee
in error at error.jl:21
in internal_query at /home/idunning/.julia/SQLite/src/SQLite.jl:56
in query at /home/idunning/.julia/SQLite/src/SQLite.jl:72
in query at /home/idunning/.julia/SQLite/src/SQLite.jl:71
in include at boot.jl:238
while loading /home/idunning/.julia/SQLite/test/test.jl, in expression starting on line 5

On a Ubuntu system the Sqlite package does find the correct shared object for the library, /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 but the lib subdirectory of the package is still populated with various .dll and .so files that are not used. Is it possible through some of the facilities in the BinDeps package to download only the necessary versions of the library, which would be none in this case?

we can get a ResultSet data type

rsData  = query(db,strSQL)

but if we can transpose rsData as matrix , it may looks like:

A =["a" 1 2 ; "b" 2  4]
B=A'
,....
rsData =rsData'

we can loop it at col not row to get array or dict{}, the speed would increase .

windows7, 64bit, julia 0.3.3

julia> Pkg.add("SQLite")

INFO: Building SQLite
INFO: Updating WinRPM package list
INFO: Downloading http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_13.1//repodata/repomd.xml

INFO: Downloading http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_13.1//repodata/repomd.xml

===============================[ ERROR: SQLite ]================================
None of the selected providers can install dependency sqlite3_lib.

Use BinDeps.debug(package_name) to see available providers while  loadingC:\Users\Administrator\.julia\v0.3\
SQLite\deps\build.jl, in expression starting on line 21
================================[ BUILD ERRORS ]===========================

WARNING: SQLite had build errors.
 - packages with build errors remain installed in C:\Users\Administrator\.julia\v0.3
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

=============================================================================

julia> BinDeps.debug("SQLite")

INFO: Reading build script...
The package declares 1 dependencies.
 - Library "sqlite3_lib"
    - Providers:
      - RPM package libsqlite (can't provide)

julia> Pkg.build("SQLite")

INFO: Building WinRPM
INFO: Downloading http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUSE_13.1//repodata/repomd.xml
INFO: Downloading http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUSE_13.1//repodata/repomd.xml
INFO: Building SQLite

===============================[ ERROR: SQLite ]================================
None of the selected providers can install dependency sqlite3_lib.

Use BinDeps.debug(package_name) to see available providers

while loading C:\Users\Administrator\.julia\v0.3\SQLite\deps\build.jl, in expression starting on line 21

=============================================================================
================================[ BUILD ERRORS]========================
WARNING: SQLite had build errors.
 - packages with build errors remain installed in C:\Users\Administrator\.julia\v0.3
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

=============================================================================

A simple example of writing a small data set to a sqlite db and reading it back out would be appreciated. I would add it myself, but I can't figure out out after the DataStreams rework.

The non-streaming changes seem straightforward enough:

using SQLite, DataFrames, DataStreams
db = SQLite.DB("test.sqlite")
df = DataFrame(num = [1,2], str=["sdsd", "sdg"])
SQLite.drop!(db, "df", ifexists=true) 

but I can't figure out how to write even a simple DataFrame AbstractArray with arrays of column names and types to the db. It looks like create and append have been migrated to SQLite.Sink(schema::Data.Schema,db::DB,tablename;temp::Bool=false,ifnotexists::Bool=true), and I'm stuck on creating a Schema to pass along. The source of DataStreams.jl suggests that I should be able to do this with something like

sch = Data.Schema([string(n) for n in names(df)],[eltype(df[n]) for n in names(df)], nrow(df))

but it looks like i'm not particularly close.

I think a close function is missing. Generally, one would want to close a DB connection when he finishes using it.

It seems that the SQLite package have problems with creating tables where colnames are identical to hex-values.

create(db, "clearance", q[2:end, :], ["nuclide", "OF", "1a", "2a", "3a"])
Error: SQLite.SQLiteException("unrecognized token: \"1A\"")

Alternative names are possible, but is it also possible to fix it? The colnames are given by law in my case, so I really don't want to change them.

I have written an own Julia module for interfacing to Sqlite, which I don't plan to publish. It is hardly documented, and I wouldn't have the time to support it (I'm happy to send the code to anyone who is interested). My module is used in projects where the queries potentially return very large amounts of data, and from this experience I would like to propose extensions for this package, by which one can avoid allocating huge amounts of memory, provided that the results of queries can be further processed row by row:

1. Have something like query(db::SQLiteDB, sql::String, values=[], rowcb, handthrough). "rowcb" (row callback) is a user supplied Julia function, that is called for each row returned by the query, i.e. until sqlite3_step returns SQLITE_DONE. In my implementation the call is rowcb(handthrough, r::Any[]), where r is a vector containing a row, column by column, and handthrough is any data, that need to be handed to the callback function. When used with these arguments the query function does not return anything.

2. Same idea, but perhaps more elegantly: provide a

query_iterator(db::SQLiteDB, sql::String, values=[]) as well as start,next,done methods. Then one can code, for example:

for irow in query_iterator(db, "select * from mytable;")
do_something(irow)
end

Best regards,
Stephan ([email protected])

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.3

• On 2014-08-15 the testing status was Tests fail, but package loads.
• On 2014-08-16 the testing status changed to Package doesn't load.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("SQLite")' log
INFO: Cloning cache of SQLite from git://github.com/quinnj/SQLite.jl.git
INFO: Installing ArrayViews v0.4.6
INFO: Installing BinDeps v0.3.0
INFO: Installing DataArrays v0.2.0
INFO: Installing DataFrames v0.5.7
INFO: Installing GZip v0.2.13
INFO: Installing Reexport v0.0.1
INFO: Installing SHA v0.0.2
INFO: Installing SQLite v0.1.6
INFO: Installing SortingAlgorithms v0.0.1
INFO: Installing StatsBase v0.6.3
INFO: Installing URIParser v0.0.2
INFO: Building SQLite
===============================[ ERROR: SQLite ]================================

None of the selected providers can install dependency sqlite3_lib.
Use BinDeps.debug(package_name) to see available providers

while loading /home/idunning/pkgtest/.julia/v0.3/SQLite/deps/build.jl, in expression starting on line 21

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: SQLite had build errors.

 - packages with build errors remain installed in /home/idunning/pkgtest/.julia/v0.3
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

================================================================================
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of SQLite
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using SQLite' log

ERROR: could not open file /home/idunning/pkgtest/.julia/v0.3/SQLite/src/../deps/deps.jl
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_1699 at /home/idunning/julia03/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.3/SQLite/src/SQLite_consts.jl, in expression starting on line 1
while loading /home/idunning/pkgtest/.julia/v0.3/SQLite/src/SQLite.jl, in expression starting on line 10
while loading /home/idunning/pkgtest/.julia/v0.3/SQLite/testusing.jl, in expression starting on line 1

>>> test log
no tests to run
>>> end of log

I haven't had a chance to debug this fully, but I was seeing about 1/4 of my rows of BLOB fields resulting in prematurely truncated values. In Python they would turn out fine (these were zlib compressed JSON), but in Julia it would result in decompressed strings that were truncated.

I haven't had time to look into it more, (eg: it could also be the fault of the Zlib.jl library, but I doubt it) so this is partially as a reminder to myself, but I thought I'd put it here before I forgot about it.

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2014-08-15 the testing status was Tests fail, but package loads.
• On 2014-08-16 the testing status changed to Package doesn't load.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("SQLite")' log
INFO: Installing ArrayViews v0.4.6
INFO: Installing BinDeps v0.3.0
INFO: Installing DataArrays v0.2.0
INFO: Installing DataFrames v0.5.7
INFO: Installing GZip v0.2.13
INFO: Installing Reexport v0.0.1
INFO: Installing SHA v0.0.2
INFO: Installing SQLite v0.1.6
INFO: Installing SortingAlgorithms v0.0.1
INFO: Installing StatsBase v0.6.3
INFO: Installing URIParser v0.0.2
INFO: Building SQLite
===============================[ ERROR: SQLite ]================================

None of the selected providers can install dependency sqlite3_lib.
Use BinDeps.debug(package_name) to see available providers

while loading /home/idunning/pkgtest/.julia/v0.4/SQLite/deps/build.jl, in expression starting on line 21

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: SQLite had build errors.

 - packages with build errors remain installed in /home/idunning/pkgtest/.julia/v0.4
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

================================================================================
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of SQLite
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using SQLite' log
ERROR: could not open file /home/idunning/pkgtest/.julia/v0.4/SQLite/src/../deps/deps.jl
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_3500 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.4/SQLite/src/SQLite_consts.jl, in expression starting on line 1
while loading /home/idunning/pkgtest/.julia/v0.4/SQLite/src/SQLite.jl, in expression starting on line 10
while loading /home/idunning/pkgtest/.julia/v0.4/SQLite/testusing.jl, in expression starting on line 1


>>> test log
no tests to run
>>> end of log

just added a pull request from a branch dg/testing forked from @quinnj branch jq/updates.
it is broken to many small commits. these include a bugfix; an addition of close() functions instead of finalizers - this is important since, for example, the DB cannot be closed before Stmt handlers are finalized and the finalization timing and order are not guaranteed (better allow users to civilly release resources) ; and more such tidbit changes. related pull request: #77

As you said in the other issue now's probably a good time to think about tagging a new release, especially since we've had several issues which we're solved by directing people to the master branch.

I've written a couple of IJulia Notebook tutorials which I'll submit a PR for soon, they're not great but they'll allow us to neaten up the README a bit.

Is there anything else you think needs to happen before a new release?

i want to know whether Data[i,j] with a ResultSet type makes a copy of Data, the way is same as Array?
thank you!

I have written an own Julia module for interfacing to Sqlite, which I don't plan to publish. It is hardly documented, and I wouldn't have the time to support it (I'm happy to send the code to anyone who is interested). My module is used in projects where the queries potentially return very large amounts of data, and from this experience I would like to propose extensions for this package, by which one can avoid allocating huge amounts of memory, provided that the results of queries can be further processed row by row:

1. Have something like query(db::SQLiteDB, sql::String, values=[], rowcb, handthrough). "rowcb" (row callback) is a user supplied Julia function, that is called for each row returned by the query, i.e. until sqlite3_step returns SQLITE_DONE. In my implementation the call is rowcb(thunk, r::Any[]), where r is a vector containing a row, column by column, and handthrough is any data, that need to be handed to the callback function. When used with these arguments the query function does not return anything.

2. Same idea, but perhaps more elegantly: provide a

query_iterator(db::SQLiteDB, sql::String, values=[]) as well as start,next,done methods. Then one can code, for example:

for irow in query_iterator(db, "select * from mytable;")
do_something(irow)
end

Best regards,
Stephan ([email protected])

Let's assume I have a DataFrame with a bunch of rows, is there any way I can drop that directly into a table (assuming the DF matches the table structure, etc etc)? If not, what do you think is that fastest way of doing this?

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.2) and the nightly build of the unstable version (0.3). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.3

• On 2014-07-12 the testing status was Tests fail, but package loads.
• On 2014-07-14 the testing status changed to Package doesn't load.

Tests fail, but package loads. means that PackageEvaluator found the tests for your package, executed them, and they didn't pass. However, trying to load your package with using worked.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

INFO: Installing ArrayViews v0.4.6
INFO: Installing BinDeps v0.2.14
INFO: Installing DataArrays v0.1.12
INFO: Installing DataFrames v0.5.6
INFO: Installing GZip v0.2.13
INFO: Installing Reexport v0.0.1
INFO: Installing SQLite v0.1.6
INFO: Installing SortingAlgorithms v0.0.1
INFO: Installing StatsBase v0.5.3
INFO: Installing URIParser v0.0.2
INFO: Building SQLite
INFO: Package database updated
Warning: could not import Sort.sortby into DataFrames
Warning: could not import Sort.sortby! into DataFrames
ERROR: repl_show not defined
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
while loading /home/idunning/pkgtest/.julia/v0.3/DataFrames/src/dataframe/reshape.jl, in expression starting on line 163
while loading /home/idunning/pkgtest/.julia/v0.3/DataFrames/src/DataFrames.jl, in expression starting on line 110
while loading /home/idunning/pkgtest/.julia/v0.3/SQLite/src/SQLite.jl, in expression starting on line 3
while loading /home/idunning/pkgtest/.julia/v0.3/SQLite/testusing.jl, in expression starting on line 1
INFO: Package database updated

Note this is possibly due to removal of deprecated functions in Julia 0.3-rc1: JuliaLang/julia#7609

ref JuliaLang/METADATA.jl#3881

Dropping Julia 0.3 support necessitates a new minor package version, otherwise it becomes needlessly difficult to ever make a branch and future bugfix release that does support 0.3.

On the first try to load SQLite with using SQLite julia (0.5, latest master) returns LoadError: too many parameters for type PointerString [...] while loading [...] SQLite/src/SQLite.jl in expression starting on line 129

On a second try it works.

I am using Julia version 0.3. I ran Pkg.add("SQLite") and tried using the package as documented, but got errors. The interface instead seems to work as described here:

http://statcompute.wordpress.com/2014/02/08/julia-and-sqlite/

Did the interface change and the package repository has not been updated yet?

Currently in METADATA SQLite requires DataFrames 0.2 but is this essential? There are ugly depracation warnings from using it.

Any chance I could convince you to use the standard capitalization for the SQLite library?

Posting here for easy tracking, I will try to take a look at these when I can since it sounds like low-hanging fruit.

using SQLite

produces

WARNING: Base.Uint8 is deprecated, use UInt8 instead.
likely near C:\Users\amellnik\.julia\v0.4\SQLite\src\show.jl:33
....
WARNING: Base.FloatingPoint is deprecated, use AbstractFloat instead.
likely near C:\Users\amellnik\.julia\v0.4\SQLite\src\SQLite.jl:139
WARNING: Base.String is deprecated, use AbstractString instead.
likely near C:\Users\amellnik\.julia\v0.4\SQLite\src\SQLite.jl:244

and

db = SQLiteDB("test.sqlite")

produces

WARNING: Base.Uint16 is deprecated, use UInt16 instead.
  likely near In[2]:1
....
WARNING: Base.Nothing is deprecated, use Void instead.
  likely near In[2]:1
....     
in __register#31__ at C:\Users\amellnik\.julia\v0.4\SQLite\src\UDF.jl:198
WARNING: Base.Nothing is deprecated, use Void instead.

I have an InexactError() when I try to access some colums of a table:

julia> query("SELECT r, n FROM outputs JOIN parameters ON parameters.id=outputs.sim_id;")
ERROR: InexactError()
 in push! at array.jl:457
 in push! at /home/tpo28/.julia/v0.3/DataArrays/src/datavector.jl:15
 in query at /home/tpo28/.julia/v0.3/SQLite/src/SQLite.jl:120
 in query at /home/tpo28/.julia/v0.3/SQLite/src/SQLite.jl:84

Here is the structure of the two tables:

julia> query("SELECT * FROM outputs LIMIT 5;")
5x4 DataFrame
|-------|----|--------|------|----|
| Row # | id | sim_id | time | n  |
| 1     | 1  | 53     | 79   | 10 |
| 2     | 2  | 53     | 80   | 10 |
| 3     | 3  | 53     | 81   | 10 |
| 4     | 4  | 53     | 82   | 10 |
| 5     | 5  | 53     | 83   | 10 |

and

> query("SELECT * FROM parameters LIMIT 5;")
5x4 DataFrame
|-------|----|----------|----|----|
| Row # | id | r        | K  | n0 |
| 1     | 1  | 0.3      | 10 | 1  |
| 2     | 2  | 0.4      | 10 | 1  |
| 3     | 3  | 0.5      | 10 | 1  |
| 4     | 4  | 0.530612 | 10 | 1  |
| 5     | 5  | 0.561224 | 10 | 1  |

My 2 cents on the issue is that SQLite.jl line 120 is pushing the latest read element in an array. When I try to limit the query so that all elements of n are Int64, it works; but whenever there starts being some Float64 in there, it fails.

I'm experience this error when trying to create a table:

julia> data=readtable("normalizedAndRounded.csv")
julia> SQLite.connect("main")
sqlite connection
-----------------
File: main
Connection Handle: Ptr{Void} @0x00007f813ad41160
Contains resultset? No
julia> createtable(data)
ERROR: [sqlite]: near "(": syntax error
 in error at error.jl:21
 in internal_query at /Volumes/Data/Home/.julia/SQLite/src/SQLite.jl:69
 in internal_query at /Volumes/Data/Home/.julia/SQLite/src/SQLite.jl:64
 in df2table at /Volumes/Data/Home/.julia/SQLite/src/SQLite.jl:185
 in createtable at /Volumes/Data/Home/.julia/SQLite/src/SQLite.jl:153
 in createtable at /Volumes/Data/Home/.julia/SQLite/src/SQLite.jl:148

The error occurs on both OS X (10.9) and Ubuntu, with both running v. 0.3.0 of Julia.

Might be a good idea to allow users to specify the IF EXISTS clause in drop and IF NOT EXISTS in create and createindex.

Hi,

First some system information ...

julia> versioninfo()
Julia Version 0.2.0
Commit 05c6461 (2013-11-16 23:44 UTC)
Platform Info:
System: Windows (i686-w64-mingw32) [Windows 7]
WORD_SIZE: 32
BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY)
LAPACK: libopenblas
LIBM: libopenlibm

Now the steps for reproducing the error:

julia> using SQLite

julia> SQLite.connect("C:\local\appdata\Ditto\Ditto.db")
Please submit a bug report with steps to reproduce this fault, and any error mes
sages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0xcdf765 -- ??? at ???: offset cdf765
??? at ???: offset cdf765
jl_apply_generic at ???: offset 6978f8a6
jl_dump_function at ???: offset 697ba7d2
jl_dump_function at ???: offset 697b9d89
jl_toplevel_eval at ???: offset 697c773b
jl_enter_handler at ???: offset 6979501e
jl_apply_generic at ???: offset 6978f8a6
jl_apply_generic at ???: offset 6978f8a6
jl_prepare_ast at ???: offset 69794777
uv_msafd_poll at ???: offset 698a6106
RtlFreeHeap at ???: offset 76e52ce8
free at ???: offset 758a98da
RtlFreeHeap at ???: offset 76e52ce8
free at ???: offset 758a98cd
jl_module_parent at ???: offset 6979bfc8
jl_show_any at ???: offset 69795ec4
jl_apply_generic at ???: offset 6978f8a6
??? at ???: offset 401889
julia_trampoline at ???: offset 697bf19a
tgamma at ???: offset 435b85
??? at ???: offset 4013ee
BaseThreadInitThunk at ???: offset 76beed5c
RtlInitializeExceptionChain at ???: offset 76e637eb
RtlInitializeExceptionChain at ???: offset 76e637be

C:\local\appinstall\julia\bin>

I tried using the sqlite3.dll which comes with this package and the newest one from the SQLite download site.

I tested the this SQLite package on Ubuntu 12.04 and didn't have any problems.
If you need further information, please let me know.

After a while I'm using julia again and update all the packages, so also SQLite. In need to run an earlier project I try to retain the most code. But with the new version of SQLite I have some problems, that seems to be a step back for me.

It was a very comprehensible way to use symbols in results of a query to address specific columns like sample[:sample_date] instead of sample[3].
Further I get errors from commands like

julia> sample[:]
ERROR: 'endof' has no method matching endof(::SQLite.ResultSet)

julia> sample[:, 2:end]
ERROR: 'ndims' has no method matching ndims(::SQLite.ResultSet)
 in trailingsize at abstractarray.jl:58

Are there other ways to retain a comprehensible way from earlier versions and a solution for : and end or at least a way to downgrade SQLite to an earlier version?

Something like sample[ find(sample.colnames .== "sample_date")[1] ] works, but is unnecessary bloated.

I had the problem below installing on windows with
Windows 7, julia 0.3.2

Pkg.add("SQLite") gives me
...
===============================[ ERROR: SQLite ]================================

None of the selected providers can install dependency sqlite3_lib.
Use BinDeps.debug(package_name) to see available providers

while loading c:\Apps\Julia\packages\v0.3\SQLite\deps\build.jl, in expression st
arting on line 21

I ran WinRPM.update()
I still get an error on trying to create a database

julia> db1 = SQLite.connect("pricepars.db")
ERROR: sqlite3_lib not defined
in connect at c:\Apps\Julia\packages\v0.3\SQLite\src\SQLite.jl:47

When I query:

df = query("select * from Px11",db)

df comes out as:

3x2 DataFrame:
x1 x2
[1,] [0,0,1,1] :A1
[2,] [0,1,0,1] :B1
[3,] [0.5,0.0,0.0,0.5] :p

when I'm after the following format...

julia> px11a = DataFrame(A1 = [1, 1,0,0], B1 = [1, 0,1,0],p = [0.5,0.0,0.0,0.5])
4x3 DataFrame:
A1 B1 p
[1,] 1 1 0.5
[2,] 1 0 0.0
[3,] 0 1 0.0
[4,] 0 0 0.5

The blog:
http://statcompute.wordpress.com/2014/02/08/julia-and-sqlite/
suggests the latter format should come out, not the former.

Is this a Mac OSX sqlite3 issue? (I'm using julia v0.2)
I tried to figure out how to translate the former dataframe into the latter but failed to produce the julia code to do it. sorry I'm new to this language..

I think given how long ago DataFrames punted on 0.2, we should just follow suit and stop tagging 0.2 compatability. Maybe when 0.3 comes out we can be cleverer about maintaining a branch that works with 0.3

http://iainnz.github.io/packages.julialang.org/?pkg=SQLite&ver=0.3

Right now createtable has some very odd behavior on OS X:

using DataFrames
using Sqlite
co = Sqlite.connect("tmp.sqlite3")
df = DataFrame(A = 1:5, B = 6:10)

   # Order of arguments seems to be reverse of documented order
Sqlite.createtable(co, df)
=> ERROR: no method createtable(SqliteDB,DataFrame)

   # Best order I can guess doesn't work in a weird, stateful way
Sqlite.createtable(df, co)
=> ERROR: sqldf not defined
 in df2table at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:154
 in createtable at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:138

Sqlite.createtable(df, co)
=> ERROR: [sqlite]: Safety level may not be changed inside a transaction
 in internal_query at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:53
 in createtable at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:135

   # States are cyclical
Sqlite.createtable(df, co)
=> ERROR: sqldf not defined
 in df2table at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:154
 in createtable at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:138

Sqlite.createtable(df, co)
=> ERROR: [sqlite]: Safety level may not be changed inside a transaction
 in internal_query at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:53
 in createtable at /Users/johnmyleswhite/.julia/Sqlite/src/Sqlite.jl:135

It's bedtime here in Holland so I'll try to get to this tomorrow morning.

Thanks for working on this!

I've been working on implementing user-defined-functions today and I've noticed that in query() you only reset the prepared statements rather than finalising them.

The problem with this is that leaving statements active prevents certain changes to the database, one of these changes being the creation and modification of user-functions.

Is there a particular reason that the statements are only reset or will finalising them in query not be an issue?

I try a simple

connect("test.db")

on a 32bit aptosid (mainly Debian sid (Jessie)) and this results in a Seg Fault.

SQLite.sqlite3_lib is libsqlite3

Creating a database with sqlite from shell works.

@quinnj -- I'm doing it -- I'm opening this issue :)

I think we can figure out ˙how to do this properly and I opened up the issue to give us a platform for bouncing around ideas.

I'm hoping to dig into SQLite.jl and Tables.jl sooner rather than later, so hopefully I'll be able to contribute to the discussion before too long.

Hi Jacob

Pkg.status show the versions (does it not?).
I only executed the three lines below:

Pkg.add("SQLite")
Pkg.status()
using SQLite # after this julia crashes!

As a side note: the log below shows that the deps\build.jl file still has the old Dict notation in a few places.

Executing .juliarc.jl... done
INFO: Installing CSV v0.0.2
INFO: Installing DataStreams v0.0.2
INFO: Installing NullableArrays v0.0.2
INFO: Installing SQLite v0.2.2
INFO: Building WinRPM
INFO: Downloading https://cache.e.ip.saba.us/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUS
E_13.2/repodata/repomd.xml
INFO: Downloading https://cache.e.ip.saba.us/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUS
E_13.2/repodata/repomd.xml
INFO: Building SQLite

WARNING: deprecated syntax "{a=>b, ...}" at C:\Users\workstation.julia\SQLite\deps\build.jl:17.
Use "Dict{Any,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "{a=>b, ...}" at C:\Users\workstation.julia\SQLite\deps\build.jl:18.
Use "Dict{Any,Any}(a=>b, ...)" instead.

WARNING: deprecated syntax "[a=>b, ...]" at C:\Users\workstation.julia\SQLite\deps\build.jl:21.
Use "Dict(a=>b, ...)" instead.
INFO: Updating WinRPM package list
INFO: Downloading https://cache.e.ip.saba.us/http://download.opensuse.org/repositories/windows:/mingw:/win32/openSUS
E_13.2/repodata/repomd.xml
INFO: Downloading https://cache.e.ip.saba.us/http://download.opensuse.org/repositories/windows:/mingw:/win64/openSUS
E_13.2/repodata/repomd.xml
INFO: Package database updated
27 required packages:

• Coverage 0.2.3
• DataFrames 0.6.10
• Dates 0.4.4
• FastAnonymous 0.3.2
• Gadfly 0.3.18
• Graphics 0.1.3
• HDF5 0.5.6
• Iterators 0.1.9
• JLD 0.5.5
• JSON 0.5.0
• Jewel 1.0.7
• Libz 0.0.2
• Lint 0.1.68
• Loess 0.0.5
• Mocha 0.1.0
• NumericExtensions 0.6.2
• ODBC 0.3.10
• ProfileView 0.1.1
• ProgressMeter 0.2.1
• PyCall 1.1.2
• RDatasets 0.1.2
• SQLite 0.2.2
• SortingAlgorithms 0.0.6
• StatsBase 0.7.4
• TypeCheck 0.0.3
• WinRPM 0.1.13
• ZipFile 0.2.5
  56 additional packages:
• ArrayViews 0.6.4
• BinDeps 0.3.18
• Blosc 0.1.4
• BufferedStreams 0.0.2
• CSV 0.0.2
• Cairo 0.2.31
• Calculus 0.1.13
• Codecs 0.1.5
• ColorTypes 0.1.7
• Colors 0.5.4
• Compat 0.7.7
• Compose 0.3.18
• Conda 0.1.7
• Contour 0.0.8
• DataArrays 0.2.19
• DataStreams 0.0.2
• DataStructures 0.3.13
• Debug 0.1.6
• Distances 0.2.1
• Distributions 0.8.7
• Docile 0.5.19
• DualNumbers 0.1.5
• FactCheck 0.4.1
• FixedPointNumbers 0.0.12
• GZip 0.2.18
• Grid 0.4.0
• Gtk 0.9.2
• GtkUtilities 0.0.6
• Hexagons 0.0.4
• HttpCommon 0.2.4
• HttpParser 0.1.1
• ImmutableArrays 0.0.11
• JuliaParser 0.6.3
• KernelDensity 0.1.2
• LNR 0.0.1
• Lazy 0.10.1
• LibExpat 0.1.0
• Logging 0.1.1
• MacroTools 0.2.0
• MbedTLS 0.1.4
• MySQL 0.0.0- master (unregistered)
• NaNMath 0.1.1
• NullableArrays 0.0.2
• NumericFuns 0.2.4
• Optim 0.4.4
• PDMats 0.3.6
• Reexport 0.0.3
• Requests 0.3.2
• Requires 0.2.1
• SHA 0.1.2
• Showoff 0.0.6
• StatsFuns 0.2.0
• URIParser 0.1.1
• WoodburyMatrices 0.1.2
• Zlib 0.1.12
• lib 0.0.0- non-repo (unregistered)

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety
). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x64f749d2 -- jl_deserialize_lambdas_from_mod at C:\Program Files\Juno\reso
urces\app\julia\bin\libjulia.dll (unknown line)
jl_deserialize_lambdas_from_mod at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_deserialize_lambdas_from_mod at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_restore_incremental at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
require_from_serialized at loading.jl:84
jlcall__require_from_serialized_1832 at (unknown line)
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
require_from_serialized at loading.jl:109
require at loading.jl:219
jlcall_require_1762 at (unknown line)
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_init_serializer at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_module_expr at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_module_expr at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_parse_eval_all at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_load at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
include at boot.jl:261
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
include_from_node1 at loading.jl:304
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_interpret_toplevel_expr at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_interpret_toplevel_thunk_with at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_f_tuple at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
require at loading.jl:243
jlcall_require_1762 at (unknown line)
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_init_serializer at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_parse_eval_all at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_load at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
include at boot.jl:261
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
include_from_node1 at loading.jl:304
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_interpret_toplevel_expr at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_interpret_toplevel_thunk_with at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_f_tuple at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
require at loading.jl:243
jlcall_require_1762 at (unknown line)
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_init_serializer at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_eval_with_compiler_p at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_parse_eval_all at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
jl_load_ at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
include at boot.jl:261
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
include_from_node1 at loading.jl:304
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
process_options at client.jl:308
_start at client.jl:411
jlcall__start_510 at (unknown line)
jl_apply_generic at C:\Program Files\Juno\resources\app\julia\bin\libjulia.dll (unknown line)
unknown function (ip: 00000000004018D0)
unknown function (ip: 000000000040282B)
unknown function (ip: 000000000040140C)
unknown function (ip: 000000000040153B)
BaseThreadInitThunk at C:\Windows\system32\kernel32.dll (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)

i am happy for using SQLite.jl. it is great.

now i have a question about my project. i use SQLite.jl to fetch my stocks SQLite data ,normally,time of fetching a stock data(about 110,000 records) from SQLite is about 1.7seconds, but with the number of stocks increasing to above 50, julia occupation memory(my machine memory is 32G ) is growing rapidly , and the speed of fetching data is more and more slower, the time of fetching a stock increasing to above 4 seconds. and the time is creasing with the number of stocks.

i feel that in Julia workspace puting these datas together into a big data variable costs lots of time and memory, and i try to return Array{kbarData,1} not to return Dict{Dates.Date,Dict{String,Array{kBarData,1}},but the problem of fetching data looks alike.

on the other hand,it is very good to write lots of CSV.file to SQLite, memory occupation is normal. and i write 3000 stock csv.files to SQLite, the time of writing is almost equal.

how to solve this problem of fetching data? thank you very much!

function getDictDictMinuteDataFromSQLiteDB(code::ASCIIString,startDateTime::Dates.DateTime,closeDateTime::Dates.DateTime)
       println("\n  getDictDictMinuteDataFromSQLiteDB=>......")
       stime =time();
       cdata  = Dict{Dates.Date,Dict{String,Array{kBarData,1}}}();
       if length(code)<8
              #MinuteFutureSQLiteDBPath   = "E:\\sqliteData\\minuteFutureDB.db" 
              db= SQLite.SQLiteDB(MinuteFutureSQLiteDBPath)
              dbPath =MinuteFutureSQLiteDBPath
       else
              db= SQLite.SQLiteDB(MinuteStockSQLiteDBPath)
              dbPath =MinuteStockSQLiteDBPath
       end
        startDay =  Dates.year(startDateTime)*10000+Dates.month(startDateTime)*100+Dates.day(startDateTime)  #
        closeDay = Dates.year(closeDateTime)*10000+Dates.month(closeDateTime)*100+Dates.day(closeDateTime) #
        if isExistTableInDataBase(dbPath,code)
              strSQL = string("SELECT * FROM  ",code, "  WHERE TradeDay>  ",string(startDay), " AND TradeDay< ",string(closeDay))
              rsData  = query(db,strSQL)
              (row,col)=size(rsData)
              if  row>=1 && col>=10  # 
                    cdata    =  insertResultSetDataToDictDictData(rsData,cdata,startDateTime,closeDateTime); # convert resultdata to Dict{Dates.Date,Dict{String,Array{kBarData,1}}}
             else
                    println("table $(code) is empty！row:$(row)  col:$(col) !")
             end
        else
              println("table isnot existed：$(code) ！")
        end
        println("$(code) file，cost： $(time()-stime) seconds！")
        close(db)
        return cdata;
end

function insertResultSetDataToDictDictData(rsData::ResultSet,cData::Dict{Dates.Date,Dict{String,Array{kBarData,1}}},startDateTime::Dates.DateTime,closeDateTime::Dates.DateTime)
    (rowN,colN) =size(rsData);
    println("table row =>$rowN");
    for i=1:rowN
         barTime =getDateTimeByDateAndTimeValue(rsData[i,2],rsData[i,3],0) # 0:
         barDate =Dates.Date(barTime);
          if barTime<startDateTime ||barTime>closeDateTime
                 continue;
          end
         Close = convert(Float64,rsData[i,7])
         Code =  rsData[i,1];
         Open =  convert(Float64,rsData[i,4])
         High  = convert(Float64,rsData[i,5]);
         Low  = convert(Float64,rsData[i,6]);
         if i==1
            PreClose =Close
        else
            PreClose = convert(Float64,rsData[i-1,7])
        end
        OpenInterest =rsData[i,10]*1.0;
        Volume = convert(Float64,rsData[i,8]) # 
        Amount = convert(Float64,rsData[i,9]) #
        newbar = kBarData(Code,barTime,Close,Open,High,Low,PreClose,OpenInterest,Volume,Amount)
        if haskey(cData,barDate)
              if haskey(cData[barDate],Code)
                    push!(cData[barDate][Code],newbar)
              else
                    push!(cData[barDate],Code,[newbar])
              end
        else
              tempDict =Dict{String,Array{kBarData,1}}()
              push!(tempDict,Code,[newbar]);
              push!(cData,barDate,tempDict);
        end
    end
    return cData;
end

Whenever a select statement is executed, the program crashes when the finalizer for SQLiteStmt is run. This appears to be resolved by removing the SQLiteStmt finalizer and instead letting the SQLiteDB finalizer do the work so I'll make the changes and submit a pull request but this might warrant more investigation.

this query is failing because the "ê" unicode character

julia> s
"insert into items(channel, title, link, date) values(1, 'Você sempre pode ir embora', 'http://www.youtube.com/watch?v=G0I9E1vcb5o&feature=youtube_gdata', '2014-01-31 13:30:01')"

julia> query(s)
ERROR: [sqlite]: near "'2014-01-31 13:30:01'": syntax error
in error at error.jl:21
in internal_query at /home/paulo/.julia/SQLite/src/SQLite.jl:69
in query at /home/paulo/.julia/SQLite/src/SQLite.jl:85
in query at /home/paulo/.julia/SQLite/src/SQLite.jl:84

suggestions on settings/defines when compiling SQLITE for multidisciplinary use

Compiling in extensions.c math and string funcs would be helpful:
< www.sqlite.org/contrib/download/extension-functions.c?get=25 >

SQLITE_DEFAULT_FOREIGN_KEYS=1        enforce explicit foreign keys
SQLITE_SECURE_DELETE=1               overwrite deleted data with zeros
SQLITE_ALLOW_COVERING_INDEX_SCAN=1   improve performance
SQLITE_ENABLE_API_ARMOR=1            trap some bad behavior
SQLITE_ENABLE_COLUMN_METADATA=1      db, table, col src of SELECTed col
SQLITE_ENABLE_FTS3=1                 full text search engine (FTS3 & FTS4)
SQLITE_ENABLE_FTS3_PARENTHESIS=1     AND, NOT ops for full text search

SQLITE_ENABLE_RTREE=1?               enable 2D boxed range databases
                                     very powerful when needed, adds size

SQLITE_THREADSAFE=?
    I have no experience with this, nonetheless 0 or 1 (not 2)
    Given julia's mindmeld with mulipilicity (muti-thread,core,server..)
    it is not clear that =0 (single threaded) allows clean multiagent use.
    It is clear that =0 runs faster as there is no locking overhead and
    no need for interfacing code to check for and handle responses
    of SQLITE_BUSY and SQLITE_LOCKED.

no SQLITE_DISABLE_ options and no SQLITE_OMIT_ options

Hello,

Using Julia-0.3.0-prerelease, latest build as of today.
Below is what I get after Pkg.add("SQLite"). All dependencies of SQLite are installed.

julia> Pkg.build("SQLite")
INFO: Building LibCURL
INFO: Building SQLite
===============================[ ERROR: SQLite ]================================

None of the selected providers can install dependency sqlite3_lib
while loading C:\Users\user.julia\v0.3\SQLite\deps\build.jl, in expression starting on line 21

================================[ BUILD ERRORS ]================================

WARNING: SQLite had build errors.

• packages with build errors remain installed in C:\Users\user.julia\v0.3
• build a package and all its dependencies with Pkg.build(pkg)
• build a single package by running its deps/build.jl script

I have a pre-existing table generated by Python. The time column is a REAL and the ys column is a BLOB. I'm assuming NA is not the correct response...

julia> df = query("select time from scans where id=1")
1x1 DataFrame
| Row | time       |
|-----|------------|
| 1   | 0.00101332 |

julia> df = query("select ys from scans where id=1")
1x1 DataFrame
| Row | ys |
|-----|----|
| 1   | NA |

It occured to me that by serializing values by default in bind and sqlreturn we are effectively removing BLOBs as a storage medium. This means that if the user opens a database which uses BLOBs and tries to retrieve the BLOB, julia will throw an error since it won't be able to deserialize that value. It also means that users won't have a way to store values as BLOBs without serializing the value.

My approach would be to to define an extra method sqlreturn(context, val::Vector{Uint8}) which would call sqlite3_result_blob and have sqlreturn(context, val) call that method. Then wrap the elseif valuetype == SQLITE_BLOB branch in a try-statement. Similarly for bind and query.

I'll whip up a pull request to show what I mean when I've got time but I don't know whether this is the best way to go about things.

Creating a DB if the filename doesn't match anything seems to me to be a bit surprising, as it moves the error if you didn't intend this to a later line when you try to read from it.