julianhyde / sqlline

Shell for issuing SQL to relational databases via JDBC

Command-line shell for issuing SQL to relational databases via JDBC.

History

A fork of Marc Prud'hommeaux's sqlline project, also incorporating changes made by the LucidDB project, now modernized, mavenized and forkable in github. See also release history.

License and distribution

SQLLine is distributed under the 3-clause BSD License, meaning that you are free to redistribute, modify, or sell it with almost no restrictions.

It is distributed via the Maven Central Repository.

Demos

demos

Quick start

If you have Coursier installed, you can quickly connect to a demo Hypersonic database with:

$ coursier launch sqlline:sqlline:1.12.0 org.hsqldb:hsqldb:2.5.0 net.hydromatic:foodmart-data-hsqldb:0.4 -M sqlline.SqlLine -- -u jdbc:hsqldb:res:foodmart -n FOODMART -p FOODMART -d org.hsqldb.jdbcDriver
0: jdbc:hsqldb:res:foodmart> select avg("shelf_height" * "shelf_width" * "shelf_depth") as "avg_volume" from "product";
+-------------------------+
|       avg_volume        |
+-------------------------+
| 2147.3845245442353      |
+-------------------------+
1 row selected (0.01 seconds)
0: jdbc:hsqldb:res:foodmart> !quit

Getting started

Copy the sqlline script (or sqlline.bat for Windows), sqlline-VERSION-jar-with-dependencies.jar and a JDBC driver jar into the same directory. (Or just put sqlline on your PATH.)

$ sqlline -d com.mysql.jdbc.Driver
sqlline> !connect jdbc:mysql://localhost:3306/scott user password
sqlline> !tables
+------------+--------------+-------------+---------------+----------+
| TABLE_CAT  | TABLE_SCHEM  | TABLE_NAME  |  TABLE_TYPE   | REMARKS  |
+------------+--------------+-------------+---------------+----------+
| null       | SCOTT        | BONUS       | TABLE         | null     |
| null       | SCOTT        | DEPT        | TABLE         | null     |
| null       | SCOTT        | EMP         | TABLE         | null     |
| null       | SCOTT        | SALGRADE    | TABLE         | null     |
| null       | metadata     | COLUMNS     | SYSTEM_TABLE  | null     |
| null       | metadata     | TABLES      | SYSTEM_TABLE  | null     |
+------------+--------------+-------------+---------------+----------+
sqlline> SELECT 1 + 2 AS c;
+---+
| C |
+---+
| 3 |
+---+
sqlline> !quit

To get help:

$ sqlline --help

If you prefer, you can invoke Java directly, without using the sqlline script:

$ java -jar sqlline-VERSION-jar-with-dependencies.jar --help

Connecting using URLs

A URL (or connect string) is a string that specifies the location of your database, and perhaps credentials and other parameters specific to your database's JDBC driver. It always starts with 'jdbc:', usually followed by the machine name of the database and additional parameters.

For example, the following bash command connects to Apache Drill, assuming that Drill is installed in /opt/apache-drill-1.15.0:

$ /opt/apache-drill-1.15.0/bin/sqlline -u "jdbc:drill:drillbit=example.com;auth=kerberos"

Because ';' is a command separator in bash, the URL is included in double-quotes ('"'). You will need to quote the URL and other arguments if they contain characters that are special in your shell; different shells have different special characters, but space (' '), dollar ('$'), single-quote ('''), bang ('!') and percent ('%') are some common examples.

Read the manual.

Maven Usage

Use the following definition to use sqlline in your maven project:

<dependency>
  <groupId>sqlline</groupId>
  <artifactId>sqlline</artifactId>
  <version>1.12.0</version>
</dependency>

Building

To build or run SQLLine, you need Java 8 or higher. (When running on Java 15 and higher, you will need to set scriptEngine before you use promptscript.)

Check out and build:

git clone git://github.com/julianhyde/sqlline.git
cd sqlline
./mvnw package

Committers

• Arina Ielchiieva
• Jack Hahn
• Joe Posner
• John Pham
• John V. Sichi
• Julian Hyde (PMC)
• Marc Prud'hommeaux
• Sergey Nuyanzin (PMC Chair)
• Stephan Zuercher
• Steve Herskovitz
• Sunny Choi

More information

• License: Modified BSD License
• Home page
• API
• Source code
• Developers list: sqlline-dev at googlegroups.com (archive, subscribe)
• Twitter: @SQLLineShell
• Issues
• Release notes and history
• HOWTO

sqlline's Issues

Script fails if first line is a comment

To repro install Apache Phoenix 4.7 and run the following command:

./sqlline.py localhost ../examples/STOCK_SYMBOL.sql 

You'll see the following error message:

1/5          -- test comment
Aborting command set because "force" is false and command failed: "-- test comment"

Here's what the script looks like:

-- test comment
CREATE TABLE IF NOT EXISTS STOCK_SYMBOL (SYMBOL VARCHAR NOT NULL PRIMARY KEY, COMPANY VARCHAR);
UPSERT INTO STOCK_SYMBOL VALUES ('CRM','SalesForce.com');
SELECT * FROM STOCK_SYMBOL;

Error: Could not find or load main class sqlline.SqlLine

Hi, this is probably a mistake on my side but I couldn't understand how to install/run properly sqlline I fear.

I downloaded the source zip file for the 1.3.0 release. I unzipped it in a folder and from a terminal and within the sqlline folder I run:

mvn package

Some of the tests failed, but I can't understand why:

Running sqlline.SqlLineTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.245 sec - in sqlline.SqlLineTest
Running sqlline.SqlLineArgsTest
java.lang.IllegalArgumentException: No current connection
	at sqlline.SqlLine.getConnection(SqlLine.java:388)
	at sqlline.Commands.nativesql(Commands.java:350)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
	at sqlline.SqlLine.dispatch(SqlLine.java:791)
	at sqlline.SqlLine.runCommands(SqlLine.java:1706)
	at sqlline.SqlLineArgsTest.testNPE(SqlLineArgsTest.java:557)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:162)
	at mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:71)
	at mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:37)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
[ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ][ Hit "enter" for more ("q" to exit) ]java.io.FileNotFoundException: /tmp/sqllinenegative7589320866135718635temp (No such file or directory)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at java.io.FileInputStream.<init>(FileInputStream.java:93)
	at java.io.FileReader.<init>(FileReader.java:58)
	at sqlline.Commands.run(Commands.java:1267)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
	at sqlline.SqlLine.dispatch(SqlLine.java:791)
	at sqlline.SqlLine.initArgs(SqlLine.java:595)
	at sqlline.SqlLine.begin(SqlLine.java:643)
	at sqlline.SqlLineArgsTest.run(SqlLineArgsTest.java:78)
	at sqlline.SqlLineArgsTest.runScript(SqlLineArgsTest.java:102)
	at sqlline.SqlLineArgsTest.runScript(SqlLineArgsTest.java:85)
	at sqlline.SqlLineArgsTest.testNegativeScriptFile(SqlLineArgsTest.java:493)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:162)
	at mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:71)
	at mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:37)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Tests run: 22, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 3.02 sec - in sqlline.SqlLineArgsTest

Results :

Tests run: 25, Failures: 0, Errors: 0, Skipped: 1

[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ sqlline ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.9.1:jar (attach-javadocs) @ sqlline ---
[INFO] 

Despite the failure I tried to go in the bin directory and run

./sqlline

which produced the error

Error: Could not find or load main class sqlline.SqlLine

I also tried to not build the package and run the sqlline executable that comes in the bin folder.

Could you please help me to figure out what I am doing wrong? Could it be because there is no jdbc driver in the folder? if that's the problem, I have several JDBC drivers already installed in some directory, is there any way I can make them "visible" to sqlline regardless of the directory where I am running sqlline?

Thanks for the help and apologies for the silly questions.
Luca

Manual is not published

The link in the README goes to a 404, http://www.hydromatic.net/sqlline/manual.html

Would probably be good to get this module moved into a gh-pages branch too.

Adding dateFormat, timeFormat, timestampFormat

It would be useful, if the user of SQLline had an ability to specify date, time, timestamp formats. It would allow to chose the necessary format for interpreting java.sql.Date, java.sql.Time and java.sql.Timestamp values.

The formats can be loaded as properties for SqlLineOpts (like numberFormat).

Binary releases?

The README mentions binary releases (sqlline-VERSION-jar-with-dependencies.jar), but I can't find them anywhere. Do you have any?

!run does not support ~ expansion

Try:

> !run ~/myfile.sql
java.io.FileNotFoundException: ~/myfile.sql (No such file or directory)

Granted, this is usually handled by a shell, but we can provide a little nicer experience here.

Phoenix VARBINARY type doesn't have a useful representation in sqlline.py

This was originally filed as PHOENIX-3934 but I was asked to file it here. So I'm copying and pasting.

When interacting with a commonly-used command line client like sqlline.py a VARBINARY type is not represented in a useful way in query results. The value seems to be a representation of the Java hashcode for the byte array, which changes with every query.

The following transcript will make this obvious. Note the value of MYBYTES is hard to make use of and moreover it changes with every query. I would like to see some stable representation of the byte array itself.

0: jdbc:phoenix:localhost:2181:/hbase> create table my_table (name VARCHAR PRIMARY KEY, mybytes VARBINARY);
No rows affected (2.33 seconds)
0: jdbc:phoenix:localhost:2181:/hbase> upsert into my_table(name, mybytes) values('hello', '12312');
1 row affected (0.004 seconds)
0: jdbc:phoenix:localhost:2181:/hbase> select * from my_table;
-------------------+
NAME MYBYTES
-------------------+
hello [B@650eab8
-------------------+
1 row selected (0.017 seconds)
0: jdbc:phoenix:localhost:2181:/hbase> select * from my_table;
--------------------+
NAME MYBYTES
--------------------+
hello [B@53f48368
--------------------+
1 row selected (0.015 seconds)
0: jdbc:phoenix:localhost:2181:/hbase> select * from my_table;
--------------------+
NAME MYBYTES
--------------------+
hello [B@71b3bc45
--------------------+
1 row selected (0.015 seconds)

---

Comment from James Taylor:
Good idea. Sqlline is a separate open source project, currently hosted by [~julianhyde] over here: https://github.com/julianhyde/sqlline/issues. Please refile your JIRA there.

add ability to change/hide banner shown on startup

'!help set' should print documentation for all variables

'!help set' should print documentation for all variables. This should include showElapsedTime, maxHeight, timeout, trimScripts, which are currently not in the manual. Some of them have command-line parameters, e.g. '--showTime`. (See https://issues.apache.org/jira/browse/HIVE-6173?focusedCommentId=14266711&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14266711.)

Fixing the manual (doc/manual.xml) is a separate issue.

!close command not working

When command !close is given against an open connection the following output is displayed.
Ambiguous command: [close, closeall]

During build, manual fails to validate against docbook DTD

During build, manual fails docbook DTD with the following error:

[ERROR] Failed to execute goal com.agilejava.docbkx:docbkx-maven-plugin:2.0.17:generate-html (default) on project sqlline: Failed to transform manual.xml.: org.xml.sax.SAXParseExceptionpublicId: -//OASIS//DTD DocBook XML V4.1.2//EN; systemId: http://www.docbook.org/xml/4.2/docbookx.dtd; lineNumber: 1; columnNumber: 1; The markup declarations contained or pointed to by the document type declaration must be well-formed. -> [Help 1]

The solution is to update the DTD URL from http://www.docbook.org/xml/4.2/docbookx.dtd to https://docbook.org/xml/4.2/docbookx.dtd.

Unable to run scripts with embedded spaces in the path

Encountered this from both scripts specified as program options upon invocation and in interactive mode.

With outputformat=csv, bad generated CSV document when single quotes in input data

Hello.
When you use the outputformat=csv flag, if the source data (resulting drom a select SQL query) contains a single quote character, the resulting (the output generated) quote is not escaped. So, the resulting CSV file is incorrect. Please, ¿could you fix it?
Thanks in advance.

sqlline should give an error message and not throw a NPE

Use of -u instead of -n option to sqlline results in a NPE, sqlline should report an error and not throw a NPE.

{noformat}
[root@centos-01 ~]# cd /opt/mapr/drill/drill-1.10.0/bin
[root@centos-01 bin]# ./sqlline -u "jdbc:drill:schema=dfs.tmp;drillbit=10.10.100.201" -u mapr -p mapr
...
...
No known driver to handle "mapr". Searching for known drivers...
java.lang.NullPointerException
apache drill 1.10.0-SNAPSHOT
"what ever the mind of man can conceive and believe, drill can query"
0: mapr> !q
No known driver to handle "mapr". Searching for known drivers...
java.lang.NullPointerException
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.close(Commands.java:925)
at sqlline.Commands.quit(Commands.java:889)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.begin(SqlLine.java:621)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
No known driver to handle "mapr". Searching for known drivers...
java.lang.NullPointerException
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
at sqlline.Commands.close(Commands.java:925)
at sqlline.Commands.closeall(Commands.java:899)
at sqlline.SqlLine.begin(SqlLine.java:649)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)

{noformat}

Allow quoting of arguments to !connect

Allow quoting of the arguments to !connect. This would allow JDBC URLs, user names and passwords that contain spaces. E.g.

!connect "jdbc:calcite:schemaFactory=org.apache.calcite.adapter.druid.DruidSchemaFactory; schema.url=http://localhost:8082; schema.coordinatorUrl=http://localhost:8081" scott tiger

revisit how JDBC drivers are loaded

Since JDK 1.5, Java automatically loads drivers whose jar contains a META-INF/services/java.sql.Driver. All drivers have this file these days. This task is to (a) check that this mechanism works for MySQL's driver, (b) recommend whether SqlLine.KNOWN_DRIVERS should be removed.

Parametrize the CSV output generation

Hello.
It would be nice to have a set of command line switches to parametrize the CSV document generation.
Something like this:

• csv-fieldDelimiter
  Set the field delimiter. Defaults to , (comma).
• csv-rowDelimiter
  Set the string used to delimit record rows. Posibilities: auto, unix, mac, windows, unicode.
  Defaults to auto.
• csv-quote
  Optional character surrounding a field. Defaults to double quote.
• csv-escape
  Set the escape character. Defaults to double quote.

Thanks in advance

sqlline 1.1.7 breaks backwards comaptibility

I just tried to use a newer release of sqlline in lingual and saw that 1.1.7 breaks backwards compatibility:

lingual-client:compileJava
Download http://conjars.org/repo/sqlline/sqlline/1.1.7/sqlline-1.1.7.pom
Download http://conjars.org/repo/sqlline/sqlline/1.1.7/sqlline-1.1.7.jar
/Users/akelpe/code/lingual/lingual-client/src/main/java/cascading/lingual/shell/Shell.java:185: error: incompatible types
      result = SqlLine.mainWithInputRedirection( sqlLineArgs, inputStream );
                                               ^
  required: boolean
  found:    Status
/Users/akelpe/code/lingual/lingual-client/src/main/java/cascading/lingual/shell/Shell.java:194: error: incompatible types
        result = SqlLine.mainWithInputRedirection( sqlLineArgs, commandStream );
                                                 ^
  required: boolean
  found:    Status

While it is no big deal to fix lingual, that release should have been a 1.2 release or have deprecated backwards compatible method signatures.

Building Sqlline error

I run below command.

git clone git://github.com/julianhyde/sqlline.git
cd sqlline
mvn package

error messages:
[INFO] Scanning for projects...
Downloading: https://repo.maven.apache.org/maven2/net/hydromatic/parent/0.1/parent-0.1.pom
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project sqlline:sqlline:1.5.0-SNAPSHOT (/sources/sqlline/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM: Could not transfer artifact net.hydromatic:parent:pom:0.1 from/to central (https://repo.maven.apache.org/maven2): peer not authenticated and 'parent.relativePath' points at wrong local POM @ line 4, column 11 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

Please help.

Thanks,

sqlline should support vi mode for navigating between commands and within the commandline (similar to set -o vi in the shell)

sqlline seems to hang when using H2 db

Hi,

I've been trying to sqlline with the h2 db. (We got it working fine using the old sqlline-1_0_2.jar.)
My setup a folder with:
Driver: h2-1.4.191.jar
sqlline: sqlline-1.1.9-jar-with-dependencies.jar

I startup sqlline like so: java -Djava.ext.dirs=. -jar sqlline-1.1.9-jar-with-dependencies.jar sqline.Sqlline
Give it the connection string and it connects fine but has problems when running commands:
I can do '!dbinfo'
But commands like '!tables', or 'show schemas;'. Hangs the cli, I have to manually kill it.

I've attached a jstack -F output. But it doesn't tell me much. Perhaps someone else has an idea?

jstack.out.zip

Many thanks,
Patrick

Console output should adjust based on console size

Current behavior is to truncate column output when the console is not wide enough. The output table makes no indication that additional columns are present. Output should be produced in a more ergonomic manner. See another user's report of the same over on DRILL-3129.

Make SqlLine.begin public

It would be useful to run Sqlline redirecting output and stderr to other streams. You can create a Sqlline and call setOutputStream and setErrorStream but you can't currently call begin.

We must make the begin method public.

used to build on JDK 1.5, now requires JDK 1.6

sqlline used to build on JDK 1.5, but now requires JDK 1.6 due to call to java.sql.Statement.isClosed() in DispatchCallback.java.

It is important that sqlline can run on any JDBC driver in any environment, so I'd like to restore JDK 1.5 compatibility. We could use reflection to call isClosed, and degrade behavior a little on 1.5.

Pass JDBC properties outside of URL

I'm having trouble connecting to AWS Athena. The problem is that connections requires a property, s3_staging_dir, and their JDBC driver doesn't seem to parse the property from the JDBC URL.

This doesn't work: jdbc:awsathena://athena.us-west-2.amazonaws.com:443;s3_staging_dir=s3://aws-athena-query-results/

The docs suggest setting s3_staging_dir in the Properties object.

It would be nice if the properties file could pass extra properties to JDBC connection beyond the username and password. This would be useful for drivers that either can't parse options, or have more options than would be convenient to set in URL.

Documentation is missing

The link to the manual is broken and the POM doesn't contain code to build the docbook manual. Please fix at least one of them :-)

ability to turn off isolation level setting on connect?

I'm trying out the new Simbda JDBC driver for Big Query, and it seems to work fine with SQLLine except for during the initial connection I get an erro about setting the isolation level. I believe there's no such thing with Big Query. I was wondering if there's a way to have SQLLine not issue the set isolation level automatically? Other than this, it seems to be working, and I can ignore this...

I also have an email into the Simba support re: isolation levels, but thought I would ask this anyways.

Dan-Young:sql-line dan$ java -Djava.ext.dirs=$PWD sqlline.SqlLine --verbose=true
sqlline version 1.2.0
sqlline> !properties big-query-service.properties
Connecting to jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=[my-project];OAuthType=0;OAuthServiceAcctEmail=[service-account];OAuthPvtKeyPath=[key-path];Timeout=60;LogLevel=6;LogPath=/tmp
Connected to: Google BigQuery (version 2.0)
Driver: SimbaDriverforGoogleBigQuery (version 01.00.02.1003)
Autocommit status: true
Error: [Simba]JDBC Unsupported transaction isolation level: 4. (state=HY000,code=11975)
java.sql.SQLException: [Simba]JDBC Unsupported transaction isolation level: 4.
at com.simba.exceptions.ExceptionConverter.toSQLException(Unknown Source)
at com.simba.jdbc.common.SConnection.setTransactionIsolation(Unknown Source)
at sqlline.Commands.isolation(Commands.java:679)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:186)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:203)
at sqlline.Commands.connect(Commands.java:1064)
at sqlline.Commands.properties(Commands.java:948)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:809)
at sqlline.SqlLine.begin(SqlLine.java:686)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:291)

SQLLine fails to find JDBC driver if the Java classpath has URL escape characters, e.g. space (' ') in it

Please find the patch at #24

Build fails on Windows

Checkstyle reports many "file doesn't end with newline" errors.

Unit tests break because they expect only LF instead of CRLF.

For unit tests, replace CRLF with LF in the actual strings (i.e. before comparing them). Not sure what needs to be done for checkstyle. Maybe you have to use .gitattributes to force Git to create all files with LF only.

convert to maven build

sqlline command history is not working

I executed few commands on sqlline shell and when quit and re opened sqlline, the history is not saving.

We observed ~/.sqlline dir is created but not history file.

As a user do we need to take care of this? If can you suggest on how and where to take care of it?

Update manual

Sqlline's manual (generated from doc/manual.xml and visible here: http://www.hydromatic.net/sqlline/manual.html) is out of date. Make a pass and update it.

Among other things:

• The version number is incorrect.
• The version of jline is incorrect.
• It does not mention the '-d' command line option

Combine with Beeline (Hive's fork of sqlline)

There are two active forks of the original sqlline code base: this one, and Hive's fork, beeline. I have proposed in https://issues.apache.org/jira/browse/HIVE-6361 that we create a version of sqlline inside Hive but (unlike Beeline) not dependent on other Hive modules.

NumberFormatException while Sqlline start

Hello
looks like known issue here jline/jline2#281
IMHO jline version update will help to have it fixed

to reproduce just run sqlline

[serg@localhost csv]$ ./sqlline 
[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:580)
	at java.lang.Integer.valueOf(Integer.java:766)
	at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59)
	at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242)
	at jline.UnixTerminal.<init>(UnixTerminal.java:65)
	at jline.UnixTerminal.<init>(UnixTerminal.java:50)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at jline.TerminalFactory.getFlavor(TerminalFactory.java:211)
	at jline.TerminalFactory.create(TerminalFactory.java:102)
	at jline.TerminalFactory.get(TerminalFactory.java:186)
	at jline.TerminalFactory.get(TerminalFactory.java:192)
	at sqlline.SqlLineOpts.<init>(SqlLineOpts.java:45)
	at sqlline.SqlLine.<init>(SqlLine.java:54)
	at sqlline.SqlLine.start(SqlLine.java:372)
	at sqlline.SqlLine.main(SqlLine.java:265)

[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:580)
	at java.lang.Integer.valueOf(Integer.java:766)
	at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59)
	at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:242)
	at jline.UnixTerminal.<init>(UnixTerminal.java:65)
	at jline.UnixTerminal.<init>(UnixTerminal.java:50)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at jline.TerminalFactory.getFlavor(TerminalFactory.java:211)
	at jline.TerminalFactory.create(TerminalFactory.java:102)
	at jline.TerminalFactory.create(TerminalFactory.java:51)
	at sqlline.SqlLine.getConsoleReader(SqlLine.java:705)
	at sqlline.SqlLine.begin(SqlLine.java:639)
	at sqlline.SqlLine.start(SqlLine.java:373)
	at sqlline.SqlLine.main(SqlLine.java:265)

Environment

[serg@localhost ~]$ uname -a
Linux localhost.localdomain 4.16.7-300.fc28.x86_64 #1 SMP Wed May 2 20:09:13 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[serg@localhost ~]$ infocmp -V
ncurses 6.1.20180224
```

upgrade to jline 2

Older versions of sqlline not available on Maven Central...

I am using a tool called FlywayDB that indirectly uses sqlline via other dependencies. The latest code as of 3/14/2016 of Flyway requires sqlline 1.1.8, no longer available in Maven Central.

I can temporarily hack up the project, but wondered if there is a reason older versions of sqlline are not available in Maven Central and if there is a way to add back at least version 1.1.8 and keep the older ones moving forward?

Re-execute the previous query

Wish there was a way to re-run the previous query. Can be helpful with multi-line queries that cannot easily be cut-n-paste.
Even better - specify an offset, like running the query before the last, or before that, like:
!rerun (same as !rerun-1)
!rerun-2
!rerun-3

convert to maven directory structure

Honor DatabaseMetaData options to drive code completion

For example, case insensitivity could be driven by the Case methods (or just make it case insensitive by default). Also, it be nice to pair down what auto complete suggests based on how much SQL the driver supports. Maybe the supportsANSI92EntryLevelSQL, supportsANSI92IntermediateSQL, etc methods could control this.

I can also see using supportsCorrelatedSubqueries, supportsUnion, etc. to prune the choices for auto complete.

Sqlline version 1.1.8

Unable to resolve dependencies for sqlline 1.1.8. It has been removed from Maven central.

load driver

i've installed sqlline 1.0.2-5 in my ubuntu
i launch it, write !scan and it give me:
yes 2.3 org.hsqldb.jdbc.JDBCDriver
yes 2.3 org.hsqldb.jdbcDriver
in /usr/share/java i also have
ojdbc6-11.2.0.4.jar
ojdbc.jar
what i've to do, or where i've to put, my oracle driver?
thanks

sqlline bash script fails with MS SQL-Server

Upon !connect the output ends with:

SQLException : SQL state: 08S01 com.microsoft.sqlserver.jdbc.SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "RSA premaster secret error".

Setting the java.ext.dirs property seems to have caused the driver to not use SSL properly.

I changed my copy of the script to something like this:

BINPATH=$(dirname $0)
exec java -cp "$BINPATH/*" sqlline.SqlLine "$@"

This fixed the error.

Support JDK9; drop support for JDK 1.5

Support JDK9, also test using oraclejdk9 on Travis CI.

Clob data

How can I show a CLOB data? It appears to me like this: oracle.sql.CLOB@55e7e4bb

Thanks.

!manual command fails

!manual command fails, because manual.txt is not present.

Sqlline shell on windows/cygwin

As per Julian Hyde's suggestion in his blog I am logging this here.

I tried to run sqlline commands on windows using cygwin, although I manage to enter in the shell with no problems, no command seems to work.

"!record <logfile>" as an external command line option (e.g. "--log <logfile>")

First things first : thank you for maintaining that extraordinary tool ; I searched a lot to find this graal in a very heterogeneous db environment !

I mainly use (as of today) sqlline 1.3 as a ddl deployment tool but I still have to dance a little in order to wrap those committed scripts I have to deploy in order to get the "!record" statements in.

Provided the same functionality as a command line option, I could leave scripts unaltered
(In fact, I build an intermediary launcher as a sqlline commands script including !record and !run on the original ddl file)

Thanks

Flush output for each command when using !record option

The !record command is really useful when viewing large datasets with many columns as terminal isn't horizontally scroll-able.
But having to give !record to start and again !record to view the output for each command is a bit frustrating.
So I would like to suggest adding out.flush() for addline() and print() functions in OutputFile class.. . so that we can record outputs of multiple commands in the same file and view them after each command.

Ps. I found the tool extremely useful.. keep up the good work :)

Poor performance with drivers having slow DatabaseMetaData.getPrimaryKeys impl

Performance is rather poor for table output format when two conditions occur for the same result set.

1. The result set has a large number of columns.
2. The driver being used has a slow implementation of DatabaseMetaData.getPrimaryKeys.

For example testing has shown that for a query with ~100 columns using the HBase Phoenix thin driver the execution time can be cut from ~30 seconds to ~2 seconds by using CSV output format vs table output format. For example: select * from system.catalog;

This is due to how primary keys are detected. Currently the Rows implementation will make a metadata call for every column to determine it is a primary key for display purposes. I propose optimizing this such that a metadata call is only made for each unique table in the result set's columns.

No driver classes found

Should this work to talk to Oracle?

# wget http://download.oracle.com/otn/utilities_drivers/jdbc/11204/ojdbc6.jar
# java -cp $PWD -jar sqlline-1.2.0-jar-with-dependencies.jar
sqlline version ???
sqlline> !scan
scan complete in 281ms
No driver classes found
Compliant Compliant Version Compliant Compliant Version Driver Class
sqlline>