Problems with building oracle version on Windows about usql

[kenshaw]

This is a problem with cygwin terminals. It works fine on Powershell / cmd.exe at the moment.

You can test if the actual binary is working, by using the -c command line option:

$ usql.exe pg://booktest:booktest@localhost/booktest -c 'select * from books;'

Please note this needs the terminating semicolon still. I'm trying to get everything working properly on all shells on Windows, but there are clearly some problems.

(I hope I don't sound stupid commenting on this -- I don't really use windows much)

[kenshaw]

Oh sorry -- I misread the above. It works without Oracle but not with it? This is a new problem then.

[kenshaw]

Is there any reason you want to include oracle support in your build? The issue is that you can't legally re-distribute the Oracle binaries for instantclient. Also, while Go builds static executables for Go code, I believe the way cgo builds against the instantclient libs, they are dynamically loaded -- meaning, the person downloading your package using choco would be broken, since you can't redistribute the Oracle instantclient libs. While fine for someone working exclusively with Oracle databases, this obviously limits all the other users.

Also, note, if you're going to distribute a binary for it, PLEASE include adodb support out of the box! You also need to pass the -tags adodb to enable building adodb support. However, I might make this included by default on Windows (haven't done enough testing with it to see what the effects are).

Anyways, cool! I'm glad you're doing this; I haven't done development on Windows now for almost 20 years, so I'm out of my league. I didn't even know about chocolatey until just now.

[kenshaw]

I'm in the middle of installing Oracle on Windows for testing purposes. I hadn't tested with it, as I don't use Oracle from Windows, ever (I use it constantly from Linux/OSX, however). I'll try to figure out what the issue(s) are and fix. Apologies for not having tested Oracle better on Windows.

[majkinetor]

Is there any reason you want to include oracle support in your build?

Yes, I need to work with oracle databases too, among others.

The issue is that you can't legally re-distribute the Oracle binaries for instantclient. Also, while Go builds static executables for Go code, I believe the way cgo builds against the instantclient libs, they are dynamically loaded -- meaning, the person downloading your package using choco would be broken, since you can't redistribute the Oracle instantclient libs

The person can install libs. For example, dbeaver, universal db tool , uses maven to download all the drivers for 100 databases it supports, but it can't for oracle where it redirects you to their site. This is a problem for public distribution but not for enterprise and home usage where I can host my own private choco feed.

However, I might make this included by default on Windows (haven't done enough testing with it to see what the effects are).

I would prefer to create a choco package that distributes officially built exe. This way its more transparent and how almost all other packages work.

Anyways, cool! I'm glad you're doing this;

I wanted good general cli db tool to be honest, so I gave it a quick go :) This one sounds like a one.

Apologies for not having tested Oracle better on Windows.

No need for apologizing, I don't expect 100% everything from fos project. Thx for looking into it.

[kenshaw]

I think its a great idea to distribute choco/nuget/whatever packages for every platform. In fact, I'm planning on submitting to Debian eventually, as well, as soon as the documentation is better and it has something like ~80% of the support for psql stuff out of the box.

With choco though at least, I would recommend packaging 2 versions -- just a plain usql package without Oracle support, and usql-oracle with oracle support. At least then, those who don't use Oracle can still make use of the binary package.

[majkinetor]

Yeah, that sounds about right. Since choco packages can have parameters it could also be a single one. I have no problem maintaining choco package in the future. Bot will look at your releases and push a package to gallery as soon as you publish.

[kenshaw]

BTW -- definitely determined that the OCI lib is dynamically linked on Windows:

$ ./usql.exe or://booktest:[email protected]/xe.oracle.docker
C:/msys64/home/user/src/go/src/github.com/knq/usql/usql.exe: error while loading shared libraries: OCI.dll: cannot open shared object file: No such file or directory

For the record, I have not registered OCI.dll yet with Windows; I have simply extracted the instantclient sdk to C:\Oracle64Instant\, and have used this oci8.pc for building:

oracle_prefix=C:\Oracle64Instant\sdk

version=12.1

libdir="${oracle_prefix}\lib\msvc"
includedir="${oracle_prefix}\include"

glib_genmarshal=glib-genmarshal
gobject_query=gobject-query
glib_mkenums=glib-mkenums

Name: oci8
Description: Oracle database engine
Version: ${version}
Libs: -L${libdir} -loci
Libs.private:
Cflags: -I${includedir}

I should be able to determine soon what the root cause(s) are.

[kenshaw]

Question, have you been able to get an app on Windows working with Oracle using the rana/ora driver? I've registered all the DLLs, but am now getting this error when running:

usql.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory

Not sure if this is a problem with the rana/ora driver, or I don't have the instantclient libs setup correctly.

[kenshaw]

OK, got it working. I needed to install the MSVC runtime for 2010 "Microsoft Visual C++ 2010 Service Pack 1 Redistributable Package MFC Security Update", available here.

Oracle works fine from Windows (at least on cmd.exe; still have the separate issue of it not working properly under cygwin terminals, but I believe that's an issue with the configuration for the readline package I'm using, and not specifically a problem with usql directly).

A note about building: I noticed that when I built the executable the first time, and did not have the Oracle instantclient libs setup properly, the link step for Oracle failed silently, and did not cause go build to halt properly. Since the ora driver is imported via an anonymous import in Go, I'm not sure if this is the expected behavior or a bug/issue with Go on Windows. As such, you might want to try building like this:

$ cd $GOPATH/src/github.com/knq/usql
$ go build -a -v -x -tags oracle

And scroll through the output. When pkg-config is not properly configured, the build will succeed, even though there is this cgo error:

cd C:\msys64\home\user\src\go\src\gopkg.in\rana\ora.v4
pkg-config --cflags oci8
# pkg-config --cflags oci8
Package oci8 was not found in the pkg-config search path.
Perhaps you should add the directory containing `oci8.pc'
to the PKG_CONFIG_PATH environment variable
No package 'oci8' found
pkg-config: exit status 1

Note that this won't build an executable on Windows or on Linux, however on Windows this will not remove the pre-existing binary usql.exe, meaning that the binary that continues to exist in that path will be stale. You might then expect, as I did, that the usql.exe was built correctly.

For the record, when pkg-config fails with a bad path, you'll get an error like this:

# gopkg.in/rana/ora.v4
..\..\..\gopkg.in\rana\ora.v4\bndBfile.go:9:17: fatal error: oci.h: No such file or directory
 #include <oci.h>
                 ^
compilation terminated.

For both of these errors, on Windows at least, Go did not remove the pre-existing usql.exe output file. Comparatively, on Linux/OSX, the output file gets removed.

As such, I'm going to guess that the problem you encountered could be a misconfigured path/non-registered DLLs, etc. I'm going to close this issue for now, as when pkg-config is configured properly, and the accompanying OCI.dll is registered with Windows, and you have the VC++ 2010 redistributable installed, it should work.

(and this is why I don't think it's going to be very successful trying to provide a choco package with Oracle support precompiled)

[kenshaw]

Oh, one more note -- running usql.exe when properly built with Oracle support requires more than just registering the OCI.dll. I'm not sure exactly which DLLs need to be registered/present, as the instantclient Windows libs doesn't seem to have any kind of documentation, or "installer" with it. As such, I just copied all of the instantclient files into the same working directory as the output dir for usql.exe.

[kenshaw]

The documentation from the node-oracledb project is likely helpful:

https://github.com/oracle/node-oracledb/blob/master/INSTALL.md#winbins

[kenshaw]

The minimum DLLs needed:

$ ls
oci.dll  oraociicus12.dll  oraons.dll  usql.exe

$ ./usql.exe or://booktest:[email protected]/xe.oracle.docker -c 'select * from books;'
  book_id | author_id | isbn |        title         | year |            available             |     tags       
+---------+-----------+------+----------------------+------+----------------------------------+---------------+
        1 |         1 |    1 | my book title        | 2016 | 2017-03-08T17:31:03.282917+07:00 | empty          
        2 |         1 |    2 | changed second title | 2016 | 2017-03-08T17:31:03.282917+07:00 | cool disastor  
        3 |         1 |    3 | the third book       | 2001 | 2017-03-08T17:31:03.282917+07:00 | cool           
(3 rows)

$

Possibly if more advanced SQL is executed, there might be other errors.

[kenshaw]

Strongloop, "an IBM Company", has made instantclient-basiclite-windows.x64-12.1.0.2.0.zip available via a public Git repository. I'm assuming they have all the appropriate licensing agreements with Oracle. It can be downloaded directly, here: https://github.com/strongloop/loopback-oracle-builder/raw/master/deps/oracle/Windows/x64/instantclient-basiclite-windows.x64-12.1.0.2.0.zip

(I checked the md5sum of the files)

May I suggest if you do package this for Windows, that during installation a script gets run to download that zip, extract the relevant DLLs, and put them in the same installation directory as usql.exe? I don't think this would violate any licensing issues with Oracle!

[majkinetor]

May I suggest if you do package this for Windows, that during installation a script gets run to download that zip, extract the relevant DLLs, and put them in the same installation directory as usql.exe?

Sounds good. Could you add your successful build of usql.exe with oracle support among windows releases and I can probably make a package today.