When I have a datasource:

(pg/def-datasource lw-db
                   {:host     "localhost"
                    :port     5432
                    :database "andy"
                    :username "andy"
                    :pooled   true})

and I try to create a jar file with lein uberjar, I get a null pointer exception during the compile:

$ lein uberjar
Compiling lwchallenge.core
WARNING: update already refers to: #'clojure.core/update in namespace: taoclj.foundation, being replaced by: #'taoclj.foundation/update
java.lang.NullPointerException, compiling:(core.clj:24:1)
Exception in thread "main" java.lang.NullPointerException, compiling:(core.clj:24:1)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3657)
    at clojure.lang.Compiler$IfExpr.eval(Compiler.java:2697)
.....
Caused by: java.lang.NullPointerException
    at java.util.Hashtable.put(Hashtable.java:459)
    at com.zaxxer.hikari.HikariConfig.addDataSourceProperty(HikariConfig.java:294)
    at taoclj.foundation.datasources$create_pooled_datasource.invokeStatic(datasources.clj:27)
    at taoclj.foundation.datasources$create_pooled_datasource.invoke(datasources.clj:26)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3652)
    ... 36 more

Something going wrong at com.zaxxer.HikariConfig.

Lein is set up to use clojure 1.8.0 but the same problem occurs with clojure 1.7.0.

(defproject lwchallenge "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [org.clojure/data.xml "0.0.8"]
                 [org.taoclj/foundation "0.1.3"]
                 [clj-time "0.11.0"]
                 [org.slf4j/slf4j-log4j12 "1.7.1"]] ; <- I seem to need this when :pooled is true as well
  :main ^:skip-aot lwchallenge.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

Any idea what might be going on? I've not looked at Hikarai before.

I'm having an issue with the following code creating a Person record, even though the User record in the same transaction was not created:

(def post-confirmation-event
  "mock event from AWS Cognito"
  {"version" 1
   "region" "us-west-2"
   "userPoolId" "us-west-2_wiZEwc83y"
   "userName" "nick"
   "callerContext" {"awsSdkVersion" "aws-sdk-js-2.6.4"
                    "clientId" "3birc4sh4vmaoeudj50cd0vg8kd"}
   "triggerSource" "PostConfirmation_Confirmation"
   "request" {"userAttributes" {"sub" "0fcf454386-8cf6-40d6-b042-3e7b61a67f33"
                                "email_verified" "true"
                                "cognito:user_status" "what"
                                "phone_number_verified" true
                                "phone_number" "+123456789"
                                "email" "[email protected]"}}
   "response" {}})

(defn add-user-to-database [user]
  "adds the Cognito user to our database"
  (pg/trx-> db/vsa-db
            (pg/insert :people {:name (get user "userName")})
            (pg/insert
             :users
             (pg/with-rs
               []
               {:cognito-uuid  (get-in user ["request" "userAttributes" "sub"])
                :cognito-pool (get user "userPoolId")
                :email (get-in user ["request" "userAttributes" "email"])
                :username (get user "userName")
                :person-id (first rs)}))))

(add-user-to-database post-confirmation-event)

In the REPL it returns the primary key for the Person record, but an empty collection for the user record. Since I'm only creating one User record, is there another way to do this? I guess there's always SQL templating

I haven't quite wrapped my head around wrapping foundation database transactions with one of my own.

I'm trying to create a test fixture that looks like this:

(ns vsa-sql-management.test-helpers
  (:require [clojure.java.jdbc :as jdbc]
            [taoclj.foundation :as pg]
            [environ.core :refer [env]]))

(declare ^:dynamic *store*)

(def test-db-spec
  {:classname "org.postgresql.Driver"
   :subprotocol "postgresql"
   :subname "//127.0.0.1:5432/vsa_test"
   :user "postgres"})

(defn db-transaction-fixture [f]
  (jdbc/with-db-transaction [store test-db-spec]
    (jdbc/db-set-rollback-only! store)
    (binding [*store* store] (f))))

Ideally, I can be passing in the dynamic store variable into a pg/trx-> call so my tests know that the transaction should be rolled back. However, when I write a test like this:

(ns vsa-sql-management.seeders.vsa-questionnaire-template-test
  (:require [vsa-sql-management.seeders.vsa-questionnaire-template :as sut]
            [vsa-sql-management.test-helpers :refer [*store* db-transaction-fixture]]
            [taoclj.foundation :as pg]
            [clojure.test :as t]
            [environ.core :refer [env]]))

(t/use-fixtures :each db-transaction-fixture)

(t/deftest seed-test
  (t/testing "seeds database with questions"
    (pg/trx-> sut/vsa-db (pg/insert :people {:name "nick"}))
    (t/is (= 1 (pg/trx-> sut/vsa-db (pg/select :people {:name "nick"}))))))

I still see the created person record in my test database. I'm sure there's something simple that my stupid self is missing. Any help would be appreciated! I think if I figure out the proper place to pass the *store* variable to, I'll be all good.

When I try to execute a query that creates a database error (e.g. causing Postgres to print its own STDERR in its logs), it'd be nice to see the exception returned to be in the clj code instead of just nil (e.g. null constraints, foreign keys, etc.). If you like this idea, where should I look to implement it?

Hello,

Nice work. Have you considered a core.async interface for async connections to psql?

Hi, the following code (the same as in doc) will not work:

(defn create-products-table []
  (with-open [cnx (.getConnection db)]
  (execute cnx "CREATE TABLE products (id serial primary key not null, name varchar(255), description text);")))

I get the error

CompilerException java.lang.RuntimeException: Unable to resolve symbol: execute in this context, compiling:(project/models/product.clj:14:3)

I fixed this by:

  (:require [taoclj.foundation :as pg]
            [taoclj.foundation.execution :as execution]))

  ...

  (execution/execute cnx "CREATE TABLE products (id serial primary key not null, name varchar(255), description text);")))

Am I going wrong or I miss something in docs?

When a database column allows nulls, inserts/updates can not set the value to null because it uses the value of the parameter to determine the type. JDBC requires this type. If the value is null, the sql type can't be inferred.

When creating a def-query, you should be able to specify a file located in the resources directory of a project. This may already be functional but need to confirm and if not add this feature.

(If it is in fact impossible to use transactions, feel free to update the issue title, but I didn't want to assume too much.)

Description

The JDBC functions clojure.java.jdbc/with-db-transaction and clojure.java.jdbc/db-set-rollback-only! expect to operate on a "database connection" object compatible with the get-connection function. It appears that most people pass in the db spec they used to set up their normal connection, whether it's a DB URI, a hash with connection parameters, or whatever.

However, Foundation expects to use a javax.sql.DataSource-compatible object for every query. Since one of the allowed get-connection arguments is a {:datasource <ds>, ...} object, I tried doing this:

(clojure.java.jdbc/with-db-transaction [t-conn {:datasource db/conn}]
  (jdbc/db-set-rollback-only! t-conn)
  (pg/qry-> (:datasource t-conn) (db/alter-some-stuff! params)))

Where db/conn is the result of a def-datasource call. Although this syntax does alter the database, those changes persist.

This is not surprising: with-db-transaction is supposed to bind t-conn to a connection object. In this case, it binds a hash:

{:datasource
 #object[com.impossibl.postgres.jdbc.PGDataSource 0x5f9231ae "com.impossibl.postgres.jdbc.PGDataSource@5f9231ae"],
 :connection
 #object[com.impossibl.postgres.jdbc.PGConnectionImpl 0x129eb816 "com.impossibl.postgres.jdbc.PGConnectionImpl@129eb816"],
 :level 1,
 :rollback #<Atom@79e53c70: true>}

My theory is that the :connection part of that hash is the transaction connection, and if I were to operate on that, the changes would be rolled back. But Foundation expects to work on a PGDataSource, and it gets a new connection from it, one which does not have the transaction or rollback properties.

Am I right? And if so, is there a workaround for this situation, or just a better way of achieving my goal? I think you might have some unwelcome tension between Foundation's way of doing things and the bare-bones JDBC functions people may wish to apply.

For logging it'd be helpful if I could print out all the sequel templates (perhaps after YesSQL interpolation) so AWS Cloudwatch can pick up on them.

Placeholders in query files cause problems, and should be allowed.

Description

Something in Foundation's database code is emitting debug statements to the REPL.

Steps to Reproduce

Run a Foundation query in the REPL.

Expected Behavior

Query runs silently if there are no errors.

Actual Behavior

On first query:

(pg/qry-> conn (get-users {:limit 10 :offset 0}))
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/amacdougall/.m2/repository/ch/qos/logback/logb
ack-classic/1.1.2/logback-classic-1.1.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/amacdougall/.m2/repository/org/slf4j/slf4j-sim
ple/1.7.13/slf4j-simple-1.7.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
16:05:48.528 [nREPL-worker-1] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
16:05:48.534 [nREPL-worker-1] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.event
LoopThreads: 8
16:05:48.548 [nREPL-worker-1] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bu
ffer.address: available
16:05:48.548 [nREPL-worker-1] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Un
safe.theUnsafe: available
<much more output omitted>

On subsequent queries, debug statements related to thread pooling. My DB config has pooling: false, so this is probably just related to standard JDBC.

16:08:55.584 [threadDeathWatcher-2-3] DEBUG io.netty.buffer.PoolThreadCache - Freed 8 thread-local buffer(s) from thread: PG-JDBC EventLoop (1)

Developer Analysis

Debug logging levels should be configurable, and have a high threshold (log only on warning or errors, perhaps) by default.

Because they are so common.

I ​was able to use an insert query from a template file to write to my database, but when I tried a select query, I got this result:

=> NullPointerException   clojure.lang.Reflector.invokeNoArgInstanceMember (Reflector.java:301)

I tried this query:

(pg/qry-> conn (pg/select :users {}))

That query should simply evaluate to `SELECT * FROM users;`, right? I did get the same result when I used a template query:
```sql
SELECT * FROM users LIMIT :limit OFFSET :offset;

(pg/qry-> conn (get-users {:offset 0 :limit 30}))

Any idea what I'm doing wrong? For completeness, here's my DB spec:

; it actually comes from a profiles.clj thing, but these are the values:
(def datasource
  {:host "localhost"
    :port 5432
    :database "<known-good value>"
    :username "<known-good value>"
    :password "<<known-good value>"
    :pooled false}}

And I'm just doing:

(pg/def-datasource conn datasource)