r-dbi / rpostgres

A DBI-compliant interface to PostgreSQL

RPostgres

RPostgres is an DBI-compliant interface to the postgres database. It's a ground-up rewrite using C++ and cpp11. Compared to RPostgreSQL, it:

• Has full support for parameterised queries via dbSendQuery(), and dbBind().

• Automatically cleans up open connections and result sets, ensuring that you don't need to worry about leaking connections or memory.

• Is a little faster, saving ~5 ms per query. (For reference, it takes around 5ms to retrieve a 1000 x 25 result set from a local database, so this is decent speed up for smaller queries.)

• A simplified build process that relies on system libpq.

Installation

# Install the latest RPostgres release from CRAN:
install.packages("RPostgres")

# Or the the development version from GitHub:
# install.packages("remotes")
remotes::install_github("r-dbi/RPostgres")

Discussions associated with DBI and related database packages take place on R-SIG-DB. The website Databases using R describes the tools and best practices in this ecosystem.

Basic usage

library(DBI)
# Connect to the default postgres database
con <- dbConnect(RPostgres::Postgres())

dbListTables(con)
dbWriteTable(con, "mtcars", mtcars)
dbListTables(con)

dbListFields(con, "mtcars")
dbReadTable(con, "mtcars")

# You can fetch all results:
res <- dbSendQuery(con, "SELECT * FROM mtcars WHERE cyl = 4")
dbFetch(res)
dbClearResult(res)

# Or a chunk at a time
res <- dbSendQuery(con, "SELECT * FROM mtcars WHERE cyl = 4")
while(!dbHasCompleted(res)){
  chunk <- dbFetch(res, n = 5)
  print(nrow(chunk))
}
# Clear the result
dbClearResult(res)

# Disconnect from the database
dbDisconnect(con)

Connecting to a specific Postgres instance

library(DBI)
# Connect to a specific postgres database i.e. Heroku
con <- dbConnect(RPostgres::Postgres(),dbname = 'DATABASE_NAME', 
                 host = 'HOST', # i.e. 'ec2-54-83-201-96.compute-1.amazonaws.com'
                 port = 5432, # or any other port specified by your DBA
                 user = 'USERNAME',
                 password = 'PASSWORD')

Design notes

The original DBI design imagined that each package could instantiate X drivers, with each driver having Y connections and each connection having Z results. This turns out to be too general: a driver has no real state, for PostgreSQL each connection can only have one result set. In the RPostgres package there's only one class on the C side: a connection, which optionally contains a result set. On the R side, the driver class is just a dummy class with no contents (used only for dispatch), and both the connection and result objects point to the same external pointer.

---

Please note that the 'RPostgres' project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

rpostgres's Issues

dbDataType fails for Connection

https://github.com/rstats-db/RPostgres/blob/1fcef9a543f62ffbe8e9dc1e58fc9425c871261d/R/connection.R#L89

Why resort to SQLite here?

No show() method for driver class

Should perhaps show at least version of pgsql library (if applicable). Same for other backends.

The default method could perhaps show package name and version.

dbConnect complains that Rcpp >= 0.11.4.2 is required although Rcpp 0.11.4.5 is loaded

con <- dbConnect(RPostgres::Postgres())
Error in is(object, Cl) :
error in evaluating the argument 'drv' in selecting a method for function 'dbConnect': Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) :
namespace ‘Rcpp’ 0.11.4 is already loaded, but >= 0.11.4.2 is required

sessionInfo() shows that I have Rcpp_0.11.4.5 loaded via namespace

Think about schemas

for dbWriteTable, dbReadTable, dbListColumns and others.

Perhaps a new class "SQLIdentifier"? See also tidyverse/dplyr#244.

Add temporary table support

Permission denied openssl

@hadley

Not sure if this is the right please but here I go

I tried to installed Rpostgres in my other Mac and got this error:

Error: Permission denied - /usr/local/etc/openssl
Warning: Bottle installation failed: building from source.
==> Downloading https://www.openssl.org/source/openssl-1.0.1g.tar.gz
==> perl ./Configure --prefix=/usr/local/Cellar/openssl/1.0.1g --openssldir=/usr/local/etc/openssl zlib-dynamic shared enable-cms darwin64-x86_64-cc enable-ec_nistp_64_gcc_128
==> make depend
==> make
==> make test
==> make install MANDIR=/usr/local/Cellar/openssl/1.0.1g/share/man MANSUFFIX=ssl
i2d_SSL_SESSION.3ssl => d2i_SSL_SESSION.3ssl
installing man3/ssl.3ssl
SSL.3ssl => ssl.3ssl
Cannot create directory /usr/local/etc/openssl: Permission denied
make: *** [install_sw] Error 13
Error: Homebrew doesn't know what compiler versions ship with your version
of Xcode (6.1.1). Please `brew update` and if that doesn't help, file
an issue with the output of `brew --config`:
  https://github.com/Homebrew/homebrew/issues

Note that we only track stable, released versions of Xcode.

After updating brew and accepting Terms & Conditions for my Xcode Version 6.1.1 (6A2008a) (stupid automatic updates) the issue persisted when the solution struck me like a lighting bolt (a bit dramatic) _it was a permission issue_ (in my defence, the homebrew message is really misleading!)

so I fixed it with the following:

sudo chown -R $USER:admin /usr/local

Fingers crossed future generations will read this and won't ask the same question over and over. ;-)

Using new encoding methods for parameterised queries

And re-benchmark inserts

Convert dates to R native types

Right now dates types are exported as character. One thing I miss from the RPostgreSQL package is that dates would be converted transparently to R types.

Savepoint support

Need to determine if transaction currently running so dbBegin() can start transaction if needed.

Should depend on DBI package

DBItest: Getting started : ...1.

1. Failure (at /home/muelleki/git/R/DBItest/R/test_getting_started.R#34): DBItest: package_dependencies 
"DBI" %in% pkg_depends isn't true

RPostgres appears to require Homebrew installation on OS X

I attempted to give RPostgres a try, but after attempting to install all sorts of things from Homebrew, the installation gave up and suggested I uninstall MacPorts. Most of the things RPostgres was attempting to install were already present on my system.

Demand for RPostgres may increase in near future due to uncertain status of RPostgreSQL, a package that simply works for me (though compilation has become patchy with recent versions of R on OS X).

Multiple retries

Would be useful to have wrapper which would try query n times with exponential backoff. Probably needs classed Rcpp::stop() to implement optimally.

Data type for timestamps

should be POSIXct.

Test name: "data_timestamp".

Rename to RPostgres

Why it's different?

• Ground up rewrite in C++/Rcpp
• Somewhat faster
• Parameterised queries
• DBI 0.4 support
• Automatic connection cleanup
• Always has SSL
• Better built process

Package install failure on Centos 6.6

I cannot seem to get RPostgres to install on CentOS. I was able to get it on Ubuntu by making sure I had up to date version of postgres and libpqxx on the machine but it doesn't seem to work on CentOS

* installing *source* package ‘RPostgres’ ...
** libs
gcc -m64 -std=gnu99 -I/usr/include/R -DNDEBUG  -I/usr/local/include -I"/usr/lib64/R/library/Rcpp/include" -I"/usr/lib64/R/library/BH/include"   -fpic  -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic  -c RPostgres-init.c -o RPostgres-init.o
g++ -m64 -I/usr/include/R -DNDEBUG  -I/usr/local/include -I"/usr/lib64/R/library/Rcpp/include" -I"/usr/lib64/R/library/BH/include"   -fpic  -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic  -c RcppExports.cpp -o RcppExports.o
In file included from RPostgres_types.h:4,
                 from RcppExports.cpp:4:
PqConnection.h: In constructor ‘PqConnection::PqConnection(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >)’:
PqConnection.h:35: error: ‘PQconnectdbParams’ was not declared in this scope
PqConnection.h: In member function ‘SEXPREC* PqConnection::escapeString(std::string)’:
PqConnection.h:175: error: ‘PQescapeLiteral’ was not declared in this scope
PqConnection.h: In member function ‘SEXPREC* PqConnection::escapeIdentifier(std::string)’:
PqConnection.h:186: error: ‘PQescapeIdentifier’ was not declared in this scope
In file included from PqResult.h:9,
                 from RPostgres_types.h:5,
                 from RcppExports.cpp:4:
PqRow.h: In member function ‘bool PqRow::hasData()’:
PqRow.h:45: error: ‘PGRES_SINGLE_TUPLE’ was not declared in this scope
In file included from RPostgres_types.h:5,
                 from RcppExports.cpp:4:
PqResult.h: In member function ‘void PqResult::bind()’:
PqResult.h:82: error: ‘PQsetSingleRowMode’ was not declared in this scope
PqResult.h: In member function ‘void PqResult::bind(Rcpp::ListOf<Rcpp::Vector<16, Rcpp::PreserveStorage> >)’:
PqResult.h:105: error: ‘PQsetSingleRowMode’ was not declared in this scope
make: *** [RcppExports.o] Error 1
ERROR: compilation failed for package ‘RPostgres’
* removing ‘/usr/lib64/R/library/RPostgres’
Error: Command failed (1)

yum list libpq*
Installed Packages
libpqxx.x86_64                                                            1:4.0.1-1.rhel6                                                  @pgdg94
libpqxx-devel.x86_64                                                      1:4.0.1-1.rhel6                                                  @pgdg94
Available Packages
libpqxx-debuginfo.x86_64                                                  1:4.0.1-1.rhel6                                                  pgdg94

dbHasCompleted always seems to return TRUE

DBItest: "fetch_*".

Import DBI

Instead of requiring it

Warning with untyped NULL

SELECT NULL as a leads to the following warning:

In result_create(conn@ptr, statement) :
Unknown field type (705) in column a

Related: #30

Test name: "data_null".

Script for updating libpq

Error: Lost connection to database

Hi,
I am getting the following error in RPostgres package after I ran

dbGetQuery(con,query)

Error: Lost connection to database

dbDisconnect(con)
[1] TRUE
Warning message:
In connection_release(conn@ptr) : There is a result object still in use.
The connection will be automatically released when it is closed

Then I ran dbClearResults(con) and I get the following error:

Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘dbClearResult’ for signature ‘"PqConnection"’

Can you please tell me how to clear the resultset?

dbListConnections isn't defined for Postgres drivers...

dbListConnections(Postgres())
Error in (function (classes, fdef, mtable)  :  unable to find an inherited method for function ‘dbListConnections’ for signature ‘"PqDriver"’

Do we not need to disconnect PostgreSQL connections?

Password prompt and SSL

I am not sure whether RPostgres is the right place to implement such a thing but I've been looking / working around this for a while when working with databases and R:

• Would it possible to have something like .rs.askForPassword on the R console?
• Can communication between the database and R be encrypted?

CI for other operating systems

• Windows
• OS X

dbWriteTable() fails to raise error

if append = TRUE and table doesn't exist yet.

DBItest: "append_table_error".

64-bit integers are silently converted to 32-bit integers

Most significant 32 bits are reset to zero.

Example: 1e10 = 0x2540BE400 -> 0x540BE400 = 1410065408.

Identical behavior (!) for RSQLite and RMySQL.

Test name: "data_64_bit".

Add Postgres9.4 JSONB type.

Postgres9.4 added the JSONB type (field type: 3802). DBI/RPostgres seem to return the correct result, but there's an extra warning about the unknown field type.

postgres slightly official reference for field types

create table test (data JSONB);
INSERT INTO test(data) values ('{"name":"mike"}');

con <- dbConnect(RPostgres::Postgres(), dbname='testjson')
dbListTables(con)
dbGetQuery(con, "select * from test;")
dbDisconnect(con)

              data
1 {"name": "mike"}
Warning message:
In result_create(conn@ptr, statement) :
  Unknown field type (3802) in column data

dbWriteTable() with logical column fails

1. Error: DBItest: roundtrip_logical -------------------------------------------
ERROR:  invalid input syntax for integer: "true"
CONTEXT:  COPY test, line 1, column a: "true"

DBItest: "roundtrip_logical".

unrecognized PostgreSQL field type uuid (id:2950)

Connecting to a table with field type uuid generates a warning.

Table creation

CREATE TABLE prd_tv_sandbox.r_gender
(
respid UUID,
wave_id UUID,
gender INTEGER,
PRIMARY KEY (respid,wave_id)
);

# The R:

library (RPostgres)
library ( dplyr)
library ( DBI )


con_sb <- src_postgres(dbname = "the_db",   user = "[id]", password = "***")

gen_db <-  tbl ( con_sb, "r_gender" )

Warning messages:
1: In postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver warning: (unrecognized PostgreSQL field type uuid (id:2950) in column 0)
2: In postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver warning: (unrecognized PostgreSQL field type uuid (id:2950) in column 1)

Int64 should be saved as character

Postgres doesn't have BLOB data type

https://github.com/rstats-db/RPostgres/blob/1fcef9a543f62ffbe8e9dc1e58fc9425c871261d/R/connection.R#L102

DBItest: blob- and data-type related tests.

Package install failed on ubuntu

Tried to install this on an ubuntu virtual machine:

devtools::install_github("RcppCore/Rcpp") # done successfully
devtools::install_github("rstats-db/DBI") # done successfully
devtools::install_github("rstats-db/SQL") # done successfully
devtools::install_github("rstats-db/RPostgres") # attempting...

Package installation failed with the following message:

In file included from PqResult.h:9:0,
                 from RPostgres_types.h:5,
                 from RcppExports.cpp:4:
PqRow.h: In member function ‘bool PqRow::hasData()’:
PqRow.h:45:24: error: ‘PGRES_SINGLE_TUPLE’ was not declared in this scope
In file included from RPostgres_types.h:5:0,
                 from RcppExports.cpp:4:
PqResult.h: In member function ‘void PqResult::bind()’:
PqResult.h:82:43: error: ‘PQsetSingleRowMode’ was not declared in this scope
PqResult.h: In member function ‘void PqResult::bind(Rcpp::ListOf<Rcpp::Vector<16> >)’:
PqResult.h:104:43: error: ‘PQsetSingleRowMode’ was not declared in this scope
make: *** [RcppExports.o] Error 1
ERROR: compilation failed for package ‘RPostgres’
* removing ‘/home/vagrant/.rpackages/RPostgres’
Error: Command failed (1)

My system info:
Linux precise64 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86 _64 x86_64 x86_64 GNU/Linux

R running in RStudio

[1] "R version 3.1.2 (2014-10-31)"

Install failed on WIndows

When trying to install the package from CRAN, I get the following error:

• installing source package 'RPostgres' ...
  Warning: running command 'sh ./configure.win' had status 127
  ERROR: configuration failed for package 'RPostgres'
• removing 'C:/Users/Brian/Documents/R/win-library/3.1/RPostgres'
  Error: Command failed (1)

Use travis

(Once @jeroenooms has improved the build scripts)

Standardise function names

• prefix
• camel vs snake case

RPostgres changes number of digits after decimal point

Writing this table to database

Browse[1]> head(df)
  c8d6446849216a7aff6c2256ccc96e379 c5a4250bce927ff4ef8ed59093bd878ec cdf69869ec785c7d1b8068b6e287eb32c id
1                                 1            0.56083285459317266941           -1.49170532658561438843  1
2                                 2            0.79464945197105407715           -0.39146524387521386412  2
3                                 3            0.58925445331260561943            0.95434671620049915663  3
4                                 4            0.89141587377525866032           -1.00872132539390979566  4
5                                 5            0.58816246618516743183            1.05403643324296703021  5
6                                 6            0.49537089723162353039            1.20770345842324178243  6

DBI::dbWriteTable(dbconn, tblname, df, append = FALSE, row.names = FALSE)

Yet when I read it from DB (double-checked using psql), I see this

head(df_db)
  id        x         y
1  1 0.560833 -1.491710
2  2 0.794649 -0.391465
3  3 0.589254  0.954347
4  4 0.891416 -1.008720
5  5 0.588162  1.054040
6  6 0.495371  1.207700

Digits got truncated! Any ideas how to solve this?

Support for Postgres data type JSON

It's rather a suggestion than an issue, but I'd love to see json support in an RPostgres package.
The current CRAN solution throws an ugly warning when reading json from postgres:
" RS-DBI driver warning: (unrecognized PostgreSQL field type json ". Probably, that's rather an DBI problem, but I would be great if this new R Postgre package would resolve the issue somehow. Particularly because the current workaround to avoid the warning is to cast json to text in SQL, but might be more difficult (slower) with the new jsonb stuff.

Speed up dbExistsTable()

Listing all tables just for checking if one is existing is just slow: https://github.com/rstats-db/RPostgres/blob/1fcef9a543f62ffbe8e9dc1e58fc9425c871261d/R/tables.R#L123.

Issue building with shiny

I've had a few mix and match problems relating to a shiny app I've been trying to deploy but this error message seems unrelated to other package version issues I've seen:

[2015-05-19T06:18:27.190917678+0000] Building R package: RPostgres (0.1)
/mnt/packages/build /mnt
Warning in untar2(tarfile, files, list, exdir, restore_times) :
  skipping pax global extended headers
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘RPostgres’ ...
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/usr/include/postgresql  -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/BH/include"   -fpic  -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c RPostgres-init.c -o RPostgres-init.o
g++ -I/usr/share/R/include -DNDEBUG -I/usr/include/postgresql  -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/BH/include"   -fpic  -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -I/usr/share/R/include -DNDEBUG -I/usr/include/postgresql  -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/BH/include"   -fpic  -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c connection.cpp -o connection.o
g++ -I/usr/share/R/include -DNDEBUG -I/usr/include/postgresql  -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/BH/include"   -fpic  -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c encode.cpp -o encode.o
g++ -I/usr/share/R/include -DNDEBUG -I/usr/include/postgresql  -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/BH/include"   -fpic  -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g  -c encrypt.cpp -o encrypt.o
In file included from PqResult.h:9:0,
                 from RPostgres_types.h:5,
                 from RcppExports.cpp:4:
PqRow.h: In member function ‘bool PqRow::hasData()’:
PqRow.h:45:24: error: ‘PGRES_SINGLE_TUPLE’ was not declared in this scope
In file included from RPostgres_types.h:5:0,
                 from RcppExports.cpp:4:
PqResult.h: In member function ‘void PqResult::bind()’:
PqResult.h:82:43: error: ‘PQsetSingleRowMode’ was not declared in this scope
PqResult.h: In member function ‘void PqResult::bind(Rcpp::ListOf<Rcpp::Vector<16> >)’:
PqResult.h:105:43: error: ‘PQsetSingleRowMode’ was not declared in this scope
make: *** [RcppExports.o] Error 1
make: *** Waiting for unfinished jobs....

In file included from PqResult.h:9:0,
 ################################### End Log################################### 
Error: Unhandled Exception: Child Task 34903624 failed: Error building image: Error building RPostgres (0.1). Build exited with non-zero status: 1

I understand that shiny detects the currently installed package version and thus will attempt a build from the github repo. Any ideas on what might be causing the issue? I love the ability of this package to connect to a remote postgres database on Heroku.

> sessionInfo()
R version 3.1.3 (2015-03-09)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] knitr_1.9

loaded via a namespace (and not attached):
 [1] bitops_1.0-6     digest_0.6.8     evaluate_0.5.5   formatR_1.0      htmltools_0.2.6  httpuv_1.3.2     mime_0.3         R6_2.0.1        
 [9] Rcpp_0.11.6.1    RCurl_1.95-4.5   RJSONIO_1.3-0    shiny_0.11.1     shinyapps_0.3.63 stringr_0.6.2    tools_3.1.3      xtable_1.7-4  

RPostgres is unable to find table without manually giving the sql call

I'm trying to use dplyr with a Postgres back end.

I can connect to the database (note that this connection info is public), which works fine:

library("dplyr")
library("RPostgreSQL")
mydb = src_postgres(dbname = "srdb", host="nautilus-vm.mathstat.dal.ca", user = "srdbuser", password =  "srd6us3r!", port = 5432)

I can now see a list of the tables:

mydb

shows:

src:  postgres 8.4.10 [[email protected]:5432/srdb]
tbls: area, assessment, assessmethod, assessor, biometrics, bioparams, brptots,
  fishbasesaupcodes, geometry_columns, lmerefs, lmes, lmetostocks, management,
  mostrecent, recorder, referencedoc, reference_point_units_view,
  reference_point_values_view, risfields, risfieldvalues, spatial_ref_sys,
  stock, taxonomy, timeseries, timeseries_units_view, timeseries_values_view,
  tsmetrics, tsrelative_explicit_view

Great. Now I try to access a table:

tbl(mydb, "stock")

and I get the error:

Error in postgresqlExecStatement(conn, statement, ...) : 
  RS-DBI driver: (could not Retrieve the result : ERROR:  relation "stock" does not exist
LINE 1: SELECT * FROM "stock" WHERE 0=1
                      ^

No idea what that 0=1 business is for either. Manually constructing the SQL query seems to work though:

tbl(mydb, sql("SELECT * FROM srdb.stock"))

returns the table as expected:

Source: postgres 8.4.10 [[email protected]:5432/srdb]
From: <derived table> [?? x 8]

      stockid    tsn scientificname   commonname                 areaid
1        COD1 164712   Gadus morhua Atlantic cod   multinational-NAFO-1
2    COD2J3KL 164712   Gadus morhua Atlantic cod       Canada-DFO-2J3KL
3  COD2J3KLIS 164712   Gadus morhua Atlantic cod       Canada-DFO-2J3KL
4       COD3M 164712   Gadus morhua Atlantic cod  multinational-NAFO-3M
5      COD3NO 164712   Gadus morhua Atlantic cod multinational-NAFO-3NO
6      COD3Ps 164712   Gadus morhua Atlantic cod         Canada-DFO-3Ps
7   COD3Pn4RS 164712   Gadus morhua Atlantic cod      Canada-DFO-3Pn4RS
8       COD4T 164712   Gadus morhua Atlantic cod          Canada-DFO-4T
9     COD4VsW 164712   Gadus morhua Atlantic cod        Canada-DFO-4VsW
10    COD4TVn 164712   Gadus morhua Atlantic cod          Canada-DFO-4T
..        ...    ...            ...          ...                    ...
Variables not shown: stocklong (chr), inmyersdb (int), myersstockid (chr)

Apologies if this is just user error of me not understanding the syntax, or if this should have been filed as an issue in DBI or dplyr instead. Thanks

Why devtools::install_github("rstats-db/RPostgres") fails on Mac?

> devtools::install_github("rstats-db/RPostgres")
Downloading github repo rstats-db/RPostgres@master
Installing RPostgres
'/Library/Frameworks/R.framework/Resources/bin/R' --vanilla CMD INSTALL  \
  '/private/var/folders/gz/ys2sqm9j76x30y312y_15blm0000gn/T/RtmpQt0iXi/devtoolsb91d5912beaf/rstats-db-RPostgres-524876f'  \
  --library='/Library/Frameworks/R.framework/Versions/3.1/Resources/library'  \
  --install-tests 

* installing *source* package ‘RPostgres’ ...
/usr/local/bin/brew: /usr/local/Library/brew.rb: /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby: bad interpreter: No such file or directory
/usr/local/bin/brew: line 23: /usr/local/Library/brew.rb: Undefined error: 0
Brewing libpq in ...
./configure: line 35: /bin/brew: No such file or directory
File /Cellar/postgresql/*/include/libpq-fe.h not found.
Homebrew of libgit2 failed :(
ERROR: configuration failed for package ‘RPostgres’
* removing ‘/Library/Frameworks/R.framework/Versions/3.1/Resources/library/RPostgres’
Error: Command failed (1)

dbWriteTable() doesn't work with date or timestamp columns

ERROR:  invalid input syntax for type real: "2015-11-08"
CONTEXT:  COPY test, line 1, column a: "2015-11-08"

POSIXlt?

DBItest: "roundtrip_date", "roundtrip_timestamp".

Include views in listed tables?

This is also in tidyverse/dplyr#1007 (because it comes up when using the tbl function). I'm mentioning it here because it seems like RPostgres is going to replace RPostgreSQL eventually and might be able to fix the problem.

RPostgreSQL doesn't return views when using dbListTables, which is unexpected for some users. RPostgres uses the same query:

select tablename from pg_tables where schemaname !='information_schema' and schemaname !='pg_catalog';

I'm not sure whether RPostgres should just return views along with the tables, or whether there should be a separate dbListViews function. But I feel like there should be some way to get at them without writing a raw query.

The following gets both views and "real" tables:

SELECT table_name FROM INFORMATION_SCHEMA.tables WHERE table_schema = ANY (current_schemas(false));

Fails to compile in CentOS7 (fails to find existing libpq-fe.h)

I try to compile "rstats-db/RPostgres", but it fails to find libpq-fe.h:

> devtools::install_github("rstats-db/RPostgres")
Downloading github repo rstats-db/RPostgres@master
Installing RPostgres
'/usr/lib64/R/bin/R' --vanilla CMD INSTALL  \
  '/tmp/Rtmpyy794p/devtoolsc997c6bf0ba/rstats-db-RPostgres-1f09a16'  \
  --library='/home/esa/R/x86_64-redhat-linux-gnu-library/3.1' --install-tests 

* installing *source* package ‘RPostgres’ ...
File libpq-fe.h not found. Please install libpq, e.g: postgresql-devel (rpm) or libpq-dev (deb)
ERROR: configuration failed for package ‘RPostgres’
* removing ‘/home/esa/R/x86_64-redhat-linux-gnu-library/3.1/RPostgres’
Error: Command failed (1)

In CentOS7, in order to get Postgresql 9.3 I use pgdg repo, and devel package includes are not in a standard place:

[root@localhost bin]# rpm -qil postgresql93-devel | grep libpq-fe
/usr/pgsql-9.3/include/libpq-fe.h

And that is also, what is reported by pg_config:

[root@localhost bin]# pg_config
BINDIR = /usr/pgsql-9.3/bin
DOCDIR = /usr/share/doc/pgsql
HTMLDIR = /usr/share/doc/pgsql
INCLUDEDIR = /usr/pgsql-9.3/include
[...]

dbGetException() not implemented

DBItest: "get_exception".

dbIsValid() not implemented

DBItest: "is_valid".

Date literals

are returned as character.

Test name: "data_date".

no dbListFields method for PqConnection?

I was going through your 'Basic Usage' example when I got this error message in reply to dbListFields(con, "mtcars"):

Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘dbListFields’ for signature ‘"PqConnection", "character"’

Tried the same steps with a RPostgreSQL connection object with no problems. Actually:

> showMethods(dbListFields)
Function: dbListFields (package DBI)
conn="PostgreSQLConnection", name="character"
conn="PostgreSQLResult", name="missing"

whereas:

> showMethods(dbListTables)
Function: dbListTables (package DBI)
conn="PostgreSQLConnection"
conn="PqConnection"

R v3.1.3
RStudio v0.98.1103
DBI v0.3.1.9008
RPostgres v0.1

Can't bind NA values

1. Failure: DBItest: bind_null_positional_dollar -------------------------------
is.na(rows$a) isn't true

DBItest: "bind_null_positional_dollar".

UTF-8 tests

Full interface compliance

1. Failure: DBItest: compliance ------------------------------------------------
hasMethod("dbGetInfo", "PqDriver", driver_package) isn't true

2. Failure: DBItest: compliance ------------------------------------------------
hasMethod("dbListResults", "PqConnection", driver_package) isn't true

3. Failure: DBItest: compliance ------------------------------------------------
hasMethod("dbListFields", c("PqConnection", "character"), driver_package) isn't true

4. Failure: DBItest: compliance ------------------------------------------------
hasMethod("dbIsValid", "PqConnection", driver_package) isn't true

DBItest: "compliance".